As a Sr. Data Platform Engineer, you will enhance Stability's data platform using AWS and GCP, architect data pipelines, develop data strategies, and mentor team members. Responsibilities include managing infrastructure-as-code, analyzing large datasets, and making technical decisions about data platforms.
<Location: Remote Canada>
About the role:
We are looking for a Data Platform Engineer to enhance and expand Stability’s data platform using AWS and GCP services. This begins with understanding our needs, selecting the appropriate tech stack and tools, gaining buy-in from stakeholders, and choosing the best long-term solution. This role will act as the technical thought leader for our data platform and help hire and mentor other team members.
Responsibilities:
- Architect, implement, and monitor data pipelines, as well as conduct knowledge-sharing sessions with peers to ensure effective data use
- Work with team members and leaders to further develop our data strategy
- Develop and maintain a data catalog so that we have a full understanding of the data and events within our platform
- Maintain infrastructure-as-code (AWS CDK) provisioning, spin up new and/or revised data platform resources and environments to enhance data collection and processing from our applications and data sources
- Act as the primary technical decision-maker for the data platform and help hire and support analytical data personnel (e.g. Data Scientist, Data Analyst, BI Developer, etc)
- Analyze large amounts of data, both structured and unstructured, and create dashboards, queries, and visualizations
- Technical experience in the following areas (with examples):
- Containerized compute: Kubernetes (EKS, GKS), ECS, Docker
- Data ingestion: Airbyte, Mage AI, AWS Glue
- Pipeline orchestration: Airflow, Spark
- Data manipulation: Pandas, Numpy, Polars,
- Data lake: S3, Iceberg, Delta Lake
- Data warehouse: BigQuery, Redshift
- Database: PostgreSQL
- Data visualization: Looker, Superset/Preset
- Data quality (optional): Great Expectations, Monte Carlo
- Data catalog (optional): Amundsen, AWS Glue Catalog
- Event streaming (optional): Kafka, Flink, Redis, Pub/Sub
- Infrastructure-as-code: AWS CDK, Terraform
- CI/CD: GitHub Actions
Qualifications:
- Must have 7+ years of hands-on experience working with data platforms
- Experience being involved in the design and implementation of at least one data platform.
- Experience in streaming and/or batch analytics.
- Experience with scalable distributed data processing, management, and visualization tools.
- Experience using business intelligence tools and data frameworks.
- Strong working knowledge of Python for both provisioning cloud infrastructure and working with data.
- We’re a remote-first company, and we need people who can do their best work from home, collaborate effectively, work asynchronously, and are proactive about reaching out to peers to get things done and/or get themselves unstuck.
- Enthusiasm, a “can do” attitude, and optimism when faced with difficult challenges.
Top Skills
Airbyte
Airflow
AWS
Aws Cdk
Aws Glue
Aws Glue Catalog
BigQuery
Delta Lake
Docker
Ecs
Eks
Flink
GCP
Github Actions
Gks
Iceberg
Kafka
Kubernetes
Looker
Mage Ai
Numpy
Pandas
Polars
Postgres
Pub/Sub
Redis
Redshift
S3
Spark
Superset
Terraform
Similar Jobs
Blockchain • Fintech • Mobile • Payments • Software • Financial Services
As a Senior Business Intelligence Engineer at Cash App, you will work with cross-functional teams to optimize data models and build data pipelines. Your role involves managing key datasets, promoting best practices in data design, and enabling self-service data access for others.
Top Skills:
AirflowBigquery)PrefectPythonSnowflakeSql (Mysql
eCommerce • Fashion • Retail
The Senior Cloud Platform Developer will design and manage cloud-based data infrastructure and deployment pipelines on AWS, focusing on data engineering and machine learning engineering. Responsibilities include building scalable data pipelines, developing CI/CD pipelines, and collaborating with domain teams to create reliable data products and ML pipelines.
eCommerce • Fashion • Retail
As a Senior Cloud Platform Developer - Data/ML, you will design and manage cloud-based data infrastructure, develop scalable data and ML pipelines using AWS, and enable domain teams to deliver impactful solutions independently. Your role includes building CI/CD pipelines and collaborating with teams to create domain-specific data products aligned with Data Mesh principles.
What you need to know about the Calgary Tech Scene
Employees can spend up to one-third of their life at work, so choosing the right company is crucial, not just for the job itself but for the company culture as well. While startups often offer dynamic culture and growth opportunities, large corporations provide benefits like career development and networking, especially appealing to recent graduates. Fortunately, Calgary stands out as a hub for both, recognized as one of Startup Genome's Top 100 Emerging Ecosystems, while also playing host to a number of multinational enterprises. In Calgary, job seekers can find a wide range of opportunities.
