ACD Direct Logo

ACD Direct

Senior Data Engineer

Posted 3 Days Ago
Remote
Hiring Remotely in CAN
Senior level
Remote
Hiring Remotely in CAN
Senior level
Design, build, and maintain scalable Databricks-based ETL/ELT pipelines and data models optimized for AI/ML. Manage orchestration, data quality, governance, and CI/CD (ADO → GitHub) while collaborating with cross-functional teams in Agile delivery.
The summary above was generated by AI

About ACD

For more than a century, AIA Contract Documents (ACD) has supported architecture, engineering, and construction professionals by delivering a shared industry standard to align parties on a project.

What began in 1888 with the development of standardized construction contracts has evolved into a comprehensive suite of contract tools and foundational workflows that not only shape how the industry works today, but uniquely position ACD to help firms navigate construction’s growing complexity.

In a world driven by scale, fragmentation, and AI-generated decisions that prioritize speed over a clear understanding of risk, ACD serves as a trusted anchor, ensuring project participants can reduce disputes and negotiations, while achieving faster alignment and more predictable outcomes for the future.

AIA Contract Documents is seeking a Senior Data Engineer to support the buildout of data infrastructure powering upcoming AI initiatives. This role will play a critical part in designing and structuring data systems that enable scalable, high-quality inputs for AI and machine learning models.

The ideal candidate brings deep experience in modern data engineering practices, strong familiarity with Databricks, and the ability to operate independently in a fast-evolving environment with limited oversight.

Key Responsibilities

  • Design and implement scalable data models optimized for AI/ML use cases, ensuring data is structured for effective model training and inference

  • Architect and manage data pipelines using Databricks, including orchestration, job scheduling, and workflow optimization

  • Develop and maintain robust ETL/ELT processes to support data ingestion, transformation, and delivery across systems

  • Leverage Databricks Asset Bundles to manage deployment of data assets (pipelines, jobs, notebooks, and files) across environments

  • Collaborate with cross-functional teams to align data architecture with AI initiative requirements and business objectives

  • Ensure data quality, integrity, and governance standards are met across all pipelines and datasets

  • Contribute to CI/CD practices using Azure DevOps (ADO), with a future transition to GitHub-based workflows

  • Participate in Agile development processes, including sprint planning, stand-ups, and iterative delivery

  • Other duties as assigned

Qualifications

  • 5+ years of experience in Data Engineering or related field

  • Strong expertise in Databricks, including pipeline development, orchestration, and data architecture

  • Experience designing data models to support AI/ML applications

  • Proficiency in Python (PySpark) and SQL

  • Hands-on experience with Azure-based data environments

  • Experience with CI/CD tools such as Azure DevOps (ADO)

  • Ability to work independently with minimal oversight and ramp quickly in a fast-paced environment

  • Experience with Databricks Asset Bundles for deployment and environment management, preferred

  • Familiarity with transitioning CI/CD workflows to GitHub, preferred

  • Experience working in Agile development environments, preferred

  • Exposure to AI/ML workflows and data requirements for model development, preferred

What Success Looks Like

  • Efficient, scalable data pipelines supporting AI initiatives

  • Well-structured, high-quality datasets optimized for machine learning

  • Streamlined deployment and orchestration of Databricks assets across environments

  • Strong collaboration with stakeholders to deliver data solutions aligned with business needs

Similar Jobs

3 Days Ago
Easy Apply
Remote or Hybrid
Canada
Easy Apply
Senior level
Senior level
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Design, build, and maintain SparkSQL/PySpark data pipelines in the central data lake to ingest IoT, product, and unstructured data (video/audio). Produce reliable computed tables for analytics, model training, and dashboards; integrate external datasets; ensure high data quality and uptime; and collaborate with Data Science, ML, and cross-functional teams.
Top Skills: AirflowAWSAzureDagsterData LakeDatabricksDelta LakeETLGCPGitGitPrefectPysparkPythonRest ApisSparkSparksqlSQL
4 Days Ago
Easy Apply
Remote or Hybrid
Canada
Easy Apply
Senior level
Senior level
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Design, build, and operate scalable data ingestion, replication, and lakehouse infrastructure to move petabytes of data into a Delta Lake on S3. Improve reliability, observability, security, and developer experience for Spark/Databricks processing. Develop internal libraries and tooling (Go/Python), collaborate with cross-functional teams, and help shape long-term data platform and AI-ready infrastructure.
Top Skills: SparkAws DynamodbAws KinesisAws LambdaAws RdsAws S3Aws SqsGoJavaPythonScala
4 Days Ago
Remote or Hybrid
Senior level
Senior level
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Design and build data infrastructure, pipelines, and integration solutions using cloud and big-data tools. Develop data lakes/warehouses, ensure data quality and security, apply data modeling and DAGs, use Databricks, Airflow, and Hadoop, and collaborate with clients to deliver actionable insights.
Top Skills: Apache AirflowApache HadoopAWSAzure Data FactoryDagsData LakeData WarehouseDatabricksDimensional ModelingAzure

What you need to know about the Calgary Tech Scene

Employees can spend up to one-third of their life at work, so choosing the right company is crucial, not just for the job itself but for the company culture as well. While startups often offer dynamic culture and growth opportunities, large corporations provide benefits like career development and networking, especially appealing to recent graduates. Fortunately, Calgary stands out as a hub for both, recognized as one of Startup Genome's Top 100 Emerging Ecosystems, while also playing host to a number of multinational enterprises. In Calgary, job seekers can find a wide range of opportunities.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account