The Senior Data Engineer will design, develop, and maintain data pipelines and infrastructure, focusing on optimizing data for AI/ML models. Responsibilities include leading the creation of data warehouses, ensuring data quality, and collaborating with cross-functional teams to meet data requirements for machine learning applications.
The Senior Data Engineer is a hands-on leader responsible for designing, developing, and maintaining data pipelines and infrastructure at Cleo. This role involves setting the strategy for data systems, collaborating closely with cross-functional teams, and ensuring the scalability, reliability, and efficiency of data solutions. The Senior Data Engineer will focus on data infrastructure needs for AI/ML models, overseeing the creation of a data warehouse and associated systems from scratch, and ensuring data is properly transformed and optimized for machine learning and artificial intelligence applications. This role is integral to building the processes that support data transformation, data structures, metadata management, data quality controls, and workload management.
What You Will Be Doing
- Lead the Design and Build of Data Pipelines: Develop and maintain scalable, reliable, and efficient data pipelines that collect, process, and store large datasets. Ensure these systems are optimized for AI/ML model training and inference.
- Set Data Infrastructure Strategy: Define and execute the strategy for building and maintaining data warehouses, data lakes, and other data storage systems that support both operational and analytical needs. Ensure systems are optimized for AI/ML model workflows.
- Hands-On Data Transformation for AI/ML Models: Design and implement data transformation processes, including feature engineering, data preprocessing, and data augmentation, to ensure data is in the right format for machine learning models.
- Build Data Structures and Metadata Management: Build and manage the data structures, metadata repositories, and related systems that support data transformation, ensuring the organization has well-documented, accurate, and accessible data for AI/ML workflows.
- Data Quality Controls and Risk Management: Establish and implement data quality controls to ensure that data used in machine learning models is clean, consistent, and accurate. Identify and raise risks at all stages of the data engineering process, including data ingestion, transformation, and storage.
- Collaborate with Cross-Functional Teams: Work closely with data scientists, ML engineers, product managers, and business leaders to understand data requirements and ensure data systems meet the needs of AI/ML initiatives. Provide hands-on leadership to guide teams in transforming data for model training.
- ETL Development and Optimization for AI/ML: Lead the development and optimization of ETL (Extract, Transform, Load) processes for ML/AI data. Focus on efficiently moving and transforming large datasets, ensuring data quality and readiness for model training and deployment.
- Optimize Data for Model Training and Inference: Ensure data pipelines are designed to support the needs of AI/ML model training, such as handling large volumes of data, managing data quality, and enabling fast model iteration.
- Data Governance for AI/ML: Define and implement data governance practices to ensure the secure, compliant, and ethical use of data in AI/ML workflows.
- Stay Current: Keep up with emerging trends in data engineering, particularly those related to AI/ML model development, and implement best practices to optimize data systems for machine learning.
Your Qualifications
- Experience: 5-7+ years of experience in data engineering, with a focus on transforming data for AI/ML models and optimizing data systems to support machine learning and artificial intelligence workflows.
- Hands-On Expertise: Proven experience in hands-on data transformation and building data pipelines for AI/ML, including data preprocessing, feature engineering, and model-specific data handling.
- Leadership: Experience leading or mentoring data engineering teams, providing hands-on guidance for AI/ML-related projects and collaborating with cross-functional teams.
- Cloud and Big Data: Strong experience with cloud platforms and big data technologies, particularly in the context of AI/ML model development.
A few things we have to offer:
- Competitive compensation
- Great Healthcare + Dental + Vision
- Flexible PTO
- Culture of support, encouraging Life-Work balance
- 401k match
- FSA and HSA options
- Employee Assistance Program
- Paid Parental Leave
- Representing a company with 4,000+ clients and a 99% retention rate
- Accelerated title and salary growth potential
- A fun and energetic work environment that makes you excited to go to work every day
Top Skills
AI
Data Engineering
Ml
Similar Jobs at Cleo
The Data Scientist will analyze and interpret complex supply chain data, develop predictive models, and collaborate with teams to optimize supply chains and drive business growth. Responsibilities include processing large datasets, applying statistical models and machine learning algorithms, and communicating data insights to stakeholders.
What you need to know about the Calgary Tech Scene
Employees can spend up to one-third of their life at work, so choosing the right company is crucial, not just for the job itself but for the company culture as well. While startups often offer dynamic culture and growth opportunities, large corporations provide benefits like career development and networking, especially appealing to recent graduates. Fortunately, Calgary stands out as a hub for both, recognized as one of Startup Genome's Top 100 Emerging Ecosystems, while also playing host to a number of multinational enterprises. In Calgary, job seekers can find a wide range of opportunities.
