Senior Data Engineer responsible for designing and validating data solutions, consolidating disparate data sources, documenting functional specifications, reviewing data models, supporting production deployments, investigating data quality issues, and collaborating with stakeholders and technology leads to ensure data completeness and lineage.
Responsibilities
- Big Data - Spark, Hive, Java, CDP, 6 Years + with 3 Years of Dev experience on Big Data
- Analyze data requirements and identify disparate data sources required for consolidation and distribution.
- Document functional specifications and coordinate delivery of the same with technology team.
- Review logical and conceptual data models in alignment with business requirements.
- Work with the stakeholders to understand and gather requirements and produce business specifications
- Validate solution implementations and ensure they meet business and functional requirements.
- Provide production deployment support and investigate data quality issues.
- Work with various technology leads to ensure the gaps in the data completeness or accuracy are bridged.
Qualifications
- Subject matter expertise in financial industry - wholesale loans /lending business OR Capital Markets or Finance or Risk Reporting
- Strong hands on experience with database and SQL is required.
- Excellent documentation, analytical skills to produce process flow diagrams, business modelling, and functional design.
- Proficiency in MS Office (Word, Excel, Visio, PowerPoint) with extensive experience using Excel for data analysis.
- Experience with Data tracing/ Lineage efforts
- Knowledge of logical and physical data model
Similar Jobs
Agency • Information Technology
Senior Java Big Data Developer with 8+ years building applications on Cloudera/Hadoop ecosystems using Spark and MapReduce. Responsible for designing performant Java (1.8+) solutions, data analysis of large datasets, unit testing (JUnit), and collaborating with/managing global technology teams. Preferred skills include Unix/Python scripting and familiarity with Hive, Impala, YARN, and Kafka.
Top Skills:
SparkClouderaHadoopHiveImpalaJava 1.8JunitKafkaMapreduceNoSQLPythonUnix ShellYarn
Agency • Information Technology
Design, develop, test, and deploy high-performance Apache Spark/Scala data processing applications and ETL pipelines on Cloudera (CDH). Optimize Spark and platform performance, build data pipelines using HDFS, Hive, Impala, HBase, and Kafka, ensure data integrity and security, collaborate with data scientists and analysts, and implement version control and CI/CD for deployments.
Top Skills:
SparkCi/CdCloudera (Cdh)FlumeGitGitlabHbaseHdfsHiveImpalaJenkinsKafkaNifiOoziePostgresScalaSpark SqlSqoop
Agency • Information Technology
Build and maintain responsive Angular UI components, collaborate with UX and backend teams, implement REST APIs and Python backend scripts, integrate basic ML models into data workflows, and participate in the full SDLC within Agile teams while ensuring code quality and performance.
Top Skills:
AngularCi/CdCSS3DjangoFastapiFlaskGitHTML5JavaScriptJSONMachine LearningPythonRestTypescript
What you need to know about the Calgary Tech Scene
Employees can spend up to one-third of their life at work, so choosing the right company is crucial, not just for the job itself but for the company culture as well. While startups often offer dynamic culture and growth opportunities, large corporations provide benefits like career development and networking, especially appealing to recent graduates. Fortunately, Calgary stands out as a hub for both, recognized as one of Startup Genome's Top 100 Emerging Ecosystems, while also playing host to a number of multinational enterprises. In Calgary, job seekers can find a wide range of opportunities.
