Data Engineer

HRBrain

CONTRACT
Canada
Posted 2 weeks ago

We’re looking for a Data Engineer! Reach out if you’re interested and feel free to refer friends/colleagues!

 

Type of Employment: Contract
Title: Data Engineer
Term: 6-month Contract
Location: Remote for now – Toronto
Job ID number: C1395

 

Brief description of duties:

  • 5+ years of IT experience in large scale technology architecture, operations, and design-related disciplines
  • Data Architect/Data Engineer, successfully designing and developing enterprise big data solutions
  • SQL experience with large data sets
  • Concrete programming and scripting language experience (e.g. Python, NodeJS) with the understanding of SDLC principles
  • Working experience using AWS and Azure platform services including AWS Glue, AWS Athena, AWS SQS, AWS Kinesis, AWS Redshift, AWS EMR, AWS S3, and Azure Data Lake.
  • Experience with big data technologies including Spark, Hadoop, Hive, HDFS, Python
  • Experience with metadata management, data stewardship, data quality, IT governance, enterprise data models, and enterprise frameworks
  • Solid understanding and experience with relational databases (MySQL, SQL Server, PostgreSQL, etc.)
  • Experience working with REST APIs
  • Business information requirement analysis experience
  • Solid understanding of project management principles

 

MUST haves:

 

  • Data warehouse and data lake architecture principles
  • Data integration and streaming integration concepts and architecture
  • Data ETL pipelines for large-scale transformations
  • Database design for read-only access
  • Data warehousing technologies such as OLAP
  • Data transformation, conversion, and partitioning
  • Business intelligence, query, and reporting tools
  • Techniques for detecting and addressing data quality issues
  • Data formats for loading and unloading of data
  • Middleware (API gateways)
  • Data modeling for data warehousing/data lakes (Denormalized – Star & Snowflake Schemas, Data Vault)
  • Parallel execution
  • Indexing and statistics strategies
  • Writing complex, highly-optimized SQL and Python to interact with large data sets as needed for data engineering and processing
  • Semantic modeling and modeling for parallel access
  • Infrastructure provisioning using IaC principles and CI/CD pipelines
  • Ability to communicate highly technical and complex security concepts effectively across all levels of the organization (both IT and business)
  • Team player with excellent communication skills

 

Job Features

Job CategoryData Engineer
Data Architect/Data Engineer3+
SQL experience with large data sets3+
Programming and scripting language3+

Apply Online