Data Engineer

We’re looking for a Data Engineer! Reach out if you’re interested and feel free to refer friends/colleagues!

Type of Employment: Contract
Title: Data Engineer
Term: 6-month Contract
Location: Remote for now – Toronto
Job ID number: C1395

Brief description of duties:

• 5+ years of IT experience in large scale technology architecture, operations, and design-related disciplines
• Data Architect/Data Engineer, successfully designing and developing enterprise big data solutions
• SQL experience with large data sets
• Concrete programming and scripting language experience (e.g. Python, NodeJS) with the understanding of SDLC principles
• Working experience using AWS and Azure platform services including AWS Glue, AWS Athena, AWS SQS, AWS Kinesis, AWS Redshift, AWS EMR, AWS S3, and Azure Data Lake.
• Experience with big data technologies including Spark, Hadoop, Hive, HDFS, Python
• Experience with metadata management, data stewardship, data quality, IT governance, enterprise data models, and enterprise frameworks
• Solid understanding and experience with relational databases (MySQL, SQL Server, PostgreSQL, etc.)
• Experience working with REST APIs
• Business information requirement analysis experience
• Solid understanding of project management principles

MUST haves:

• Data warehouse and data lake architecture principles
• Data integration and streaming integration concepts and architecture
• Data ETL pipelines for large-scale transformations
• Database design for read-only access
• Data warehousing technologies such as OLAP
• Data transformation, conversion, and partitioning
• Business intelligence, query, and reporting tools
• Techniques for detecting and addressing data quality issues
• Data formats for loading and unloading of data
• Middleware (API gateways)
• Data modeling for data warehousing/data lakes (Denormalized – Star & Snowflake Schemas, Data Vault)
• Parallel execution
• Indexing and statistics strategies
• Writing complex, highly-optimized SQL and Python to interact with large data sets as needed for data engineering and processing
• Semantic modeling and modeling for parallel access
• Infrastructure provisioning using IaC principles and CI/CD pipelines
• Ability to communicate highly technical and complex security concepts effectively across all levels of the organization (both IT and business)
• Team player with excellent communication skills