Original job description
We are looking for a Data Engineer to join a team in Kitchener, Ontario and build reliable, scalable data solutions that support analytics and business decision-making. This position is well suited to someone who brings strong experience in modern data warehousing, cloud-based data platforms, and robust engineering practices. The successful candidate will work across batch and real-time pipelines, contribute to data architecture design, and help strengthen delivery within an Agile environment.
Responsibilities:
• Design, build, and maintain data pipelines that support both scheduled and near real-time processing across enterprise data platforms.
• Develop and optimize data solutions within Databricks using Python, PySpark, and Spark SQL to deliver dependable and efficient data workflows.
• Create and enhance data warehouse structures using dimensional modelling principles and layered architecture approaches such as Medallion frameworks.
• Integrate and manage cloud-based data services in Azure, including orchestration, secure secret handling, and storage components to support end-to-end data movement.
• Apply sound software engineering practices by writing maintainable code, implementing tests, and contributing to CI/CD processes within a DevOps setting.
• Collaborate with cross-functional teams in Agile delivery models to define requirements, refine technical approaches, and deliver data products iteratively.
• Analyse data structures and transformation needs to choose appropriate modelling and manipulation techniques for different business use cases.
• Support data platform improvements and related technical changes, including enhancements tied to evolving systems, integrations, or operational processes where required.
• Bachelor’s or Master’s degree in Informatics, Software Engineering, or a related field, or an equivalent combination of education and practical experience.
• At least 4 years of experience working in data warehousing, including exposure to batch and streaming data processing and modern architectural patterns.
• Minimum 4 years of hands-on experience with Databricks, dimensional data modelling, Python, and Spark-based development.
• Demonstrated experience with Azure services such as Azure Data Factory, Key Vault, and Azure Storage in support of data engineering solutions.
• Strong knowledge of coding standards, automated testing approaches, and CI/CD practices within a DevOps-oriented environment.
• Advanced understanding of data modelling, data transformation, and the strengths and limitations of different data structures across varied use cases.
• Experience working within Agile frameworks such as Scrum, Spotify model, or similar team-based delivery methodologies.
• Preferred assets include experience with Oracle databases or applications, familiarity with languages such as Bash, Java, Scala, or R, and an interest in machine learning or AI concepts.
