Data Engineer

Job Description

Data Engineer – Databricks

Initial 7-month Contract Role

Remote Working

£600 - £650, Inside IR35

We’re partnering with a Global IT Partner who are looking for a Data Engineer to design and develop scalable data pipelines in Azure Databricks.

As a Data Engineer, you will be responsible for:

• Lead end-to-end migration of data lakes to Azure with a core focus on Databricks
• Design and implement data pipelines in Azure Databricks, optimizing for performance, cost, and scalability
• Refactor and modernize existing data models to integrate with Power BI
• Coordinate phased migrations to ensure zero disruption to live operations
• Replace legacy tools by consolidating workflows within Databricks and native Azure services

Requirements:

• 10+ years in cloud data engineering, with a strong focus on building scalable data pipelines
• Expertise in Azure Databricks, including building and managing ETL pipelines using PySpark or Scala
• Solid understanding of Apache Spark, Delta Lake, and distributed data processing concepts
• Hands-on experience with Azure Data Lake Storage Gen2, Azure Data Factory, and Azure Synapse Analytics
• Proficiency in SQL and Python for data transformation and pipeline logic

Please note: you will be working a US time zone.  

Please contract olivia@ecsgrp.co.uk / 01676545407 to discuss in more detail.