Senior Data Engineer (Azure / PySpark)

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines using PySpark and Azure Data Services (Data Factory, Data Lake, Blob Storage, Synapse, Databricks).
  

• Work closely with clients to gather requirements, present solutions, and ensure deliverables align with business goals.
  

• Optimize data workflows for performance, scalability, and cost-efficiency.
  

• Ensure data quality through validation checks, monitoring, and automated alerts.
  

• Collaborate with cross-functional teams, including data scientists, BI developers, and business analysts.
  

• Create and maintain comprehensive documentation for data pipelines, architecture, and processes.
  

• Support pre-sales or solutioning discussions by providing technical insights.
  

Requirements

Required Qualifications

• Bachelor’s degree in Computer Science, Data Science, or a related technical field.
  

• 5+ years of professional experience in data engineering.
  

• Strong hands-on expertise in PySpark for large-scale data processing.
  

• Proficiency in Python for scripting, automation, and data manipulation.
  

• Proven experience with Azure cloud ecosystem: Data Factory, Data Lake, Blob Storage, Synapse, Databricks.
  

• Solid understanding of ETL design patterns, data modeling, and data warehousing.
  

• Experience interacting directly with clients or business stakeholders to define and deliver data solutions.
  

Preferred Skills

• CI/CD experience with Azure DevOps or similar tools.
  

• Familiarity with Delta Lake, data governance, and security best practices.
  

• Exposure to performance tuning for Spark jobs and Azure pipelines.
  

Soft Skills

• Excellent communication skills for both technical and non-technical audiences.
  

• Strong problem-solving and analytical abilities.
  

• Ability to work independently and collaboratively in a fast-paced environment.
  

Benefits