Data Engineer

SE/ Senior Data Engineer (with SQL, Python, Airflow, Bash)

About the Role
We are seeking a highly skilled and experienced Senior/Lead Data Engineer to join our growing Data Engineering Team. In this critical role, you will design, architect, and develop cutting-edge multi-tenant SaaS data solutions hosted on Azure Cloud. Your work will focus on delivering robust, scalable, and high-performance data pipelines and integrations that support our enterprise provider and payer data ecosystem.

This role is ideal for someone with deep experience in ETL/ELT processes, data warehousing principles, and real-time and batch data integrations. As a senior member of the team, you will also be expected to mentor and guide junior engineers, help define best practices, and contribute to the overall data strategy.

We are specifically looking for someone with strong hands-on experience in SQL, Python, and ideally Airflow and Bash scripting.

Key Responsibilities

• Architect and implement scalable data integration and data pipeline solutions using Azure cloud services.
• Design, develop, and maintain ETL/ELT processes, including data extraction, transformation, loading, and quality checks using tools like SQL, Python, and Airflow.
• Build and automate data workflows and orchestration pipelines; knowledge of Airflow or equivalent tools is a plus.
• Write and maintain Bash scripts for automating system tasks and managing data jobs.
• Collaborate with business and technical stakeholders to understand data requirements and translate them into technical solutions.
• Develop and manage data flows, data mappings, and data quality & validation rules across multiple tenants and systems.
• Implement best practices for data modeling, metadata management, and data governance.
• Configure, maintain, and monitor integration jobs to ensure high availability and performance.
• Lead code reviews, mentor data engineers, and help shape engineering culture and standards.
• Stay current with emerging technologies and recommend tools or processes to improve the team's effectiveness.

Required Qualifications

• Bachelors or Masters degree in Computer Science, Information Systems, or related field.
• 5+ years of experience in data engineering, with a strong focus on Azure-based solutions.
• Proficiency in SQL and Python for data processing and pipeline development.
• Experience in developing and orchestrating pipelines using Airflow (preferred) and writing automation scripts using Bash.
• Proven experience in designing and implementing real-time and batch data integrations.
• Hands-on experience with Azure Data Factory, Azure Data Lake, Azure Synapse, Databricks, or similar technologies.
• Strong understanding of data warehousing principles, ETL/ELT methodologies, and data pipeline architecture.
• Familiarity with data quality, metadata management, and data validation frameworks.
• Strong problem-solving skills and the ability to communicate complex technical concepts clearly.

Preferred Qualifications

• Experience with multi-tenant SaaS data solutions.
• Background in healthcare data, especially provider and payer ecosystems.
• Familiarity with DevOps practices, CI/CD pipelines, and version control systems (e.g., Git).
• Experience mentoring and coaching other engineers in technical and architectural decision-making.