Senior Data Engineer l AWS Glue & Kafka l
VE3
Full-time
On-site
Pune City, Maharashtra, India
Senior Jobs
Title: Senior Data Engineer l AWS Glue & Kafka l<\/b>
<\/div>
<\/div>
<\/div>
<\/div>
About the role:<\/b>
<\/div>
<\/div>
<\/div>
<\/div>
We’re seeking a hands -on Senior Data Engineer (~10 years’ experience) to deliver production data pipelines on AWS. You’ll design and build streaming (Kafka) and batch pipelines using Glue/EMR (PySpark), implement data contracts and quality gates, and set up CI/CD and observability. You’ve shipped real systems, coached teams, and you document as you go.
<\/div><\/span>
Requirements<\/h3>
<\/div><\/span>
Requirements<\/h3>What you’ll do:<\/b>
<\/div>
<\/div>• Architect and deliver lake/lakehouse data flows on AWS (S3 + Glue + Glue ETL/EMR).
<\/div>• Build Kafka consumers/producers, manage schema evolution, resilience, and DLQs.
<\/div>• Implement PySpark transformations, CDC merges, partitioning and optimization.
<\/div>• Add quality/observability (tests, monitoring, alerting, lineage basics).
<\/div>• Harden security (IAM least privilege, KMS, private networking).
<\/div>• Create runbooks, diagrams, and handover materials.
<\/div>
<\/div>What you’ll bring:<\/b>
<\/div>
<\/div>• Deep AWS (Glue, RDS. S3, EMR, IAM/KMS, CloudWatch).
<\/div>• Strong Kafka (MSK/Confluent, schema registry, consumer group tuning).
<\/div>• Python/PySpark in production with tests and CI/CD.
<\/div>• Data modeling (bronze/silver/gold, CDC, SCD2) and data contracts.
<\/div>• IaC (Terraform/CDK) and cost/performance tuning experience.
<\/div>• Clear communication and stakeholder engagement.
<\/div>
<\/div><\/span>
Benefits<\/h3>- Work on cutting -edge technologies and impactful projects.
<\/span><\/li>- Opportunities for career growth and development.
<\/span><\/li>- Collaborative and inclusive work environment.
<\/span><\/li>- Competitive salary and benefits package.
<\/span><\/li><\/ul>
<\/div><\/span>
<\/div>
<\/div>
• Architect and deliver lake/lakehouse data flows on AWS (S3 + Glue + Glue ETL/EMR).
<\/div>
<\/div>
• Build Kafka consumers/producers, manage schema evolution, resilience, and DLQs.
<\/div>
<\/div>
• Implement PySpark transformations, CDC merges, partitioning and optimization.
<\/div>
<\/div>
• Add quality/observability (tests, monitoring, alerting, lineage basics).
<\/div>
<\/div>
• Harden security (IAM least privilege, KMS, private networking).
<\/div>
<\/div>
• Create runbooks, diagrams, and handover materials.
<\/div>
<\/div>
<\/div>
<\/div>
What you’ll bring:<\/b>
<\/div>
<\/div>
<\/div>
<\/div>
• Deep AWS (Glue, RDS. S3, EMR, IAM/KMS, CloudWatch).
<\/div>
<\/div>
• Strong Kafka (MSK/Confluent, schema registry, consumer group tuning).
<\/div>
<\/div>
• Python/PySpark in production with tests and CI/CD.
<\/div>
<\/div>
• Data modeling (bronze/silver/gold, CDC, SCD2) and data contracts.
<\/div>
<\/div>
• IaC (Terraform/CDK) and cost/performance tuning experience.
<\/div>
<\/div>
• Clear communication and stakeholder engagement.
<\/div>
<\/div><\/span>
Benefits<\/h3>
<\/div>
<\/div><\/span>
Benefits<\/h3>- Work on cutting -edge technologies and impactful projects.
<\/span><\/li>- Opportunities for career growth and development.
<\/span><\/li>- Collaborative and inclusive work environment.
<\/span><\/li>- Competitive salary and benefits package.
<\/span><\/li><\/ul>
<\/div><\/span>
<\/span><\/li>
<\/span><\/li>
<\/span><\/li>
<\/span><\/li><\/ul>
<\/div><\/span>