Data Engineer (Python-heavy)

Job Title: Data Engineer (Python-heavy)
Location: Bengaluru, Karnataka ( On-site)
Experience: 3 – 7 years

About Us:
We are a bootstrapped AI startup working with multiple clients to build scalable, production-grade AI data pipelines. We value ownership, technical excellence, and resourcefulness in a fast-paced environment.

Role Overview:
We are looking for a strong Data Engineer with deep expertise in Python and end-to-end data pipeline development. You will own the design, development, and maintenance of streaming and batch data workflows that power AI applications.

Key Responsibilities:

Build and optimize scalable data pipelines using Python, Apache Spark, AWS Glue, and Kafka streaming

Design and implement parsing solutions for large structured and unstructured datasets

Develop, schedule, and monitor workflows with tools like Airflow or similar

Deploy and maintain pipelines on Kubernetes and manage distributed computing workloads using Ray

Collaborate closely with AI teams and stakeholders to iterate quickly and deliver robust solutions

Implement CI/CD, logging, and monitoring to ensure production reliability

Take full ownership of projects and deliver under minimal resource constraints

Must-Have Skills:

Expert in Python programming for data engineering

Experience with Apache Spark and AWS Glue ETL jobs

Strong knowledge of Kafka for real-time streaming data pipelines

Workflow orchestration and scheduling experience (Airflow, Prefect, etc.)

DevOps background including CI/CD, containerization, and production deployment on Kubernetes

Familiarity with distributed computing frameworks like Ray

Startup mindset: ownership, resourcefulness, and independent problem solving

Preferred Qualifications:

3-7 years of relevant experience in data engineering roles

Prior experience working in startups or fast-moving environments

Portfolio or examples of production-grade data systems you have built

Why Join Us?

Work on cutting-edge AI projects with significant responsibility

Opportunity to shape the data infrastructure in a growing startup

Flexible work environment with an ownership-driven culture