Senior Data Engineer

We are seeking a Senior Data Engineer to architect and build robust, scalable, and efficient data systems that power AI and Analytics solutions. You will design end-to-end data pipelines, optimize data storage, and ensure seamless data availability for machine learning and business analytics use cases. This role demands deep engineering excellence — balancing performance, reliability, security, and cost — to support real-world AI applications.

Key Responsibilities

• Architect, design, and implement high-throughput ETL/ELT pipelines for batch and real-time data processing.

• Build cloud-native data platforms: data lakes, data warehouses, feature stores.

• Work with structured, semi-structured, and unstructured data at petabyte scale.

• Optimize data pipelines for latency, throughput, cost-efficiency, and fault tolerance.

• Implement data governance, lineage, quality checks, and metadata management.

• Collaborate closely with Data Scientists and ML Engineers to prepare data pipelines for model training and inference.

• Implement streaming data architectures using Kafka, Spark Streaming, or AWS Kinesis.

• Automate infrastructure deployment using Terraform, CloudFormation, or Kubernetes operators.

Required Skills

• 7+ years in Data Engineering, Big Data, or Cloud Data Platform roles.

• Strong proficiency in Python and SQL.

• Deep expertise in distributed data systems (Spark, Hive, Presto, Dask).

• Cloud-native engineering experience (AWS, GCP, Azure): BigQuery, Redshift, EMR, Databricks, etc.

• Experience designing event-driven architectures and streaming systems (Kafka, Pub/Sub, Flink).

• Strong background in data modeling (star schema, OLAP cubes, graph databases).

• Proven experience with data security, encryption, compliance standards (e.g., GDPR, HIPAA).

Preferred Skills

• Experience in MLOps enablement: creating feature stores, versioned datasets.

• Familiarity with real-time analytics platforms (Clickhouse, Apache Pinot).

• Exposure to data observability tools like Monte Carlo, Databand, or similar.

• Passionate about building high-scale, resilient, and secure data systems.

• Excited to support AI/ML innovation with state-of-the-art data infrastructure.

• Obsessed with automation, scalability, and best engineering practices.