Job Summary:
We are seeking a highly skilled Lead Data Engineer/Associate Architect to lead the design, implementation, and optimization of scalable data architectures. The ideal candidate will have a deep understanding of data modeling, ETL processes, cloud data solutions, and big data technologies. You will work closely with cross-functional teams to build robust, high-performance data pipelines and infrastructure to enable data-driven decision-making.
Experience: 7 - 12 years
Work Location: Hyderabad (Hybrid) / Remote
Mandatory skills: AWS, Python, SQL, Airflow, DBT
Must have done 1 or 2 projects in Clinical Domain/Clinical Industry.
Responsibilities:
- Design and Develop scalable and resilient data architectures that support business needs, analytics, and AI/ML workloads.
- Data Pipeline Development: Design and implement robust ETL/ELT processes to ensure efficient data ingestion, transformation, and storage.
- Big Data & Cloud Solutions: Architect data solutions using cloud platforms like AWS, Azure, or GCP, leveraging services such as Snowflake, Redshift, BigQuery, and Databricks.
- Database Optimization: Ensure performance tuning, indexing strategies, and query optimization for relational and NoSQL databases.
- Data Governance & Security: Implement best practices for data quality, metadata management, compliance (GDPR, CCPA), and security.
- Collaboration & Leadership: Work closely with data engineers, analysts, and business stakeholders to translate business requirements into scalable solutions.
- Technology Evaluation: Stay updated with emerging trends, assess new tools and frameworks, and drive innovation in data engineering.
Required Skills:
- Education: Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
- Experience: 7 - 12+ years of experience in data engineering
- Cloud Platforms: Strong expertise in AWS data services.
- Databases: Hands-on experience with SQL, NoSQL, and columnar databases such as PostgreSQL, MongoDB, Cassandra, and Snowflake.
- Programming: Proficiency in Python, Scala, or Java for data processing and automation.
- ETL Tools: Experience with tools like Apache Airflow, Talend, DBT, or Informatica.
- Machine Learning & AI Integration (Preferred): Understanding of how to architect data solutions for AI/ML applications