Azure Data Engineer
Talent Bee Consulting
Full-time
On-site
Bengaluru, Karnataka, India
Ideal Candidate
- 3+ years in Azure Data Engineering
- Very good at ADF (Azure Data Factory)
- Hands-on experience in designing and implementing data solutions on Azure
Organization
A global provider of Automation solutions in Assembly, Inspection, and Testing for manufacturing industries. With a mission to enhance the efficiency of the manufacturing processes of their customers by providing innovative technological solutions.
Over the last 16+ years, they have been providing automation solutions that are scalable and cost-effective.
Roles and Responsibilities:
We are seeking an experienced Azure Data Engineer with 4 to 5 years of hands-on experience in Azure Data Factory (ADF), Databricks, and Python. In this role, you will be responsible for designing, implementing, and managing data solutions on Microsoft Azure, with a specific focus on ADF, Databricks, and Python scripting. You will collaborate with cross-functional teams to gather data requirements, develop efficient data pipelines, and ensure data quality and integrity..
Key Responsibilities:
- Design and implement data solutions on Azure using ADF, Databricks, Stream Analytics / Spark
- Develop and optimize data pipelines to extract, transform, and load data from various sources into Azure data lakes o
- Collaborate with data architects and business stakeholders to understand data requirements and translate them into technical solutions.
- Implement data integration and data transformation processes using ADF and Databricks.
- Perform data profiling, validation, and cleansing activities to ensure data accuracy and consistency.
- Develop and maintain data orchestration workflows and scheduling using ADF.
- Monitor and troubleshoot data pipelines to identify and resolve issues in a timely manner.
- Optimize data storage and retrieval processes for performance and cost-efficiency.
- Implement data security and privacy measures, including data encryption and access controls.
- Document technical specifications, data mappings, and data lineage for future reference.
Candidate:
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- 3 to 5 years of hands-on experience in designing and implementing data solutions on Azure, with a focus on ADF, Databricks, Stream Analytics / Spark
- Strong proficiency in ADF, including pipeline development, data integration, and orchestration.
- Experience with Databricks for data transformation, ETL processes, and data analytics.
- Proficiency in Python scripting for data manipulation, transformation, and automation.
- Solid understanding of data modeling, data warehousing concepts, and data integration patterns.
- Experience with data profiling, data quality management, and data governance practices.
- Familiarity with Azure services like Azure Data Lake Storage, Azure SQL Database, and Azure Cosmos DB.
- Strong analytical and problem-solving skills with the ability to troubleshoot and resolve data-related issues.
- Excellent communication and collaboration skills to work effectively with cross-functional teams.
Location
Bangalore
Salary
6-12 LPA