Senior Data Engineer -W2

Key Responsibilities:

• Develop and maintain robust data pipelines using technologies such as Apache Spark, Python, and AWS services.
• Design and implement data models for efficient storage and retrieval of structured and unstructured data.
• Collaborate with cross-functional teams to understand data requirements and provide technical expertise.
• Optimize and troubleshoot data processes to ensure high performance and reliability.
• Create and maintain documentation for data engineering processes, workflows, and best practices.
• Implement ETL processes to transform and load data from diverse sources into data warehouses or lakes.
• Participate in code reviews and contribute to the continuous improvement of data engineering practices and standards.
• Develop and maintain data quality and governance frameworks to ensure data accuracy and integrity.
• Identify opportunities to enhance data infrastructure and propose innovative solutions to address business needs.
• Stay current with industry trends and advancements in data engineering to drive continuous innovation.

Required Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of professional experience in data engineering or a related role.
• Proven expertise in Python and SQL for data manipulation, analysis, and scripting.
• Extensive experience with Apache Spark for big data processing and analytics.
• Strong understanding of data modeling principles and techniques.
• Advanced knowledge of ETL processes and data warehousing concepts.
• Experience with cloud platforms such as AWS for building and deploying data solutions.
• Proficiency in data pipeline development using tools and frameworks like Apache Airflow or similar.
• Solid understanding of distributed computing principles and big data technologies.
• Excellent problem-solving skills and the ability to optimize data processes for performance and scalability.
• Experience in implementing data governance and quality assurance measures.
• Strong communication and collaboration skills to work effectively within a team and across departments.
• Ability to multitask and manage priorities in a fast-paced, dynamic environment.
• Relevant certifications in data engineering or cloud platforms are a plus.
• Deep passion for data-driven decision-making and continuous learning in the field of data engineering.