Lead Data Engineer - Databricks

Join us as we embark on a journey of collaboration and innovation, where your unique skills and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference.

As a Lead Data Engineer at JPMorgan Chase within the Corporate Technology - FINTECH team, you are an integral part of an agile team that works to enhance, build, and deliver data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. As a core technical contributor, you are responsible for maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm’s business objectives.

Job responsibilities

• Generates data models for their team using firmwide tooling, linear algebra, statistics, and geometrical algorithms
• Delivers data collection, storage, access, and analytics data platform solutions in a secure, stable, and scalable way
• Implements database back-up, recovery, and archiving strategy 
• Evaluates and reports on access control processes to determine effectiveness of data asset security​ with minimal supervision
• Adds to team culture of diversity, equity, inclusion, and respect

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• 3+ years of experience using technologies such as Databricks , Pyspark, AWS, is essential and creating ETL Pipeline from scratch is a must.
• 3+ years of experience working with AWS (Lambda, Step Function, SQS, SNS, API Gateway, secrets manager and storage services) is a must.
• 3+ years of experience in software engineering and object-oriented programming skills with expertise in Python and Terraform 
• Hands on experience with open-source frameworks/libraries, such as Apache NiFi, Apache Airflow and Autosys.
• Strong understanding of REST API development using FASTAPI or equivalent frameworks.
• Advanced at SQL (e.g., joins and aggregations)

• Familiar with development tools such as Jenkins, Jira, Git/Stash, spinnaker
• Familiarity with unit testing frameworks such as pytest or unittest.
• Extensive experience in statistical data analysis, with the ability to select appropriate tools and identify data patterns for effective analysis, as well as experience throughout the data lifecycle