Description
Position at Ness Digital Engineering (India) Private Limited
Why Ness
We know that people are our greatest asset. Our staff’s professionalism, innovation, teamwork, and dedication to excellence have helped us become one of the world’s leading technology companies. It is these qualities that are vital to our continued success. As a Ness employee, you will be working on products and platforms for some of the most innovative software companies in the world.
You’ll gain knowledge working alongside other highly skilled professionals that will help accelerate your career progression.
You’ll also benefit from an array of advantages like access to trainings and certifications, bonuses, and aids, socializing activities and attractive compensation.
Key Responsibilities:
- Design, develop, and optimize ETL pipelines in PySpark and SQL within the Databricks environment.
- Re-engineer existing ETL workflows running on Postgres to leverage Databricks Notebooks and Delta Lake architecture.
- Implement and manage the Bronze → Silver → Gold data model.
- Generate extract files using a metadata catalog that maps Delta columns to destination Postgres schema.
- Collaborate with Data Architects/Engineers, POs and DevOps to implement scalable and secure data pipelines.
- Contribute to cost optimization by eliminating legacy Postgres workflows.
- Ensure data quality, lineage and performance optimization at every stage.
Required Skills:
- Proven expertise in PySpark, Databricks, and Delta Lake.
- Strong proficiency in SQL and data transformation logic.
- Hands-on experience with data migration from RDBMS (Postgres) to big data platforms.
- Solid understanding of data modeling (especially multi-layer architectures).
- Experience in developing or consuming data catalogs.
- Proficient in debugging and performance tuning in distributed environments.
Not checking every single requirement?
If this role sounds good to you, even if you don’t meet every single bullet point in the job description, we encourage you to apply anyway. For most of the candidates that applied, we found a role that was a very good fit with their skills.
Let’s meet and you may just be the right candidate for one of our roles.
At Ness Digital Engineering we are willing to build a work culture that is based on diversification, inclusion, and authenticity.