Senior Big Data Engineer - Pyspark

Job description:

 

"Data engineering development & testing with experience in Py-Spark and SQL”.

Experienced big data developer with hands on Py-Spark & Hadoop/HDFS development skills.  Ideal candidate would be a big data engineer passionate about data, data modelling, data enrichment for deriving deep insights accessible via Visualization dashboards or third party tools.

 

Must have skills

      •     Must be a Big Data engineer and passionate about data.

      •     Hands on Py-Spark, Hadoop, SQL, HDFS, Kafka experience plus optionally Python

      •     Hands on Experience in big data lake house like Databricks

      •     Working knowledge and hands-on experience in known Big Data platforms and Hadoop tools & technologies

      •     Experience with extraction, enrichment, and export large volumes data.

      •     Experience in map-reduce and well-known big data algorithms.

      •     Database development experience with a solid understanding of core database concepts, relational database design ODS and DWH

      •     Peer reviews, Unit testing, deployment to production working with several teams to deliver big data solutions.

      •     Experience in Finance big data projects will be a plus