Senior Data Engineer (Durban/ Hybrid) (DWR024)

Senior Data Engineer (Durban / Hybrid) (SA citizens only)
Are you a technical powerhouse with a passion for high-performance computing and data engineering? We're looking for a Senior Data Engineer to join our team—someone who thrives in fast-paced, complex environments and wants to build scalable, cutting-edge data solutions.

• Design, develop, and maintain end-to-end technical aspects of all data pipelines required to support data managers
• Support ETL processes including, data ingestion, transformation, validation, and integration processes using various tools and frameworks
• Optimize data performance, scalability, and security
• Provide technical guidance and support to data analysts 
• Design data integrations and data quality frameworks
• Work and collaborate with the rest of the IT department to help develop the strategy for long term scientific Big Data platform architecture
• Document and effectively communicate data engineering processes and solutions.
• Make use of and help define the right cutting edge technology, processes and tools needed to drive technology within our science and research data management departments.

• Experience: 7+ years in Data Engineering
• Bachelor's degree in CS, IT, Engineering, Math, or similar
• Experience in Data Warehousing, or Big Data Processing
• Solid Skills in: Data modeling, data pipeline design, data governance
• VMware, OpenStack, KVM, and high-performance compute clusters
• Skilled in technologies such as Hadoop, Kafka, Spark, NiFi, or Cloud-based tools (AWS, GCP, Azure)
• Advanced skills with: Linux/Unix/Windows OS, containerization (Docker, Kubernetes) Python, R, SQL, PowerShell

Market related