Head of Data Engineering
MindMerge Consulting
Full-time
On-site
Petaling Jaya, Selangor, Malaysia
Our client is a well known leading data information provider in Malaysia and currently looking for a Head of Data Engineering. This role will be responsible for leading the design, development and optimization of the organization data infrastructure, ensuring accessibility, retrievability, security, quality, scalability, efficiency, retention and ethical handling of the company’s data assets. The role will oversee data architecture, pipelines, governance and engineering best practices while collaborating with product, analytics and business teams to unlock the value of the company’s data assets.
Key Responsibilities
Data Infrastructure & Architectures
- Build and maintain scalable, secure and high-performance data platforms (on-premise and cloud).
- Implement robust data integration pipelines to ingest data from multiple sources (batch and real-time) to support credit scoring, analytics and digital services.
- Design processes to create data insights, data features and operationalize analytics models.
- Operationalize data pipelines for automated and semi-automated execution.
- Implement reporting of data universe, data quality across the company.
- Oversee and deliver internal and external data extraction services in line with business needs and client SLAs.
Data Governance & Quality
- Collaborate with Data Governance & Compliance together with Cybersecurity & Risk team to ensure adherence to regulatory requirements (e.g. BNM, PDPA) and internal policies.
- Suggest and implement technology upgrades to data platforms.
- Support key initiatives including migration of the company’s data services to the cloud.
Data Innovation & Optimization
- Evaluate and adopt modern data engineering tools, cloud platforms (AWS, Azure, GCP) and big data technologies (Spark, Kafka, Hadoop, Databricks) including workflow tools and dashboarding.
- Champion automation, CI/CD for data pipelines, and DataOps best practices.
- Support AI/ML initiatives by ensuring availability of clean, structured, and accessible data.
Stakeholder Management
- Work closely with data analytics, product and technology teams to deliver high-impact data solutions.
- Act as subject matter expert for data engineering, advising on opportunities and risks.
Requirements
- Bachelor’s degree or higher in Computer Science, Information Systems, Data Engineering or a related discipline.
- 10+ years of hands-on experience in data engineering with at least 3 years in a leadership role.
- Proven experience in designing and managing large-scale data platforms in financial services, fintech or technology sectors (credit reporting, or regulatory data environment is a plus).
- Understanding of financial industry and related data (CCRIS, SSM, Angkasa, Idaman) is a plus.
- Strong knowledge of data pipelines and data development platforms such as Hadoop, Spark/PySpark, Jupyter, cloud native services.
- Experience but not limited to AWS data & analytics native services is a plus: AWS Glue, Athena, Lake Formation, Redshift, EMR, SageMaker.
- Proficiency with ETL, Integration tools (e.g. AWS Glue, Talend, Informatica, Microsoft SSIS, Apache NiFi or equivalent).
- Strong SQL skills and experience with relational and non-relational databases (e.g. Postgres, SQL Server, Oracle, MongoDB).
- Knowledge of data governance, security, and privacy regulations (e.g. GDPR, PDPA).
- Experience with version control systems (e.g. Git), CI/CD pipelines, and workflow orchestration tools (e.g. Apache Airflow).
- Excellent problem-solving, analytical thinking, and debugging skills.
- Strong stakeholder management skills.
- High attention to detail, integrity and a strong sense of responsibility.
- Able to manage multiple projects in fast-paced environment