Senior Data Engineer
Flintex Consulting
Full-time
On-site
Singapore, Singapore, Singapore
Senior Jobs
Responsibilities<\/u><\/b>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Integrate data from multiple sources, such as databases, APIs, or streaming platforms, to provide a unified view of the data<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Identify and resolve data quality issues, monitor data pipelines for errors, and implement data governance and data quality frameworks<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Enforce data security and compliance with relevant regulations and industry\-specific standards<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Implement data access controls, encryption mechanisms, and monitor data privacy and security risks<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Optimise data processing and query performance by tuning database configurations, implementing indexing strategies, and leveraging distributed computing frameworks<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Optimize data structures for efficient querying and develop data dictionaries and metadata repositories<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Identify and resolve performance bottlenecks in data pipelines and systems<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Collaborate with cross\-functional teams, including data scientists, analysts, and business stakeholders<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Document data pipelines, data schemas, and system configurations, making it easier for others to understand and work with the data infrastructure<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Monitor data pipelines, databases, and data infrastructure for errors, performance issues, and system failures<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Set up monitoring tools, alerts, and logging mechanisms to proactively identify and resolve issues to ensure the availability and reliability of data<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠It would be a plus if he has software engineering background<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
Requirements<\/u><\/b>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Bachelor's or master's degree in computer science, information technology, data engineering, or a related field<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Strong knowledge of databases, data structures, algorithms<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Proficiency in working with data engineering tools and technologies including knowledge of data integration tools (e.g., Apache Kafka, Azure IoTHub, Azure EventHub), ETL\/ELT frameworks (e.g., Apache Spark, Azure Synapse), big data platforms (e.g., Apache Hadoop), and cloud platforms (e.g., Amazon Web Services, Google Cloud Platform, Microsoft Azure)<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Expertise in working with relational databases (e.g., MySQL, PostgreSQL, Azure SQL, Azure Data Explorer) and data warehousing concepts.<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Familiarity with data modeling, schema design, indexing, and optimization techniques is valuable for building efficient and scalable data systems<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Proficiency in languages such as Python, SQL, KQL, Java, and Scala<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Experience with scripting languages like Bash or PowerShell for automation and system administration tasks<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Strong knowledge of data processing frameworks like Apache Spark, Apache Flink, or Apache Beam for efficiently handling large\-scale data processing and transformation tasks<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Understanding of data serialization formats (e.g., JSON, Avro, Parquet) and data serialization libraries (e.g., Apache Avro, Apache Parquet) is valuable<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Having experience in CI\/CD and GitHub that demonstrates ability to work in a collaborative and iterative development environment<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
⢠Having experience in visualization tools (e.g. Power BI, Plotly, Grafana, Redash) is beneficial<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
<\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
Preferred Skills & Characteristics<\/u><\/b><\/span>
<\/span><\/span> <\/div>
<\/span><\/span> <\/div>
Consistently display dynamic independent work habits, goal oriented, passionate in growth mindsets, possess a âcan doâ attitude, and self\-motivated professional. Self\-driven and proactive in keeping up with new technologies and programming<\/span>
<\/span><\/span> <\/div>
<\/div><\/span>
<\/body> <\/html>"}}],"isMobile":false,"iframe":"true","jobType":"Full time","applyName":"Apply Now","zsoid":"651761644","FontFamily":"Arial, Helvetica, sans\-serif","jobOtherDetails":[{"fieldLabel":"Work Experience","uitype":2,"value":"4\-5 years"},{"fieldLabel":"Industry","uitype":2,"value":"Technology"},{"fieldLabel":"Salary","uitype":1,"value":"SGD 5500 \- SGD 5800"},{"fieldLabel":"City","uitype":1,"value":"Singapore"},{"fieldLabel":"State\/Province","uitype":1,"value":"Central Singapore"},{"fieldLabel":"Zip\/Postal Code","uitype":1,"value":"079903"}],"headerName":"Senior Data Engineer","widgetId":"418400000000381837","isJobBoard":"false","userId":"418400000000179003","attachArr":[],"customTemplate":"5","isCandidateLoginEnabled":false,"jobId":"418400000043777922","FontSize":"13","googleIndexUrl":"https:\/\/flintex.zohorecruit.com\/recruit\/ViewJob.na?digest=PmtGoOx0iV9gs6xXwgvAe7GK50MwNBKcb43UVbp0smI\-&embedsource=Google","location":"Singapore","embedsource":"CareerSite","indeedCallBackUrl":"https:\/\/recruit.zoho.com\/recruit\/JBApplyAuth.do","logoId":"3xpxkf7cb8cd512f54263a74d5094e14b9c51"}
<\/span><\/span> <\/div>
<\/div><\/span>
<\/body> <\/html>"}}],"isMobile":false,"iframe":"true","jobType":"Full time","applyName":"Apply Now","zsoid":"651761644","FontFamily":"Arial, Helvetica, sans\-serif","jobOtherDetails":[{"fieldLabel":"Work Experience","uitype":2,"value":"4\-5 years"},{"fieldLabel":"Industry","uitype":2,"value":"Technology"},{"fieldLabel":"Salary","uitype":1,"value":"SGD 5500 \- SGD 5800"},{"fieldLabel":"City","uitype":1,"value":"Singapore"},{"fieldLabel":"State\/Province","uitype":1,"value":"Central Singapore"},{"fieldLabel":"Zip\/Postal Code","uitype":1,"value":"079903"}],"headerName":"Senior Data Engineer","widgetId":"418400000000381837","isJobBoard":"false","userId":"418400000000179003","attachArr":[],"customTemplate":"5","isCandidateLoginEnabled":false,"jobId":"418400000043777922","FontSize":"13","googleIndexUrl":"https:\/\/flintex.zohorecruit.com\/recruit\/ViewJob.na?digest=PmtGoOx0iV9gs6xXwgvAe7GK50MwNBKcb43UVbp0smI\-&embedsource=Google","location":"Singapore","embedsource":"CareerSite","indeedCallBackUrl":"https:\/\/recruit.zoho.com\/recruit\/JBApplyAuth.do","logoId":"3xpxkf7cb8cd512f54263a74d5094e14b9c51"}