Data Engineer

KEY RESPONSIBILITIES

Writing Codein Scala or Python

Design, Develop and test BigData solutions using selected tools

Read, extract, transform, stage and load data

Writing code/scripts, web scraping, calling APIs, SQL, queries, etc.

Process unstructured data and to generate modelled data

Build self monitoring and self healing/zero touch operations

Ability to design solutions independently based on high-level architecture

Collaborate with other development and research teams

PREFERRED QUALIFICATIONS

3+ years of experience as a Big Data Engineer /Software Engineer or similar role

Bachelor's Degree in Computer Science or equivalent preferred

Programming experience, ideally in Python, Scala or Java

Willingness to learn new programming languages

Experience with Spark/Hadoop Ecosystem

Experience with building stream-processing systems, such Spark-Streaming

Experience with integration of data from multiple data sources

Experience with various messaging systems, such as Kafka

Knowledge of statistics

Knowledge of data cleaning, wrangling, visualization and reporting

Experience in Apache Nifi, PostgreSQL, MapReduce, TimescaleDU, data mining, machine learning, natural language processing, or information retrieval is a plus

Experience processing large amounts of structured and unstructured data

Integrating data from multiple sources

Experience with Big Data ML toolkits, such as SparkML, Mahout, or H20 is a plus