What awaits you/ Job Profile | - Analyze, Organize data and Build data systems with multiple data sources
- · Working closely with business and market team for the Data analysis.
|
What should you bring along | - Build data systems and pipelines
- Standardize the data assets which maximize data reusability
- Evaluate business needs and objectives
- Implementing data pipeline to ingest and cleanse raw data from various source systems including SaaS, Cloud based and On-prep databases
- Combine raw information from different sources
- Explore ways to enhance data quality and reliability
- Implementing functional requirements including incremental loads, data snapshots and identity resolution
- Optimizing data pipelines to improve performance and cost, while ensuring a high quality of data within the data lake:
- Monitoring services and jobs for cost and performance, ensuring continual operations of data pipelines, and fixing of defects.
- Constantly looking for opportunities to optimize data pipelines to improve performance.
|
Must have technical skill | - · Must have coding skills in Spark/Pyspark, Python and SQL
- · Must have Knowledge of AWS tools, Glue, Athena ,step functions and S3
|
Good to have technical skills | - Good knowledge of CI/CD (Github / Github Actions, Terraform)
- Knowledge on Agile methodologies.
|