Detailed Description:
As a Principal Data Engineer within Oracle's Consumer Global Industries Unit, you will architect and build foundational components of a next-generation data platform supporting Oracle Industry Applications. You will be responsible for designing and implementing scalable, high-performance data pipelines, and contribute significantly to data integration, transformation, and delivery processes. You will collaborate with data architects on best practices for data modeling and architecture (including medallion architecture), champion modern engineering methods, and ensure compliance with security, data quality, and governance requirements. Your expertise in cloud-based data services, ETL/ELT, and real-time/batch data processing will directly enable high-value, AI-enriched solutions across multiple industries.
Required Experience:
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or a related discipline.
- 7+ years of experience in data engineering with demonstrated expertise designing, building, and maintaining large-scale enterprise data platforms.
- Proven hands-on experience with cloud ecosystem data services (OCI, AWS, Azure), their data engineering tools, and AI/ML service integration.
- Advanced skills in building reliable ETL/ELT pipelines, data integration, transformation, validation, and orchestration frameworks.
- Strong programming skills (e.g., Python, Kafka, Java, Scala) with experience in distributed data processing technologies (e.g. Spark, Databricks, Apache Beam).
- In-depth knowledge of data governance, quality, security, and compliance best practices within cloud-first or hybrid environments.
- Proficiency with industry-standard tools for workflow orchestration (e.g., Airflow), data cataloging, and monitoring.
- Experience in regulated industries and familiarity with relevant compliance standards is a plus.
- Excellent problem-solving, communication, and collaboration skills within global and cross-functional teams.
Key Responsibilities:
- Design and develop robust, scalable, and secure data pipelines (batch and real-time) to ingest, process, and deliver data from multiple sources to Oracle Industry Applications.
- Implement and optimize data architectures in cloud environments (OCI, AWS, Azure) including data lakes, data warehouses, ETL/ELT, and orchestration frameworks.
- Collaborate with data architects to implement best practices for data modeling (conceptual, logical, and physical) and support platform-wide standards such as medallion architecture (Bronze/Silver/Gold).
- Design and establish monitoring, alerting, and data quality frameworks to ensure reliability and accuracy of data pipelines.
- Develop and enforce best practices for data security, lineage, metadata management, and compliance requirements.
- Work closely with analysts, ML engineers, business stakeholders, and cross-functional teams to enable industry-specific data use cases and analytics for Oracle Industry Applications.
- Evaluate, recommend, and onboard new data technologies to continuously improve platform performance and capabilities.
- Lead troubleshooting and root cause analysis for data issues, ensuring continuous platform availability and reliability.
- Mentor and guide junior data engineers and drive technical excellence and innovation across engineering efforts.
Career Level - IC4