Data Engineer

We are seeking a skilled Data Engineer to design, develop, optimize, and manage robust, highly available data analytics infrastructure, reports, and data models. This role drives the delivery of high-quality data analytics solutions for data ingestion, storage, consumption, and management to generate actionable insights. Reporting to the Technical Team Lead and Software Development Manager, you will play a critical role in advancing our data capabilities.

Key Responsibilities

• Data Pipeline Development: Build and maintain scalable data pipelines for efficient data ingestion, processing, and storage.
• ETL Processes: Develop and automate ETL workflows to integrate data from diverse sources seamlessly.
• Database Management: Manage SQL and NoSQL databases to handle structured and unstructured data effectively.
• Data Quality Assurance: Ensure data integrity, consistency, and security through proactive monitoring.
• Collaboration: Partner with data scientists, analysts, and software engineers to enable data-driven decision-making.
• Documentation: Create clear, detailed documentation for data workflows, architectures, and processes.
• Performance Optimization: Enhance database performance through tuning, query optimization, and indexing.
• Cloud & Big Data Technologies: Support and expand expertise in cloud-based platforms like AWS, Azure, or Google Cloud.

Key Performance Indicators

• Pipeline Efficiency: Percentage of data pipelines executed on schedule.
• Data Accuracy: Reduction in data errors and inconsistencies.
• ETL Performance: Improved speed and efficiency of ETL processes.
• Query Performance: Reduced database query response times.
• Collaboration Success: Number of effective collaborations with data scientists and analysts.
• Code & Documentation Quality: Adherence to coding standards and comprehensive documentation.

Qualifications & Experience

• Bachelors or Honours Degree in Engineering, Data Science, Computer Science, or Information Systems.
• 3 - 5 years in data engineering, analytics, or data management roles.
• 3+ years working with sales, channel, or business development teams.
• 3+ years delivering customer-facing projects.
• 3+ years in the cybersecurity industry.
• 3+ years using agile methodologies for project delivery.

Systems & Proficiency

• Certification in AWS, Microsoft, or other Business Intelligence technologies.
• Expertise in Databricks and Python Notebooks.
• Advanced skills in Power BI, DAX, Advanced Excel, Python, SQL Server, and SQL Scripting.
• Proficiency in creating technical architectures, entity relationship diagrams, and process flow diagrams.
• Strong command of Microsoft Office.

Skills & Competencies

• Ability to manage multiple programs, balancing strategic planning with fast-paced execution.
• Strong communication, negotiation, and consensus-building skills with stakeholders and teams.
• Exceptional presentation skills, comfortable presenting to executive leadership.
• Superior analytical, organizational, and time-management abilities.
• Thrives in high-pressure environments and adapts quickly to new skills.
• Trusted to handle confidential information with discretion.
• Self-motivated, team-oriented, with a structured approach and proactive ownership of tasks.

ETL Discretion Data Engineering Software Engineers Indicators Diagrams Analysts Collaboration Data Quality Project Delivery Intelligence SQL Server Pipelines Azure NoSQL Big Data Assurance Agile Presentation Skills Data Integrity Key Performance Indicators Business Intelligence Ownership Information Systems Data Management AWS Data Science Excel Decision-Making Data Analytics Agile Methodologies Software Development Storage Optimization Analytics Infrastructure Quality Assurance Power BI Strategic Planning Databases Business Development Negotiation Computer Science Security Documentation Software Planning Design Engineering SQL Python Business Microsoft Office Leadership Science Sales Communication Management