Careernet Technologies logo

Data Engineer

Careernet Technologies
Full-time
On-site
Bengaluru, Karnataka, India
We are looking for a Senior Software Engineer, Ideal candidate would be someone with a passion for Clean Code, Scalable architectures, Test Driven Development and DevOps. Someone who loves getting hands dirty with code and learns everyday.   
 
 
RESPONSIBILITIES
 
  • Hands on developer/lead to work on data modeling, code, and test iteratively on technical solutions for monitoring and analytics requirements.
  • Coordinates with product and leads development team, leads scrum team/board management.
  • Implement ETL processes, UI and dashboards for appropriate trends and alerts in BI tools (Superset, Spotfire, QuickSight, etc.) for monitoring.
  • Plan, develop, execute and maintain automated unit, functional test cases; develop modular, robust, and maintainable automation scripts.
  • Implement automation in the areas of Cloud resource provisioning, unit testing, code coverage, static code analysis and vulnerability scanning with CI/CD process.
  • Must have strong analytical and technical skills with passion to deep dive on data in troubleshooting , devise techniques for problem resolution.
  • Promote a culture of best practices with peer code reviews and extreme ownership for continuous incremental delivery.
  • Maintain and document standards, best practices, and design/architecture patterns.
 
QUALIFICATIONS AND SKILLS
  • 2-7 years of experience with ETL automation, SQL and BI Tools experience is a must (Superset, TIBCO Spotfire, etc.)
  • Programming experience Python/Java and desire to write and review code 70% of time. 
  • Familiarity with relational and big data technology stack such as Postgres, Hadoop and columnar Storage formats Parquet
  • Familiarity of Linux platforms with knowledge of shell scripting.
  • Hands-on experience with Spark, Spark SQL, Spark on K8s, M/R Framework and Big Data Technology Stack.
  • Experience developing cloud-hosted (AWS), containerized applications and services is plus.
  • AWS Components: EKS, EMR, EC2, S3 storage, Lambda, Relational Database Service, Simple Notification Service (SNS), Elasticache, etc.
  • Experience with source code control tools. Gitlab experience is a plus.
  • Knowledge of networking and protocols: HTTP/HTTPS, TLS/SSL/certificates, TCP/IP is plus
  • Knowledge of CI/CD processes, testing frameworks, practices and tools (Jenkins, jUnit, Terraform, JFrog, Jacoco, SonarQube, etc.) is a plus