
Senior Data Engineering Lead
- Gurgaon, Haryana
- Permanent
- Full-time
- Design, develop, and maintain scalable data/code pipelines using Azure Databricks, Apache Spark, and Scala
- Collaborate with data engineers, data scientists, and business stakeholders to understand data requirements and deliver high-quality data solutions
- Optimize and tune Spark applications for performance and scalability
- Implement data processing workflows, ETL processes, and data integration solutions
- Ensure data quality, integrity, and security throughout the data lifecycle
- Troubleshoot and resolve issues related to data processing and pipeline failures
- Stay updated with the latest industry trends and best practices in big data technologies
- Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
- Minimum 12 months in the current Role/Grade/Level
- Not on Active CAP at the time of applying for the IJP
- Employees currently at Grade 27 will move to the next grade and Grade 28 (Lateral Movement) can apply for this position
- Last Review rating of Meets Expectations or Exceeds Expectation
- 8+ years Proven experience with Azure Databricks, Apache Spark, and Scala
- 8+ years experience with Microsoft Azure
- Experience with data warehousing solutions and ETL tools
- Solid understanding of distributed computing principles and big data processing
- Proficiency in writing complex SQL queries and working with relational databases
- Proven excellent problem-solving skills and attention to detail
- Solid communication and collaboration skills.