AVP - Senior Lead Engineer - Data
M&G View all jobs
- Pune, Maharashtra
- Permanent
- Full-time
- Lead data engineering projects to build and operationalize data solutions for business using Azure services in combination with custom solutions – Azure Data Factory, Azure Data Flows, Azure Databricks, Azure Data Lake Gen 2, Azure SQL etc
- Proven experience on leading a team of data engineers providing technical guidance and ensuring alignment with agreed architectural principles
- Experience in migrating on-premise data warehouses to data platforms on AZURE cloud
- Designing and implementing data engineering, ingestion and transformation functions using ADF, Databricks
- Proficient in Py-Spark
- Experience in building Python based APIs on Azure Function Apps
- Experience on Azure Logic apps
- Experience in Lakehouse/Datawarehouse implementation using modern data platform architecture
- Capacity Planning and Performance Tuning on ADF & Databricks pipelines
- Support data visualization development using Power BI
- Exposure across all the SDLC process, including testing and deployment
- Experience in relational and dimensional modelling, including big data technologies
- Experience in Azure DevOps – Build CI/CD pipelines for ADF, ADLS, Databricks, Azure SQL DB etc
- Experience of working in secured Azure environments using Azure KeyVaults, Service Principals, and Managed Identities
- Good to have knowledge on Apigee (Googles API Management)
- Understanding of data masking, encryption and other practices used in handling sensitive data
- Ability to interact with Business for requirement gathering and query resolutions
- Working on off shore office based development teams, collaborating within a team environment and participating in typical project lifecycle activities such as requirement analysis, testing and release
- Develop Azure Data skills within the team through knowledge sharing sessions, articles, etc.
- Adherence to organisations Risk & Controls requirements
- Should have skills for Stakeholder management, process adherence, planning & documentationss
- Azure Data Factory,
- Azure Data Lake Storage V2
- Azure SQL
- Azure DataBricks
- Pyspark
- Azure DevOps
- Power BI Report
- Technical leadership
- People Management
- Confidence & excellent communication
- Overall 12+ years of experience
- 5 + Experience on Azure data engineering
- 5 + experience of managing data deliveries
- 5 + experience in people management
- Graduate/Post-graduate. Preferably with specialisation in Computer Science, Statistics, Mathematics, Data Science, Engineering or related discipline