
Sr Eng Data Engineering
- Bangalore, Karnataka
- Permanent
- Full-time
- Work with platform data engineering, core platform, security, and technical quality to design, implement and deploy data engineering solutions.
- Develop pipelines for ingestion, transformation, orchestration, and consumption of various types of data.
- Design and deploy data layering pipelines that use modern Spark based data processing technologies such as Databricks and Delta Live Table (DLT).
- Integrate data engineering solutions with Azure data governance components not limited to Purview and Databricks Unity Catalog.
- Implement and support security monitoring solutions within Azure Databricks ecosystem.
- Design, implement, and support data monitoring solutions in data analytical workspaces.
- Configure and deploy Databricks Analytical workspaces in Azure with IaC (Terraform, Databricks API) with J&J DevOps automation tools within JPM/Xena framework.
- Implement automated CICD processes for data processing pipelines.
- Support DataOps for the distributed DSP data architecture.
- Function as a data engineering SME within the data platform.
- Manage authoring and execution of automated test scripts.
- Build effective partnerships with DSP architecture, core infrastructure and other domains to design and deploy data engineering solutions.
- Work closely with the DSP Product Managers to understand business needs, translate them to system requirements, demonstrate in-depth understanding of use cases for building prototypes and solutions for data processing pipelines.
- Operate in SAFe Agile DevOps principles and methodology in building quality DSP technical solutions. Author and implement automated test scripts as mandates DSP quality requirements.
- Bachelor’s degree or equivalent experience in software or computer science or data engineering.
- 8+ years of overall IT experience.
- 5-7 years of experience in cloud computing and data systems.
- Advanced Python programming skills.
- Expert level in Azure Databricks Spark technology and data engineering (Python) including Delta Live Tables (DLT).
- Experience in design and implementation of secure Azure data solutions.
- In-depth knowledge of the data architecture – infrastructure, network components, data processing
- Proficiency in building data pipelines in Azure Databricks.
- Proficiency in configuration and administration of Azure Databricks workspaces and Databricks Unity Catalog.
- Deep understanding of principles of modern data Lakehouse.
- Deep understanding of Azure system capabilities, data services, and ability to implement security controls.
- Proficiency with enterprise DevOps tools including Bitbucket, Jenkins, Artifactory.
- Experience with DataOps.
- Experience with quality software systems.
- Deep understanding of and experience in SAFe Agile.
- Understanding of SDLC.
- Master’s degree or equivalent.
- Proven healthcare experience.
- Azure Databricks certification.
- Ability to analyze use cases and translate them into system requirements, make data driven decisions
- DevOps's automation tools with JPM/Xena framework.
- Expertise in automated testing.
- Experience in AI and MLs.
- Excellent verbal and written communication skills.
- Ability to travel up to 10% of domestic required.