Databricks Automation & AI Platform Engineer

Xenon7

Hyderabad, Telangana
Permanent
Full-time

2 months ago
Apply easily

About us:Where elite tech talent meets world-class opportunities!At Xenon7, we work with leading enterprises and innovative startups on exciting, cutting-edge projects that leverage the latest technologies across various domains of IT including Data, Web, Infrastructure, AI, and many others. Our expertise in IT solutions development and on-demand resources allows us to partner with clients on transformative initiatives, driving innovation and business growth. Whether it's empowering global organizations or collaborating with trailblazing startups, we are committed to delivering advanced, impactful solutions that meet today’s most complex challenges.We are building a community of top-tier experts and we’re opening the doors to an exclusive group of exceptional AI & ML Professionals ready to solve real-world problems and shape the future of intelligent systems.RequirementsRole OverviewWe are seeking a senior Databricks Automation & AI Platform Engineer with deep expertise in Python, Databricks platform components, and AI/MLOps/LLMOps workflows. The engineer will support the Next‑Gen Platform Integration team by building automation frameworks, enabling scalable deployment workflows, and integrating AI capabilities across Databricks environments.This role requires strong systems‑level Python knowledge, Databricks governance experience, and hands‑on AWS automation skills.Key Responsibilities🔹 Databricks Automation & Integration

Build automation workflows for MLOps, LLMOps, and application deployment within Databricks.
Enhance workspace onboarding automation including Unity Catalog, permissions, and environment setup.
Develop reusable modules for workspace provisioning, catalog configuration, model deployment, and governance workflows.
Integrate Mosaic AI components (Gateway, Model Serving, Agents) into platform automation.

🔹 Platform Engineering & Deployment

Develop Python automation scripts for Databricks services including:

Unity Catalog
MLflow
Mosaic AI
Model Serving
Databricks Apps
Ensure consistency, reliability, and scalability across multi‑workspace environments.
Implement automated rollback strategies, fail‑fast mechanisms, and environment stability checks.

🔹 AWS Integration & Automation

Build integrations using AWS Lambda, API Gateway, and Service Principal authentication.
Automate Databricks Job orchestration, monitoring, and deployment pipelines.
Implement secure, scalable automation bridging AWS and Databricks.

Required Skills🔹 Technical Skills

Strong hands‑on Python experience including:

GIL behavior
Multiprocessing vs multithreading
Memory overhead trade‑offs
Data structure time complexities
Deep understanding of Databricks ecosystem:

Unity Catalog governance
Delta architecture (Bronze/Silver/Gold)
MLflow tracking & model lifecycle
Mosaic AI Gateway & LLMOps workflows
Model Serving & Databricks Apps
Experience with Docker image immutability, container state management, and environment reproducibility.

🔹 Cloud & Automation Skills

AWS Lambda, API Gateway, IAM, and automation pipelines.
Service Principal‑based authentication for secure Databricks automation.
Experience with platform engineering, deployment automation, and production stability patterns.

Ideal Candidate Profile

6+ years of Python development and cloud automation experience.
Hands‑on Databricks platform engineering and governance expertise.
Strong understanding of MLOps/LLMOps and AI deployment workflows.
Experience working with platform, ML, and data engineering teams.
Immediate availability preferred.

Xenon7