TECHNICAL ARCHITECT
- Pune, Maharashtra
- Permanent
- Full-time
- Lead the design and implementation of Databricks-based data solutions.
- Architect and optimize data pipelines for batch and streaming data.
- Provide technical leadership and mentorship to a team of data engineers.
- Collaborate with stakeholders to define project requirements and deliverables.
- Ensure best practices in data security, governance, and compliance.
- Troubleshoot and resolve complex technical issues in Databricks environments.
- Stay updated on the latest Databricks features and industry trends.
- Experience in data engineering using Databricks or Apache Spark-based platforms.
- Proven track record of building and optimizing ETL/ELT pipelines for batch and streaming data ingestion.
- Hands-on experience with Azure services such as Azure Data Factory, Azure Data Lake Storage, Azure Databricks, Azure Synapse Analytics, or Azure SQL Data Warehouse.
- Proficiency in programming languages such as Python, Scala, SQL for data processing and transformation.
- Expertise in Spark (PySpark, Spark SQL, or Scala) and Databricks notebooks for large-scale data processing.
- Familiarity with Delta Lake, Delta Live Tables, and medallion architecture for data lakehouse implementations.
- Experience with orchestration tools like Azure Data Factory or Databricks Jobs for scheduling and automation.
- Design and implement the Azure key vault and scoped credentials.
- Knowledge of Git for source control and CI/CD integration for Databricks workflows, cost optimization, performance tuning.
- Familiarity with Unity Catalog, RBAC, or enterprise-level Databricks setups.
- Ability to create reusable components, templates, and documentation to standardize data engineering workflows is a plus.
- Ability to define best practices, support multiple projects, and sometimes mentor junior engineers is a plus.
- Must have experience of working with streaming data sources and Kafka (preferred)
- Bachelor's degree in Computer Science, Data Engineering, or a related field
- Extensive experience with Databricks, Delta Lake, PySpark, and SQL
- Databricks certification (e.g., Certified Data Engineer Professional)
- Experience with machine learning and AI integration in Databricks
- Strong understanding of cloud platforms (AWS, Azure, or GCP)
- Proven leadership experience in managing technical teams
- Excellent problem-solving and communication skills
- Global cutting-edge IT projects that shape the future of digital and have a positive impact on environment.
- Wellbeing programs & work-life balance - integration and passion sharing events.
- Attractive Salary and Company Initiative Benefits
- Courses and conferences
- Attractive Salary
- Hybrid work culture
At Atos, we embrace diversity as the ultimate engine of ingenuity for our clients, and we constantly strive to create a culture where people feel supported and encouraged. Read more about our commitment .Whether it is fighting climate change, promoting digital inclusion, or ensuring trust in data management – tech for good sits at the core of our identity. With numerous global recognitions for our ESG practices, we are committed to building a better future for all by harnessing the power of technology. Learn more