Azure Data Architect

Bangalore, Karnataka
Permanent
Full-time

1 month ago

Overview:Azure Data ArchitectBangaloreAptean is changing. Our ERP solutions are transforming a huge range of global businesses, from food producers to manufacturers. In a world of generic enterprise software, we provide targeted solutions that bring together the very best technology and drive greater results. With over 4500 employees, 90 different products and a global client base, there’s no better time to advance your career at Aptean.Are you ready for what’s next, now? We are! If being part of a dynamic, high growth organization excites you and you are a Senior Data Architect, and eager to learn and grow, then this opportunity is for you! Our fast-paced environment and dynamic global R&D department is eager for a mover and shaker to step into this role and become an integral part of our team.Job Summary:We are looking for a seasoned Data Architect with deep expertise in Spark to lead the design and implementation of modern data processing solutions. The ideal candidate will have extensive experience in distributed data processing, large-scale data pipelines, and cloud-native data platforms.This is a strategic role focused on building scalable, fault-tolerant, and high-performance data systems.Key Responsibilities:

Architect, design, and implement large-scale data pipelines using Spark (batch and streaming).
Optimize Spark jobs for performance, cost-efficiency, and scalability.
Define and implement enterprise data architecture standards and best practices.
Guide the transition from traditional ETL platforms to Spark-based solutions.
Lead the integration of Spark-based pipelines into cloud platforms (Azure Fabric/Spark pools).
Establish and enforce data architecture standards, including governance, lineage, and quality.
Mentor data engineering teams on best practices with Spark (e.g., partitioning, caching, join strategies).
Implement and manage CI/CD pipelines for Spark workloads using tools like GIT or DevOps.
Ensure robust monitoring, alerting, and logging for Spark applications.

Required Skills & Qualifications:

10+ years of experience in data engineering, with 7+ years of hands-on experience with Apache Spark (PySpark/Scala).
Proficiency in Spark optimization techniques, Monitoring, Caching, advanced SQL, and distributed data design.
Experience with Spark on Databricks and Azure Fabric.
Solid understanding of Delta Lake, Spark Structured Streaming, and data pipelines.
Strong experience in cloud platforms ( Azure).
Proven ability to handle large-scale datasets (terabytes to petabytes).
Familiarity with data lakehouse architectures, schema evolution, and data governance.
Candidate to be experienced in Power BI, with at least 3+ years of experience.

Preferred Qualifications:

Experience implementing real-time analytics using Spark Streaming or Structured Streaming.
Certifications in Databricks, Fabric or Spark would be a plus.

If you share our mindset, you can share in our success. To find out more about joining Aptean, get in touch today.Learn from our differences. Celebrate our diversity. Grow and succeed together.Aptean pledges to promote a company culture where diversity, equity and inclusion are central. We are committed to applying this principle as we interact with our customers, build our teams, cultivate our leaders and shape a company in which any employee can succeed, regardless of race, color, sex, national origin, sexuality and gender identity, religion, disability, age, status as a protected veteran or any other group status protected by law. Celebrating our diverse experiences, opinions and beliefs allows us to embrace what makes us unique and to use this as an asset in bringing innovative solutions to our customer base.“At Aptean, our global and diverse employee base is our greatest asset. It is through embracing and understanding our differences that we are able to harness our individual power to maximize the success of our customers, our employees and our company.” – TVN Reddy

Aptean

Apply Now