Principal Data Platform Engineer

NationsBenefits

  • Hyderabad, Telangana
  • Permanent
  • Full-time
  • 1 month ago
Job Title: Principal Data Platform EngineerLocation: Hyderabad, IndiaEmployment Type: Full-timeAbout NationsBenefits:At NationsBenefits, we're committed to improving health outcomes and member satisfaction through innovative supplemental benefits and healthcare solutions. Behind the scenes, our global engineering teams build and maintain robust systems that support millions of members nationwide. As we continue to scale, we're investing in our data platform capabilities to ensure our engineers and analysts have the infrastructure, governance, and tooling they need to deliver data products efficiently and compliantly.We're seeking a Principal Data Platform Engineer to help evolve and elevate the data infrastructure, frameworks, and practices that power our data-driven organization. In this role, you'll be a key contributor to an established platform -- refining what works, optimizing what can be better, and guiding our teams toward data excellence at scale.About the RoleWe're seeking a Principal Engineer with deep expertise in modern data platforms to help us implement and evolve our data platform vision. This role isn't about fixing a broken system---far from it. Our data platform is stable, performant, and used by hundreds of data consumers daily. But we're ready to take it to the next level through strategic architecture and forward-thinking data infrastructure.This is a high-impact, strategic role focused on designing and implementing robust, compliant data architectures that align with our established data domains. You'll lead the charge in architecting scalable data flow processes, implementing modern data infrastructure patterns, and building the foundation for our next generation of data products.The successful candidate will not only be able to find opportunities for improvement, but will understand how those fit into a larger enterprise architecture and how they can be implemented without introducing defects or downtime in our 24x7 system. You will be a driven self-starter who will take ownership of problems from investigation to resolution. Moreover, you will do this while partnering with other senior engineering leaders and cross-functional teams to share both their knowledge of what has been done (and why) and your knowledge of what can be done.What You'll Do
  • Leverage end-to-end visibility of our data platform stack (Databricks, Unity Catalog, SQL Server, Airflow, EventHub/Kafka) to design and implement next-generation data flow architectures
  • Act as a subject matter expert for data platform architecture and compliance frameworks within HIPAA and multitenant environments
  • Design and implement scalable data flow processes across all platform services, ensuring optimal performance and governance
  • Lead strategic initiatives to modernize data infrastructure patterns, including:
  • Cross-service data flow optimization and orchestration
  • Modern data storage and access patterns
  • Integrated data cataloging and lineage systems
  • Performance bottleneck identification and resolution across the platform
  • Design and implement data architectures within established data domains, determining optimal service deployment strategies (SQL databases, search indexes, data warehouses, caches)
  • Collaborate across engineering, analytics, information security, and DevOps teams to create actionable platform improvements and drive those plans through the SDLC process to production
  • Act as a change agent, championing modern best practices in data platform design, governance, and compliance
  • Mentor data engineers and cross-functional team members through code reviews, architectural guidance, and knowledge sharing
  • Establish and maintain data platform standards, guidelines, and best practices across the organization
  • Produce and present detailed findings and architectural plans to technical stakeholders and leadership
  • Maintain a balance between ideal solutions and pragmatic tradeoffs to keep the system stable while evolving it
  • Take ownership of data domain architecture decisions, working closely with domain owners to optimize their data strategies
What We're Looking For
  • 7+ years of software engineering experience, with significant hands-on expertise in Python and modern data stack technologies
  • Proven experience architecting and optimizing data platforms using Databricks, Unity Catalog, and cloud data services in high-throughput environments
  • Strong understanding of data modeling, governance frameworks, and compliance requirements (HIPAA experience highly preferred)
  • Experience with Infrastructure as Code (Terraform), orchestration tools (Airflow), and streaming platforms (Kafka/EventHub)
  • A track record of leading data platform initiatives across teams and organizations
  • Experience identifying performance issues through profiling tools, logs, and query optimization
  • A holistic systems thinker who can comfortably navigate between infrastructure, application, and governance layers
  • Excellent communication and mentoring skills -- you know how to drive change through influence and technical leadership
Why You'll Love Working Here
  • Work on a system that's already working well – and help make it even better.
  • Collaborate with teams who value thoughtful engineering, technical rigor, and clean architecture.
  • Be empowered to make a real difference through strategic guidance and technical insight.
  • Join a culture that values long-term thinking, knowledge sharing, and continuous improvement.

NationsBenefits