Apache Spark Senior Technical Lead

Chennai, Tamil Nadu
Permanent
Full-time

13 days ago
Apply easily

Company DescriptionAbout Sopra Steria
Sopra Steria, a major Tech player in Europe with 50,000 employees in nearly 30 countries, is recognised for its consulting, digital services and solutions. It helps its clients drive their digital transformation and obtain tangible and sustainable benefits. The Group provides end-to-end solutions to make large companies and organisations more competitive by combining in-depth knowledge of a wide range of business sectors and innovative technologies with a collaborative approach. Sopra Steria places people at the heart of everything it does and is committed to putting digital to work for its clients in order to build a positive future for all. In 2024, the Group generated revenues of €5.8 billion.
The world is how we shape it.Job DescriptionJob Title: Technical Lead/Senior Technical LeadLocation: ChennaiTotal Experience Expected: 06-12 yearsRole Overview:We are looking for a skilled Data Engineer to build and maintain scalable data pipelines, manage large datasets, and support analytics initiatives. This role involves working with Big Data technologies like Spark, Scala, SQL/PLSQL, and cloud-ready environments.Experience: * 8+ years of relevant experience in data engineering / big data development

Proven track record of handling large-scale data solutions

Key Responsibilities: * Maintain, develop, and design data ingestion and transformation pipelines using Spark and Scala

Process and optimize SQL and PL/SQL queries for large datasets
Integrate and process data from multiple sources for analytics and reporting
Collaborate with BI and Analytics teams to support reporting and dashboard needs
Ensure data quality, performance, and security
Provide innovative solutions to complex business/technical problems
Identify and resolve technical, integration and development issues
Implement best practices, standards and processes to ensure quality of the final product
Responsible for development of features independently
Continuous build, automated testing, and release management
Participate in status meetings and provide regular updates

Required Skills:Technical: * Expertise in Spark and Scala for large-scale data processing

Proficiency in Python and SQL, including hands-on experience with PL/SQL
Experience in data integration/ETL workflows;
Strong knowledge of relational databases such as Oracle and PostgreSQL

Professional Competencies: * Excellent verbal and written communication skills

Result-oriented with strong analytical and problem-solving skills
Ability to work in a fast-paced, agile, and collaborative environment

Desired Skills: * Exposure to cloud data services (AWS preferred)

Proficiency in Linux/Unix shell scripting for automation and data operation
Experience working in Agile environments – Scrum and/or SAFe frameworks
Exposure to DevOps culture and practices, including CI/CD and automation
Understanding of software engineering lifecycle – Development, Testing, and Production Support
Experience in production issue handling and transformation support management

Good to Have Skills: * Exposure to Dataiku preferred

Awareness of cloud and big data architecture principles

QualificationsBE / B.Tech/ MS/ M.Tech/ MCAAdditional InformationAt our organization, we are committed to fighting against all forms of discrimination. We foster a work environment that is inclusive and respectful of all differences.All of our positions are open to people with disabilities.

Sopra Steria