Senior Data Engineer
FM India
- Bangalore, Karnataka
- Permanent
- Full-time
- Develop solid knowledge of structured and unstructured data sources within each product journey (Underwriting and Risk; Client Service, Sales and Marketing; Claims; Account and Location Engineering) as well as emerging data sources (purchased data sets; external data; tc.)
- Apply data modeling techniques to create efficient, well structured data sets that support analytics, reporting, and downstream consumption.
- Implement ETL / ELT solutions using approved analytics and cloud platforms including Fabric, Synapse Analytics, SQL Server, SSIS, Azure services Data Factory, and data lakes others as required.
- Ensure tables and views are designed for data integrity, efficiency, performance, and ease of comprehension.
- Partner with Data Analytics team members, product owners, developers, solution architects, business analysts, data engineers, data analysts, data scientists and others to understand data and reporting needs
- Develop solutions using data modeling techniques
- Validate code through detailed and disciplined testing
- Participate in peer code review to ensure solutions are accurate
- Ensure data solutions meet standards for accuracy, performance, scalability, and maintainability through disciplined testing and peer review.
- Design and implement the data flow, infrastructure pipelines, ETL/ELT, structured and unstructured data movement and storage solutions into and out of Data Analytics data assets.
- Design, develop, and maintain data pipelines and integrations for structured and unstructured data across enterprise product domains (e.g., Underwriting, Risk, Claims, Client Services, Sales and Marketing).
- Analyze and resolve data quality issues by identifying root causes and implementing corrective actions.
- Support data profiling, cleansing, and transformation activities in partnership with analytics and business teams.
- Monitor pipeline performance, storage capacity, and system reliability; recommend and implement optimizations as needed.
- Provide production support, including timely remediation of issues using appropriate validation and deployment practices.
- Ensure data solutions meet standards for accuracy, performance, scalability, and maintainability through disciplined testing and peer code review.
- Design, develop, and maintain data pipelines and integrations for structured and unstructured data sources across enterprise product domains (e.g., Underwriting, Risk, Claims, Client Services, Sales and Marketing).
- Implement ETL/ELT solutions using approved cloud and analytics platforms (e.g., Synapse, SQL Server, Azure services Data Factory, Fabric, data lakes).
- Analyze and assess reported data quality issues, quickly identifying root causes and implementing corrective actions.
- Support data profiling, cleansing, and transformation activities in partnership with analytics and business teams.
- Monitor pipeline performance, storage capacity, and system reliability; identify and implement optimization opportunities.
- Address production issues quickly using appropriate validation and deployment steps; provide ongoing production support.
- Consult with DBAs and team members on configuration and maintenance of warehouse infrastructure.
- Understand and design data relationships between business and data subject areas
- Follow standards for naming conventions, code documentation and code review
- Partner with product owners, solution architects, business analysts, data analysts, data scientists, and engineers to understand data and reporting needs and translate them into technical solutions.
- Support team members with data cleansing tasks
- Conduct data profiling to identify data anomalies
- Participate actively in hybrid agile delivery practices, including backlog refinement, task estimation, prioritization, and escalation of colliding priorities.
- Assist team members with data preparation tasks
- Support developers, data analysts and data scientists who need to interact with data
- Analyze and assess reported data quality issues, quickly identifying root cause
- Monitor system performance and identify opportunities for optimization
- Monitor storage capacity and reliability
- Address production issues quickly, with appropriate validation and deployment steps
- Provide clear and professional communication to users, management, and teammates
- Provide ad hoc data extracts and analysis to respond to tactical business needs
- Identify work tasks and capture them in the team backlog
- Organize known tasks, following provided prioritization
- Escalate colliding priorities
- Provide production support
- Network with product teams to keep abreast of database changes as well as business process changes which result in data interpretation changes
- 3-5 year of experiencerequired to perform essential job functions.
- Strong SQL skills
- Data modeling abilities
- Relational 3rd Normal Form and non-relational Kimball/Inmon database theory
- Design, build, maintain data lakes and data warehouses
- ETL /ELT design and support
- Programming languages, SQL, Python, Spark, Kafka, Azure Cloud services, JSON
- SQL
- Pyspark / Spark,
- Fabric Platform,
- Data lakes, data warehouses
- Ability to read and create data models
- ETL/ELT Pipeline development experience
- Ability to keep pace with rapidly evolving cloud services and AI technologies
- Demonstrate importance of data governance, quality and observability
- Preferred: Orchestration and automation knowledge for managing complex data workflows
- Communication and collaboration skills
- Relational (3NF) and non-relational (Inmon/Kimball) database theory
- AI/LLM Integration experience
- Azure Data Factory; Kafka, Synapse Analytics Platform