Lead Data Integration Engineer(Snowflake)
DTCC
- Chennai, Tamil Nadu
- Permanent
- Full-time
- Competitive compensation, including base pay and annual incentive.
- Comprehensive health and life insurance and well-being benefits, based on location.
- Pension / Retirement benefits
- Paid Time Off and Personal/Family Care, and other leaves of absence when needed to support your physical, financial, and emotional well-being.
- DTCC offers a flexible/hybrid model of 3 days onsite and 2 days remote (onsite Tuesdays, Wednesdays and a third day unique to each team or employee).
- Design, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.
- Gather requirements and business process knowledge to transform the data in a way that's geared towards the needs of end users.
- Creates logical and physical data flow models for ETL applications.
- Ensure that the data architecture is scalable and maintainable.
- Work with data modelers and business analysts to understand project requirements and convert them technical document and Jira stories.
- Analyze data to identify potential issues within ETL pipelines, notify end-users and propose adequate solutions.
- Document procedures for ETL pipelines and work with stake holders for engagement and approval.
- Automate data processes for event driven and time based schedule using orchestration and workflow tools.
- Minimum of 7 years of related experience
- Bachelor's degree and/or equivalent experience
- Experience with developing SQL queries and PL/SQL scripts in cloud warehouse like Snowflake or similar technologies
- Strong understanding on cloud warehouse architecture and services in AWS or similar cloud platform
- Optimization and performance tuning of data processing workflows for improved efficiency
- Knowledge of building data pipelines using Python using pandas data frame and SQL connectors
- Preferred knowledge of Java and developing application for reading data from message queues
- Knowledge of data warehouse principles like star schema model using slowly changing dimension and facts
- Experience in code deployment and release management policies and procedures
- Experience is scheduling jobs using Autosys or similar tools
- Strong soft skills including attention to detail, verbal and written communication, team oriented and self-motivated