Senior Data Engineer
NucleusTeq View all jobs
- India
- Permanent
- Full-time
Experience: 5+ Years
Employment Type: Full-timeJob SummaryWe are looking for a highly skilled Data Engineer with 5+ years of experience in building scalable data pipelines and ETL platforms. The ideal candidate should have strong expertise in Python, Apache Airflow, and data ingestion frameworks, along with hands-on experience in designing robust, modular, and production-grade data systems.Key Responsibilities
- Design, develop, and maintain scalable ETL/data pipeline frameworks
- Build and manage data ingestion pipelines from multiple structured and unstructured sources
- Develop and orchestrate workflows using Apache Airflow
- Implement modular and microservices-based architecture for Python data platforms
- Write efficient and complex SQL queries for data transformation and analysis
- Design and implement REST APIs with proper validation and error handling
- Monitor, troubleshoot, and optimize pipeline performance
- Analyze logs and system performance using tools like Splunk
- Collaborate with cross-functional teams to ensure data quality and reliability
- Strong programming experience in Python
- Hands-on experience with Apache Airflow (DAG creation, scheduling, monitoring)
- Solid understanding of ETL concepts and pipeline design
- Experience working with RDBMS (e.g., Snowflake, PostgreSQL, MySQL, etc.)
- Strong proficiency in writing complex SQL queries
- Experience in building data ingestion frameworks
- Knowledge of REST API development and integration
- Familiarity with microservices architecture and modular design patterns
- Experience with Splunk queries/log analysis for debugging and performance tuning
- Experience with any Data Catalog platform (e.g., Open-metadata, Alation, Collibra, DataHub, etc.)
- Exposure to AWS services (S3, Opensearch service, ALB and K8S etc.)
- Experience with data governance or metadata management tools
- Strong problem-solving and analytical skills
- Ability to work in a fast-paced, scalable data environment
- Good communication and collaboration skills