Regular Data Engineer
- Bangalore, Karnataka
- Permanent
- Full-time
- Design, build, and maintain scalable ETL/ELT data pipelines to support analytics, reporting, and machine learning use cases.
- Develop and optimize data models and warehouse structures in Snowflake to ensure high performance and reliability.
- Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver clean, reliable, and accessible datasets.
- Implement and manage data orchestration workflows (e.g., Airflow, AWS Step Functions) to ensure timely and accurate data delivery.
- Apply best practices in data governance, quality, and security across all pipelines and storage layers.
- Monitor, troubleshoot, and optimize data processes for performance, scalability, and cost efficiency.
- Leverage AWS cloud services to design resilient, scalable, and secure data architectures.
- Use Python and SQL for advanced data transformations, automation, and performance optimization.
- Document data pipelines, models, and workflows to ensure maintainability and transparency.
- Contribute to the continuous improvement of the data engineering platform, adopting new tools, frameworks, and processes."
- Strong proficiency in Python for ETL/ELT development and data pipeline automation.
- Hands-on experience with AWS services (S3, Glue, Lambda, EMR, RDS) for building and managing scalable data solutions.
- Expertise in Snowflake for cloud data warehousing, performance tuning, and large-scale data management.
- Advanced SQL skills for data modeling, transformations, and query optimization.
- Practical experience with dbt for modular, version-controlled data transformations and modeling."
- Experience with data orchestration tools (Airflow, Prefect, AWS Step Functions). • Familiarity with big data frameworks (Apache Spark, Kafka, Kinesis). • Knowledge of CI/CD pipelines and version control (GitHub/GitLab, Jenkins). • Understanding of data modeling techniques (star/snowflake schemas, data lakehouse). • Exposure to monitoring & observability tools (CloudWatch, Datadog, Prometheus). • Awareness of containerization and deployment (Docker, Kubernetes).