GCP Data Engineer - Chennai
- Chennai, Tamil Nadu
- Permanent
- Full-time
Key Responsibilities:
- Design and implement scalable and secure data pipelines using GCP-native tools such as:
- Cloud Dataflow, Cloud Dataproc, BigQuery, Cloud Composer
- Build and manage ETL/ELT workflows to ingest data from various sources (structured and unstructured)
- Optimize and monitor data pipelines for performance and cost
- Ensure data quality and integrity across the data lifecycle
- Collaborate with data scientists, analysts, and application teams for data access and transformation needs
- Implement CI/CD for data engineering pipelines using tools like Cloud Build or Jenkins
- Develop and maintain data models and schemas (dimensional or normalized)
- Document data flows, architecture, and metadata for internal usage
- Apply best practices in data security, access control, and compliance (e.g., GDPR, HIPAA)
- 3+ years of experience in data engineering, with at least 2+ years on GCP
- Strong knowledge and hands-on with GCP services:
- BigQuery, Cloud Storage, Pub/Sub, Dataflow, Dataproc, Cloud Functions
- Proficiency in SQL, Python, and Java or Scala (preferred)
- Experience with orchestration tools such as Apache Airflow or Cloud Composer
- Experience with real-time data streaming and batch data processing
- Good understanding of data warehousing, data lake architecture, and data governance
- Familiarity with Terraform, Infrastructure as Code (IaC) is a plus
- GCP certifications such as Professional Data Engineer or Associate Cloud Engineer are desirable