Senior Data Engineer
Emmes Global
- Bangalore, Karnataka
- Permanent
- Full-time
- Data for AI:
- Design and implement data pipelines specifically tailored for AI training and model development.
- Partner with AI data scientists and engineers to understand their data needs and ensure efficient data access.
- Implement data pre-processing and feature engineering techniques for optimal model performance.
- AWS expertise:
- Design,develop, and maintain data infrastructure on the AWS cloud platform, leveraging services like S3,Redshift, Glue, Lambda, and Kinesis.
- Build and maintain data lakes and warehouses optimized for AI workloads.
- Implement automated data transformations,cleansing, and validation processes to ensure data quality.
- Coding proficiency:
- Write code using Python,SQL, and other relevant programming languages, with a focus on libraries and frameworks commonly used in AI (e.g., TensorFlow, PyTorch).
- Automation and monitoring:
- Automate data pipelines using CI/CD tools and related techniques.
- Monitor and troubleshoot data pipelines for performance, reliability,and data quality issues.
- 5 to 8 years of experience as a Data Engineer or similar role,with a demonstrated experience in supporting data pipelines for AI applications and their data requirements.
- Proven experience with AWS services,including S3, Redshift, Glue, Lambda, and Kinesis.
- Strong experience with Python and SQL.
- Strong understanding of AI libraries and frameworks.
- Experience with data warehousing and data modeling concepts.
- Excellent problem-solving and analytical skills.
- Ability to work independently and as part of a collaborative team.
- Strong communication and collaboration skills.