
Senior Data Migration Engineer
- India
- Permanent
- Full-time
- Design, develop, test, and deploy high-performance and scalable data solutions using Python, PySpark, SQL
- Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications.
- Implement efficient and maintainable code using best practices and coding standards.
- Work with Databricks platform for big data processing and analytics.
- Develop and maintain ETL processes using Databricks notebooks.
- Implement and optimize data pipelines for data transformation and integration.
- Utilize AWS services (e.g., S3, Glue, Redshift, Lambda) and Databricks to build and optimize data migration pipelines.
- Leverage PySpark for large-scale data processing and transformation tasks.
- Stay updated on the latest industry trends, tools, and technologies related to Python, SQL, and Databricks.
- Share knowledge with the team and contribute to a culture of continuous improvement.
- Utilize expertise in SQL to design, optimize, and maintain relational databases.
- Write complex SQL queries for data retrieval, manipulation, and analysis.
- Education: Bachelor's degree in Computer Science, Engineering, Data Science, or a related field. Advanced degrees are a plus.
- 4 to 8 Years of experience in Databricks and big data frameworks
- Proficient in AWS services and data migration
- Experience in Unity Catalogue
- Familiarity with Batch and real time processing
- Data engineering with strong skills in Python, PySpark, SQL
- Certifications: AWS Certified Solutions Architect, Databricks Certified Professional, or similar are a plus.
- Strong problem-solving and analytical skills.
- Excellent communication and collaboration abilities.
- Ability to work in a fast-paced, agile environment.
- Design, develop, test, and deploy high-performance and scalable data solutions using Python, PySpark, SQL
- Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications.
- Implement efficient and maintainable code using best practices and coding standards.
- Work with Databricks platform for big data processing and analytics.
- Develop and maintain ETL processes using Databricks notebooks.
- Implement and optimize data pipelines for data transformation and integration.
- Utilize AWS services (e.g., S3, Glue, Redshift, Lambda) and Databricks to build and optimize data migration pipelines.
- Leverage PySpark for large-scale data processing and transformation tasks.
- Stay updated on the latest industry trends, tools, and technologies related to Python, SQL, and Databricks.
- Share knowledge with the team and contribute to a culture of continuous improvement.
- Utilize expertise in SQL to design, optimize, and maintain relational databases.
- Write complex SQL queries for data retrieval, manipulation, and analysis.