Data Architect/Senior Data Engineer - Databricks, AWS, Pyspark, S3
- Thiruvananthapuram, Kerala
- Permanent
- Full-time
- Designing and implementing scalable data solutions on AWS cloud platform
- Developing and maintaining data pipelines using Databricks and pyspark
- Collaborating with cross-functional teams to understand data requirements and deliver effective solutions
- Ensuring data quality and integrity through data validation and testing processes
- Optimizing data workflows for improved performance and efficiencyQualifications:
- Bachelor's degree in Computer Science, Engineering, or related field
- 6+ years of experience working as a Data Engineer with expertise in AWS, Databricks, Pyspark and S3
- Strong understanding of data architecture, data modeling, and data warehousing concepts
- Experience with ETL processes, data integration, and data transformation
- Excellent problem-solving skills and ability to work in a fast-paced environmentIf you are a talented AWS Data Engineer looking to make a significant impact in the MedTech industry, we encourage you to apply for this exciting opportunity at Quest Global.Work ExperienceRequired Skills and Experience
- Experience in implementing Cloud based analytics solutions in Databricks(AWS), S3 etc.
- Scripting experience in building data processing pipelines in Databricks (pySpark).
- Strong knowledge of Data Platform and Cloud (AWS) eco-systems, in developing, designing, and operating application development platforms.
- Working experience with AWS Native services – DynamoDB , Glue, MSK, S3, Athena, Cloud watch, lambda, IAM etc
- ETL development and Analytics Applications development.
- Experience in migration of Data from Excel, Flat file etc
- Exposure to all stages of SDLC primarily in Database Architecture, Logical and Physical database modelling
- Strong SQL development skills including query tuning and performance optimization.
- Strong development skills in Python , pySpark etc
- Experience in writing unit test cases using PyTest or any other tools