Job Description:UST is looking for a highly skilled Azure Data Engineer with 5 to 10 yrs of exp to join our team and play a key role in designing and implementing efficient data solutions on the Azure cloud platform. The ideal candidate should have a strong background in data engineering, possess expertise in Azure services, and demonstrate proficiency in data processing technologies, particularly PySpark.Responsibilities:Data Pipeline Development:· Design, implement, and optimize end-to-end data pipelines on Azure, focusing on scalability and performance.· Develop and maintain ETL workflows for seamless data processing.Azure Cloud Expertise:· Utilize Azure services such as Azure Data Factory, Azure SQL Database, and Azure Databricks for effective data engineering.· Implement and manage data storage solutions on Azure.Data Transformation with PySpark:· Leverage PySpark for advanced data transformations, ensuring high-quality and well-structured output.· Implement data cleansing, enrichment, and validation processes using PySpark.Performance Optimization:· Optimize data pipelines, queries, and PySpark jobs to enhance overall performance and scalability.· Identify and address performance bottlenecks within data processing workflows.Requirements:· Proven experience as a Data Engineer, emphasizing expertise in Azure.· Proficiency in Azure Data Factory and other relevant Azure services.· Expertise in PySpark for data processing and analytics is a must.· Experience with data modeling, ETL processes, and data warehousing.Skills:Azure Data Factory, Pyspark, Azure DatabricksAbout Company:UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.
