Data EngineerWe are seeking a highly skilled and motivated Data Engineer to join our growing team. The ideal candidate will have a strong background in building and maintaining scalable data pipelines, with hands-on experience in ETL processes, Python programming, Apache Airflow, Apache Spark, and Neo4j graph database technology. You will play a crucial role in designing, implementing, and managing our data infrastructure to support our data-driven initiatives.Responsibilities:
Design, build, and maintain robust and scalable ETL pipelines to ingest data from a wide variety of sources.
Develop and optimize data processing workflows and data models for efficiency and reliability.
Utilize Apache Spark for large-scale data processing and complex data transformations.
Orchestrate and schedule data pipelines using Apache Airflow , ensuring timely and accurate data delivery.
Write clean, efficient, and well-documented Python code for data manipulation and automation.
Model, implement, and manage graph databases using Neo4j to uncover relationships and insights within our data.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver actionable insights.
Ensure data quality and integrity through rigorous testing and validation.
Monitor, troubleshoot, and resolve issues related to data pipelines and infrastructure.
Stay up to date with emerging technologies and best practices in data engineering.
Qualifications:
Proven experience as a Data Engineer or in a similar role.
Strong proficiency in Python and its data-related libraries (e.g., Pandas, PySpark).
Hands-on experience with ETL concepts and tools.
Demonstrable experience with Apache Spark for distributed data processing.
In-depth knowledge of workflow management and orchestration using Apache Airflow .
Experience with graph databases, specifically Neo4j , including data modelling and Cypher query language.
Solid understanding of SQL and NoSQL databases.
Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and their data services.
Excellent problem-solving and analytical skills.
Strong communication and collaboration abilities
Bachelor’s degree in computer science, Engineering, or a related field.
