
Lead Assistant Manager
- Gurgaon, Haryana
- Permanent
- Full-time
- Design, develop, and maintain scalable and efficient data pipelines using PySpark and Databricks.
- Perform data extraction, transformation, and loading (ETL) from diverse structured and unstructured data sources.
- Write and optimize complex SQL queries for high performance and scalability across large datasets.
- Build and maintain interactive dashboards and data visualizations using Plotly Dash or similar frameworks.
- Collaborate closely with data scientists, analysts, and business stakeholders to gather and understand data requirements.
- Ensure data quality, consistency, and integrity throughout the data lifecycle using validation and monitoring techniques.
- Develop and maintain modular, reusable, and well-documented code and technical documentation for data workflows and processes.
- Implement data governance, security, and compliance best practices.
- 5+ years of relevant experience in Data Engineering tools
- Programming Languages: Python and SQL
- Python Frameworks: Plotly Dash, Flask, Fast API
- Data Processing Tools: pandas, NumPy, PySpark
- Cloud Platforms: Databricks (for scalable computing resources)
- Version Control & Collaboration: Git, GitHub, GitLab
- Deployment and Monitoring: Databricks ,Docker, Kubernetes