Bigdata, pyspark
- Pune, Maharashtra
- Permanent
- Full-time
Expertise in Apache Spark .
Database modelling and working with any of the SQL or NoSQL Database is must.
Working knowledge of Scripting languages like shell/python.
Experience of working with Cloudera is Preferred
Orchestration tools like Airflow or Oozie would be a value addition.
Knowledge of Table formats like Delta or Iceberg is plus to have.
Working experience of Version controls like Git, build tools like Maven is recommended.
Having software development experience is good to have along with Data Engineering experience.
Big Data and Java – QA (Testing) lead /QA - Bigdata Spark/Scala and Java exp 1. Proficient in Hadoop, PySpark, Python and and related technologies
2. Deep technical knowledge of modern, distributed computing frameworks and platforms,
3. Advanced working knowledge in implementing data pipelines on top of cloud technology stacks/On-premise
3. Strong knowledge of SQL and performance tuning
4. Solid knowledge of Linux environments
5.Orchestration with Apache Nifi, Apache Airflow
Nice to have knowledge or prior experience with any of the following
1. Working knowledge of implementing CI / CD pipelines.
2. Log analysis and monitoring using Splunk
3. Databricks, AWS, Snowflake