Senior Big Data Engineer (Java/Scala)
- Bangalore, Karnataka
- Permanent
- Full-time
- Choosing the right technologies for our use cases, deploy and operate.
- Setting up Data stores structured, semi structured and non-structured.
- Secure data at rest via encryption
- Implement tool to access securely multiple data sources
- Implement solutions to run real-time analytics
- Use container technologies
- 6+ years of relevant technology experience
- Programming Skills: o Strong proficiency in Java or Scala. o Solid understanding of SQL and experience working with structured, semi-structured, and unstructured data.
- Big Data & Data Engineering: o Deep experience with Big Data frameworks such as Apache Spark, Hadoop, and Kafka. o Hands-on experience in designing and managing data pipelines, ETL workflows, and real-time streaming applications. o Familiarity with data warehousing solutions (e.g., Hive, Trino, BigQuery, Redshift, Snowflake).
- Architecture & Infrastructure: o Background in microservices-based architectures and API integrations. o Hands-on expertise with Kubernetes and Docker for containerization and orchestration.
- Cloud & DevOps: o Experience with cloud platforms like AWS, Google Cloud Platform (GCP), or Azure, including services such as EMR, Dataproc, or BigQuery. o Understanding of data security best practices, including encryption and secure access controls.
- Development Practices: o Knowledge of Agile/Scrum methodologies.
- Soft Skills: o Team-oriented, respectful, and able to collaborate effectively in cross-functional teams. o Strong problem-solving skills and a growth mindset with a willingness to learn and adapt.