Consultant, Data Engineering - India
Varite
- Bangalore, Karnataka
- Contract
- Full-time
An American technology company that develops sells, repairs, and supports computers and related products and services. Products include personal computers (PCs), servers, data storage devices, network switches, software, computer peripherals, HDTVs, cameras, printers, and electronics. It is now expanding from offering computers only to delivering a range of technology for enterprise customers, like data storage, information security, virtualization, analytics, and cloud computing. Currently ranked 31st on the Fortune 500 list.About The Job:
ACCOUNTABILITIES
- Develops technical tools and programming to cleanse, organize and transform data and to maintain, protect and update data structures and integrity on an automated basis.
- Applies data extraction, transformation, and loading techniques in order to tie together large data sets from a variety of sources.
- Partners with both internal & external sources to design, build and oversee the deployment and operation of technology architecture, solutions and software.
- Designs, develops and programs methods, processes and systems to capture, manage, store and utilize structured and unstructured data to generate actionable insights and solutions.
- Responsible for the maintenance, improvement, cleaning, and manipulation of data in the businesss operational and analytics databases.
- Proactively analyzes and evaluates the businesss databases in order to identify and recommend improvements and optimization.
- Uses knowledge of existing and emerging data science engineering principles, theories, and techniques to inform business decisions; and produce accurate business insights.
- Completes projects and assignments of moderate scope and complexity under normal supervision to ensure customer and business needs are met.
- Applies discretion and independent judgement to interpret data trends and summarize data insights.
- Assists in the preliminary data exploration, data preparation for accurate model development.
- Establishes working relationships with others outside area of Data Science Engineering expertise.
- Prepares presentations of project outputs for external customers with assistance.
- Experience in supporting large scale data engineering projects
- Leads or participates in the systems software development lifecycle, which includes research, new development, modification, security, correction of errors, reuse, re-engineering and maintenance of software products and data pipelines
- Develops technical tools and programming to cleanse, organize and transform data and to maintain, protect and update data structures and integrity on an automated basis.
- Design & develop processes, automations and systems to capture, manage, store and utilize structured and unstructured data to generate actionable insights and solutions
- Responsible for the maintenance, improvement, cleaning, and manipulation of data in the data lake.
- Attend to data job failures and data issues reported by users
- Proactively analyzes and evaluates the data pipeline failures and triage issues appropriately based on the SLAs
- Work with engineering teams to review solutions for operational ease & are built per standards.
- Works in collaboration with data engineers, data scientists, architects and business users to support the data needs and create actionable insights to identify data quality issues.
- Designs, codes, tests and debugs software according to Client’s standards, policies and procedures.
- Mentoring junior team members on technical and functional skills. Should be a great team player. Functional knowledge of business processes is required.
Minimum Requirement
- 8+ Years of experience in Data Engineering/ETL/Dashboarding/Data warehousing
- Hands on experience on both structured and unstructured data
- 5+ years hands on experience in the below tools
- Spark, Java, Iceberg, Kafka, ECS, Python
- StreamSets, Informatica, GPSS, TDAM, ANSI SQL
- Possesses and applies a broad knowledge of application programming processes and procedures to the completion of complex assignments.
- Competent to analyze diverse and complex problems.
- Advanced ability to effectively troubleshoot program errors.
- Build high reliability, high quality, high volume data pipelines
- Setup batch, micro batch, streaming pipelines
- Data ingestion, transformation, processing – batch & near real time
- Automated tests and tie outs, self-healing data jobs
- Build products that can support themselves with none to minimal support after rollout
- Ability to communicate complex insights in a precise and actionable manner
- Mindset to think differently; alignment to Industry standards; awareness of emerging technologies and industry trends
If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions. VARITE offers a Candidate Referral program, where you'll receive a one-time referral bonus based on the following scale if the referred candidate completes a three-month assignment with VARITE.Experience Level Bonus Referral: 0-2 years INR 5,000
2-6 years INR 7,500
6+ years INR 10,000About VARITE: VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA. VARITE is currently a primary and direct vendor to the leading corporations in the verticals of Networking, Cloud Infrastructure, Hardware and Software, Digital Marketing and Media Solutions, Clinical Diagnostics, Utilities, Gaming and Entertainment, and Financial Services.Equal Opportunity Employer:
VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.