Big Data Analyst (Hadoop & Dataproc)
Hadoop/Dataproc Platform Analyst
We are seeking a highly skilled Hadoop/Dataproc Expert to support a strategic initiative focused on the reverse engineering, analysis, and retirement/migration of our existing Hadoop/Google Cloud Dataproc platform. This role is critical to enabling the project team to fully understand the current data ecosystem and prepare for a seamless transition.
Key Responsibilities:
Platform Analysis & Assessment: Conduct a comprehensive review of the Hadoop/Dataproc environment, including cluster configurations, resource usage, and job execution patterns. Identify and document all active components, services, and dependencies.
Data & Metadata Extraction: Pull and catalog all data sources, datasets, and associated metadata across the platform. Map data lineage and relationships to support reverse engineering efforts.
Feed & Log Analysis: Analyze incoming and outgoing data feeds, including batch and streaming pipelines. Review system logs, audit trails, and job histories to identify integration points and data flow patterns.
Collaboration & Documentation: Work closely with data engineers, architects, and project leads to provide insights and technical guidance. Document findings in a structured format to support migration planning and platform decommissioning.
Required Skills & Experience:
-
Proven expertise in Hadoop ecosystem (HDFS, Hive, Pig, Spark, etc.) and Google Cloud Dataproc.
-
Strong understanding of platform management, including cluster operations, job orchestration, and performance tuning.
-
Familiarity with cloud migration strategies and legacy platform retirement best practices.
-
Excellent analytical, communication, and documentation skills.
Recommended Jobs
Retail Sales Worker - Montgomery County Commissary
The Retail Sales Lead Worker coordinates store operations, supporting and training the utility workers and cashiers. Assign the day’s work tasks to the Sales Associates and General Utility Workers; e…
Plumbing Service Technician
Job Description Job Description Description: Summary We are looking for employee’s that offer a positive attitude to support our culture, an innovative thinker to help better our processes, a…
Cycle Counter
Responsibilities: Shift: Tuesday - Saturday 12-9pm Position Responsibilities: Conduct daily and weekly cycle counts, perform full warehouse inventory annually, and ensure accuracy and integ…
Help Desk Analyst are looking for transition their carrer in Oracle HCM Analyst
Oracle HCM Analyst- Entry Level (Career Transition Opportunity) Role: Oracle HCM Analyst Location: US(Remote) Visa Status: US Citizen, Work permit/Citizen/PR Pay Rate: $60/hr- $73/…
Benefits Analyst
At STERIS, we help our Customers create a healthier and safer world by providing innovative healthcare and life science product and service solutions around the globe. Position Summary The Bene…
Painter
Job Description Job Description PAINTERS/SUB-CONTRACTORS NEEDED! Fresh Coat Painters of Canton is looking for detail and quality-oriented professional painters to join our team. We strive to be…
Call Center Quality Monitor (Remote Lawence, KS)
Description & Requirements Maximus is looking for a Remote Call Center Quality Monitor in the Lawrence, KS area. The Call Center Quality Monitor will evaluate recorded and/or transcribed interac…
Mold Setter (2nd or 3rd shift starting at $24.37/hour + shift premium)
PRIMARY FUNCTION: The Injection Mold Setter will be responsible for the efficient and safe setup, changeover and basic troubleshooting of injection molding machines and related auxiliary equipment…
SOUS CHEF (FULL TIME)
We are hiring immediately for full time SOUS CHEF positions. Location : Atrium Medical Center - One Medical Center Drive, Middleton, OH 45005. Note: online applications accepted only . …
Entry Level Automotive Lube/Oil Technician
**WORK FOR THE LARGEST PRIVATELY HELD AUTOMOTIVE DEALER GROUP IN NORTHWEST OHIO** Baumann Ford Oregon has a great opportunity for an Entry Level Lube Service Technician. This role will mainly be …