Data Engineer (Python & PySpark Focus)
Data Engineer (Python & PySpark Focus)
We are seeking a highly motivated and experienced Data Engineer to join our team focusing on building optimizing and deploying robust scalable data solutions. The ideal candidate will possess deep expertise in Python and PySpark to drive complex data transformations and support high-volume performance-critical simulation initiatives.
Key Responsibilities
* Design build and maintain high-performance ETL/ELT data pipelines using Python and PySpark.
* Apply expertise in Pythons data analysis libraries including Pandas and NumPy to perform complex data manipulation cleansing and transformation.
* Develop and manage data processing jobs leveraging PySpark for distributed computing across large-scale datasets.
* Implement DevOps practices and tooling (e.g. Docker Jenkins Terraform CloudFormation) for the automated deployment and orchestration of Python applications and data pipelines.
* Collaborate with data scientists and analysts to ensure data quality availability and consistency for advanced modeling and reporting.
* Utilize AWS or other cloud services (e.g. S3 Glue EMR Snowflake) to architect and maintain cloud-based data ecosystems.
* Write and optimize complex SQL queries for data extraction integrity checks and performance tuning.
Required Technical Skills
* 5 years of experience in Data Engineering or a related technical field.
* Expert-level proficiency in Python including a strong command of core concepts and specialized data libraries (Pandas NumPy).
* Solid hands-on experience with PySpark for building scalable data workflows.
* Strong background in DevOps principles and tools for deploying Python-based data applications (e.g. containerization CI/CD).
* Experience with cloud platforms (AWS strongly preferred) and associated data services (e.g. AWS Glue S3 Lambda Snowflake).
* Advanced knowledge of SQL and experience with modern data warehousing and database performance tuning.
* Familiarity with distributed data processing technologies (e.g. Apache Spark Hadoop).
Required Skills :
Basic Qualification :
Additional Skills :
Background Check : No
Drug Screen : No
Recommended Jobs
Appliance Service Technician
The CSC ServiceWorks Story: At CSC ServiceWorks, we make everyday life easier for 40 million people across the U.S. and Canada. With nearly 100 years of experience and a team of 2,300 professionals, …
Sales Manager
About Milan Laser Hair Removal Milan Laser Hair Removal is one of the nation’s premier laser hair removal providers. That’s because we use the top of the line laser, and all our treatments are perfor…
Production Operator/ 20hr/ 1st And 3rd Shift
Job Title: Production Operator/20hr/ 1st and 3rd shift Job Description We are seeking a dedicated Production Operator to oversee multiple operations on a production line. This role involves cro…
Disassembler (Medical Equipment)
Snelling Staffing is hiring a Disassembler (Medical Equipment) for a company in Canton, OH ~ Weekly Pay ~$16 an hour ~ Must have knowledge of hand tools ~ Dayshift 8am -530p Duties inclu…
Product Manager - Clean Room (P4380)
84.51° Overview: 84.51° is a retail data science, insights and media company. We help The Kroger Co., consumer packaged goods companies, agencies, publishers and affiliates create more personalize…
Heavy Equipment Sales Specialist
Who We Are Earthborne Equipment has proudly served the Warrington, PA and surround areas for over 49 years. We are a highly successful and fast-growing dealership. We provide our customers with an e…
Home Health Aide / Caregiver
Nova Home Health Services is seeking compassionate Home Health Aides and/or Caregivers to provide assistance with daily living and personal care services in a home setting in accordance with an estab…
Software Engineer, Data Platform
Ready to be pushed beyond what you think you're capable of? At Coinbase, our mission is to increase economic freedom in the world. It's a massive, ambitious opportunity that demands the best of us, ev…
Senior Product Manager, Category Marketing
What Category Marketing contributes to Cardinal Health The Category Marketing Manager is responsible for developing strategy and process for supplier marketing that grows category growth and mar…
Food Service Worker I - The University of Akron-Admin
Job Description The Food Service Worker I is responsible for performing various food and beverage-related tasks. This position requires individuals with a high attention to detail, strong dedicati…