Data Engineer
Job Summary:
We are seeking a skilled and motivated Data Engineer with strong expertise in PySpark, SQL to join our Data Engineering team. You will be responsible for designing, building, and maintaining scalable data pipelines and processing systems that support our business intelligence, analytics, and machine learning initiatives.
Key Responsibilities:
- Develop and maintain scalable, robust data pipelines using Scala and big data technologies.
- Work with large datasets from multiple sources to ingest, transform, and make data available for analytics and reporting.
- Collaborate with Data Scientists, Analysts, and other engineers to understand data requirements and deliver efficient solutions.
- Optimize ETL jobs for performance and cost.
- Ensure data quality, governance, and consistency across all environments.
- Monitor production jobs, troubleshoot issues, and ensure system reliability.
- Implement best practices for data engineering, including code reviews, testing, and documentation.
Required Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
- 6+ years of experience as a Data Engineer or in a similar role.
- Strong skills in PySpark and SQL (experience with functional programming is a plus).
- Hands-on experience with big data tools like Apache Spark , Kafka , Hadoop , or Hive .
- Proficiency in building ETL pipelines and working with structured and unstructured data.
- Experience with cloud platforms (e.g., AWS , Azure , GCP ) and their data services.
- Familiarity with version control systems (e.g., Git ), CI/CD, and DevOps practices.
- Solid understanding of data warehousing and data modeling concepts.
Preferred Qualifications:
- Knowledge of SQL and database systems such as Postgres, Redshift, or Snowflake.
- Exposure to containerization technologies like Docker and orchestration tools like Airflow or Kubernetes.
Recommended Jobs
Product Manager
Who We Are: Headquartered in New York City, Take-Two Interactive Software, Inc. is a leading developer, publisher, and marketer of interactive entertainment for consumers around the globe. We de…
QA Automation Engineer IV
Job description Greetings We’re looking for a highly experienced QA Automation Engineer IV to join our remote team and play a key role in delivering quality software through automation, conti…
Full Stack Java Developer - On-Site Coordinator
What We’re Looking For We’re seeking an experienced Full Stack Java Developer - On-Site Coordinator with strong backend and UI development skills. You’ll be a key contributor to enterprise-grade a…
Food Prep Worker
The Food Prep Worker prepares, and portions food items and performs various duties in accordance with production requirements and Aramark sanitation standards. Uses daily worksheets and standardized …
Postdoctoral Associate- Disease Resistance in Planaria
Summary A funded postdoctoral position in stem cell biology and regeneration is available in the lab of Dr. Blair Benham-Pyle at Baylor College of Medicine in Houston, TX. The Benham-Pyle laborato…
Service Technician
This opening is shared across our multiple locations: Brinson Ford of Corsicana - 1951 St HWY 31 E Corsicana, TX 75110. Brinson CDJR - 1971 St HWY 31 E Corsicana, TX 75110. Brinson Chevrolet - 825 E F…
Mechanical Engineer
Mechanical Engineer The ideal candidate will assist in various engineering projects, contributing to design, analysis, and implementation processes. This role is ideal for individuals seeking to d…
Cleared Software Engineer
The Company Jericho Security is the ultimate battleground where cyberdefense clashes with the constantly evolving menace of generative artificial intelligence. The threat of malicious AI attacks …
Full Time Family Practice Job TX
A health system in southwest Texas is seeking a BE/BC Family Medicine physician to provide Clinic services in an established practice.This opportunity is truly exceptional, providing a superior stand…
Senior Embedded Software Developer
METECS is an engineering and applied technology company that provides high-fidelity simulation and software development for NASA's Johnson Space Center. We are seeking a Senior Embedded Software Deve…