Data Engineer 2

My3Tech
Austin, TX

Position: Data Engineer 2

Location: Austin, TX 78751 (Remote within TX)

Duration: 9 months Contract with possibility to extension

Seeking Data Engineer 2 who will be responsible for developing, maintaining, and optimizing big data solutions using the Databricks Unified Analytics Platform. This role supports data engineering, machine learning, and analytics initiatives within this organization that relies on large-scale data processing.

Duties include:

  • Designing and developing scalable data pipelines
  • Implementing ETL/ELT workflows
  • Optimizing Spark jobs
  • Integrating with Azure Data Factory
  • Automating deployments
  • Collaborating with cross-functional teams
  • Ensuring data quality, governance, and security.

Candidate Skills and Qualifications:

Years

Required/Preferred

Experience

4

Required

Implement ETL/ELT workflows for both structured and unstructured data

4

Required

Automate deployments using CI/CD tools

4

Required

Collaborate with cross-functional teams including data scientists, analysts, and stakeholders

4

Required

Design and maintain data models, schemas, and database structures to support analytical and operational use cases

4

Required

Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses

4

Required

Implement data validation and quality checks to ensure accuracy and consistency

4

Required

Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging

4

Required

Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices

4

Required

Proficiency in Python and R programming languages

4

Required

Strong SQL querying and data manipulation skills

4

Required

Experience with Azure cloud platform

4

Required

Experience with DevOps, CI/CD pipelines, and version control systems

4

Required

Working in agile, multicultural environments

4

Required

Strong troubleshooting and debugging capabilities

3

Required

Design and develop scalable data pipelines using Apache Spark on Databricks

3

Required

Optimize Spark jobs for performance and cost-efficiency

3

Required

Integrate Databricks solutions with cloud services (Azure Data Factory)

3

Required

Ensure data quality, governance, and security using Unity Catalog or Delta Lake

3

Required

Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL

3

Required

Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake

1

Preferred

Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow)

1

Preferred

Databricks Certified Associate Developer for Apache Spark

1

Preferred

Azure Data Engineer Associate

---

Posted 2025-10-17

Recommended Jobs

Full Time Radiology Job Plano, TX

CompHealth CompHealth
Plano, TX

This opportunity is in Plano, located 19 miles north of downtown Dallas, It is the 9th-largest city in Texas and 4th-largest city in the Dallas/Fort Worth region. Plano is home to many leading global…

View Details
Posted 2025-09-10

General Manager

Fuzzy's Taco Shop
Tyler, TX

Fuzzy's Taco Shop is hiring a Highly Skilled General Manager Are you a dynamic leader who thrives in a fast-paced environment? Do you enjoy working with a fun and loving team? Fuzzy's Taco Shop is…

View Details
Posted 2025-10-07

Line cook experience

Mast Market
Bedford, TX

Job Title : Line Cook Location: This position is based in Bedford, NY. Schedule : Full-time or Part-time; Must be available for weekends and evenings About Us: Mast Market is a neighb…

View Details
Posted 2025-10-18

Full Time ObGyn Job Houston, TX

CompHealth CompHealth
Houston, TX

Come live and practice in Houston, TX. It is known for attractions that include a world-famous historic park, professional sports, and the largest rodeo in the country. You will enjoy restaurants of e…

View Details
Posted 2025-09-10

Senior Backend Engineer

Logicmatrix Ai
Fort Worth, TX

Senior Backend Engineer Minimum Years of Experience Required: 6+ Years Salary range: $140,000 - $180,000  Eligibility to Work: To be eligible for this role, applicants should be a Permanen…

View Details
Posted 2025-09-14

Bilingual Software Developer

Cinter Career
Dallas, TX

Client (type/ industry): Software Engineering Company  Working Location: San Jose, CA or Dallas, TX  ~ Employment Type: Full-time (40hours/week)  ~ Language: Japanese and English  ~ Salary: …

View Details
Posted 2025-09-14

Surgery Scheduling Coordinator - Orthopedics (Sugar Land)

Houston Methodist
Sugar Land, TX

At Houston Methodist, the Surgery Scheduling Coordinator position is responsible for coordinating the scheduling and related processes for inpatient and outpatient surgical procedures. This position…

View Details
Posted 2025-09-10

Credentialing Customer Success Manager

Calpion/plutus Health
Dallas, TX

About Plutus Health Inc.: Plutus Health Inc. is a leading provider of Revenue Cycle Management (RCM) services, certified in SOC2 compliance and recognized among the Inc. 5000 fastest-growing priva…

View Details
Posted 2025-09-22

Staff Site Reliability Engineer - Big data Platform and Cloud Engineering

Visa
Austin, TX

Company Description Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and govern…

View Details
Posted 2025-09-22

Part-Time Dishwasher

Conservatory at Champion Forest
Spring, Montgomery County, TX

D iscover your Purpose! The Conservatory at Champion Forest , a Discovery Senior Living community is growing, thriving and has meaningful job opportunities available in a safe and supportive work …

View Details
Posted 2025-08-19