Cloud Data Engineer

Job Description

Job Title : Cloud Data Engineer

Summary :

The C loud Data Engineer collaborates with cross-functional teams to deliver efficient data engineering processes, data solutions, and operations support.

Wage Type : Salaried

Essential Duties & Responsibilities:

To perform this job successfully, an individual must be able to perform each of the essential duties satisfactorily. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

• Support clear communications both within the organization and to business partners around any data initiatives, planning, and problem resolutions, including Databricks workspace collaboration.
• Operations Support
  • Be the go-to person for production support. Troubleshoot issues, identify bottlenecks, and ensure smooth data operations.
  • Work closely with our partners and internal staff to address any data-related concerns promptly.
  • Keep a watchful eye on system performance and proactively address any anomalies.
• Data Engineering
  • Design, develop, and maintain scalable cloud-based data architectures on AWS and Databricks.
  • Implement efficient ETL/ELT processes using Databricks notebooks, Delta Lake, and Apache Spark.
  • Collaborate with cross-functional teams to understand data requirements and ensure data solutions meet business needs.
  • Utilize AWS services such as S3, Glue, Redshift, and EMR, and integrate with Databricks Workspaces for unified data processing.
  • Ensure data security, integrity, and compliance with industry standards and regulations, including role-based access control (RBAC) in Databricks.
  • Troubleshoot and optimize existing ETL processes for performance and reliability using Databricks performance tuning techniques.
  • Stay updated on emerging cloud technologies and recommend improvements to enhance data engineering capabilities, including Databricks Lakehouse architecture.
  • Collaborate with data scientists and analysts to provide accessible and high-quality data for analytics and reporting via Databricks SQL and Unity Catalog.
• Carries out responsibilities in a manner consistent with our values and operating principles, in accordance with policy and applicable laws, and with a commitment to continuous improvement and process excellence.
• Any other duties as assigned.

Key Deliverables :

• Production support of the cloud-based data ecosystem through troubleshooting, debugging, analysis, and code deployments, including Databricks job orchestration and monitoring via Databricks Jobs UI.
• Support the data warehouse and Lakehouse platform maintenance, including patches, upgrades, and workspace configuration.

Organizational Structure :

Reports to: Head of Data Engineering & Integrations

Supervises: N/A

Qualifications:

Education:

• Bachelor's degree in Computer Science, Information Technology, or a related field.

Required Knowledge/Skills:

• Proven experience as a Cloud Data Engineer with expertise in AWS services.
• Strong background in designing and implementing ETL processes.
• Familiarity with data modeling, database design, and optimization techniques.
• Excellent problem-solving skills and attention to detail.
• Ability to work collaboratively in a team environment.
• Strong communication skills to articulate technical concepts to non-technical stakeholders.

Desired Experiences :

• Describe any "required" experiences - these are must have experiences
• AWS certifications in relevant areas.
• Experience with Apache Nifi, Apache Nifi Registry
• Knowledge of data warehousing concepts and practices.
• Familiarity with DevOps practices in a cloud environment.
• Knowledge of Banking and Financial Services
• Knowledge of Ataccama

Talents:

• Strong positivity.
• Mission driven, competitive, goal oriented, and motivated to develop themselves and others.
• Energetic, resourceful, and appropriate work intensity to get the work done

• Strong people acumen and relationship skills; Naturally pre-disposed to quickly establish positive personal and professional relationships.

Other :

• Ability to interpret a variety of instructions furnished in written, oral, diagram or schedule form.
• Must be able to lift to 20 pounds.
  

TransPecos Banks will not accept unsolicited resumes from any source other than the candidate. We will consider any candidate for whom an Agency submits an unsolicited resume, to have been referred to us by the Agency free of any charges or fees, other than those agencies we engage on a specific search. TransPecos Banks will not pay a fee for any placement resulting from the receipt of an unsolicited resume.