Data Engineer II - AWS / Python
- Supports the review of controls to ensure sufficient protection of enterprise data.
- Reviews and makes customizations in one or two tools to generate a product at the business's or customer's request.
- Updates logical or physical data models based on new use cases.
- Frequently uses SQL and understands NoSQL databases and their niche in the marketplace.
- Contributes to the team's culture by promoting collaboration and innovation.
- Designs, develops, and maintains robust data pipelines to automate the extraction, transformation, and loading (ETL) of data from various sources into data warehouses or data lakes.
- Integrates data from multiple sources, ensuring consistency, accuracy, and reliability.
- Collaborates with product owners to understand data requirements and provide solutions.
- Documents data pipeline processes and workflows for future reference and training.
- Formal training or certification on data engineering disciplines and 2+ years applied experience
- Advanced proficiency in SQL (e.g., joins and aggregations)
- Working understanding of NoSQL databases
- Proficiency in programming languages such as Python/Java for data processing tasks
- Proficient in Object-Oriented Programming (OOP) concepts, with a strong ability to design and implement robust, reusable, and maintainable code structures across various programming languages
- Experience with cloud platforms, particularly Amazon Web Services (AWS), including EMR, Glue, Lambda, and ECS, to design, deploy, and manage scalable and efficient cloud-based solutions
- Hands-on experience with frameworks like Apache Spark, leveraging its capabilities for large-scale data processing and analytics to drive efficient and insightful data solutions
- Experience in utilizing Cucumber and Gherkin for behavior-driven development (BDD)
- Proficiency in Unix scripting, data structures, data serialization formats such as JSON, AVRO, or similar, and big-data storage formats such as Parquet
- Excellent communication skills to collaborate with cross-functional teams and stakeholders
- Understanding of data security best practices and compliance requirements.
- Java programming knowledge, including experience with Maven, CI/CD pipelines, and Docker
- Provisioned infrastructure using a high-level configuration language
- Utilized Splunk to monitor and analyze system performance
- Flexibility and eagerness to learn new technologies and skills
Recommended Jobs
SOUS CHEF - Houston, TX
Morrison Healthcare is a leading national food and nutrition services company exclusively dedicated to serving more than 600 hospitals and healthcare systems. Morrison's hospital kitchens, r…
ADMINISTRATIVE ASSISTANT (WAREHOUSE OPERATIONS) (FULL TIME)
We are hiring immediately for a full time ADMINISTRATIVE ASSISTANT (WAREHOUSE OPERATIONS)position. Location : TAMU Tarleton Mn - 503 North Neblett Avenue, Stephenville, TX 76401. (Openings…
Accounts Payable Supervisor
Position Summary The Accounts Payable Supervisor oversees the full-cycle accounts payable function, ensuring accurate, timely, and compliant processing of invoices. This position is responsibl…
Expeditor - Shipping & Receiving Clerk
Job ID#: 31145 Expeditor – Shipping & Receiving Immediate Opportunity – Needed ASAP! Contract to Hire Pay rate to $19 hour depending on experience IN OFFICE – 77087 As an Expeditor, y…
Application Security Engineer
Your Opportunity Application Security Engineer Carrollton, TX (on-site) The Application Security Engineer is responsible for championing the integration of security at every stage of the …
Access Control & Low Voltage Technician
A-1 Locksmith is a leading provider of locksmith, safe, and security services in the DFW area, known for its rich history and trusted reputation. As a rapidly growing company, we offer a unique blend…
Technical Project Manager
Job Title: Technical Project Manager Employment Type: Fulltime About Cloudelligent Cloudelligent is Cloud-native consultancy and AWS advanced consulting partner ! We specialize in pro…
Principal Consultant- Project & Portfolio Management Consulting
Description This position is for a Principal Consultant within our Project and Portfolio Management Consulting practice. You will be primarily responsible for project delivery along with client rel…
Estimating System User Product Manager
Requisition ID: 287459 ~ Relocation Authorized: None ~ Telework Type: Part-Time Telework ~ Work Location: Houston, TX Extraordinary teams building inspiring projects: Since 189…
R
For those who want to put their love of pets into action, there are few jobs more rewarding than that of Vet Assistant. While working closely with other pet health care professionals at Banfield, you…