Talentiqo
Location
Texas | United States
Job description
Title: Sr Data Engineer
Location: 100% Remote role
Duration: Direct Hire
Responsibilities:
Collecting, organizing, managing, and converting raw data into a format that can be easily analyzed by Business Intelligence analysts and data scientists.
Building and maintaining data pipelines that collect and transport data from various sources client's data storage systems.
Using algorithms and programming languages such as SQL and Python to prepare data for analysis.
Working closely with the management and end-users to understand and address business requirements related to data storage, management, and analysis.
Creating data analysis tools and developing new data validation methods to ensure data accuracy and completeness.
Identifying ways to make data more reliable, efficient, and accessible to relevant stakeholders.
Creating and maintaining the organization's software and hardware architecture to support efficient and secure data storage and management.
Conducting research and troubleshooting to address potential problems that may arise in the data storage and management systems.
Play a key role in designing and crafting a modern Data and Information Delivery and Analytics platform in the cloud to support retail service and product distribution for the US market.
Create and maintain optimal data pipeline architecture.
Assemble large, complex data sets that meet functional/non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologies.
Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
Work with stakeholders including the Executive, Operations, FP&A, and Supply Chain teams to assist with data-related technical issues and support their data infrastructure needs.
Keep our data separated and secure during transmission and at rest.
Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
Work with data and analytics experts to strive for greater functionality in our data systems.
Requirements:
BS in Computer Science, Data Science, or equivalent.
7+ years of professional software development or data engineering experience.
5+ years of experience using and strategizing the use of DBT and Airflow.
Strong working knowledge of SQL, of datastores and their tradeoffs (including relational, columnar, and document stores), data modeling, data structures, data manipulation.
Strong knowledge of Extract, Transform, Load (ETL) pipeline design, tooling, and support.
Experience designing, building and optimizing 'big data' data pipelines, architectures and data sets.
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Strong analytic skills related to working with unstructured datasets.
Build processes supporting data transformation, data structures, metadata, dependency and workload management.
Proven ability to architect, implement, and optimize high throughput data pipelines.
Experience deploying production systems in the cloud (i.e., AWS, Azure).
Strong communication skills in writing and conversation.
Experience with tools we use every storage:
Snowflake, AWS Storage Services (e.g., S3, RDS, Glacier)
ETL/BI: Astronomer, DBT, Domo, Tableau, PowerBI
Proven passion and talent for teaching fellow engineers and non-engineers.
Experience with encryption at rest, including multiple approaches and tradeoffs.
Experience in Retail operations.
Job tags
Salary
