Data Engineer reports to the Manager of Data Services and is responsible for developing Data Engineering solutions that conform to group standards, budgets and agreed estimated timelines.
This role wrangles data in support of Data Science projects and constructs, tests, and maintains scalable data solutions for structured and unstructured data to support reporting, analytics, ML and AI.
Data Engineer assists with optimization the performance of bigdata eco-systems while assisting with research and building of proof of concepts to test out theories recommended by Senior and Lead Data Engineers.
Primary Job Accountabilities/ Responsibilities
Gains a thorough understanding of the requirements and ensure that work product aligns with customer requirements.
Works within the established development guidelines, standards, methodologies, and naming conventions
Builds processes to ingest, process and store massive amount of data.
Assists with optimization the performance of bigdata ecosystems.
Wrangles data in support of data science projects
Performs productionization of ML and statistical models for Data Scientists & Statisticians
Constructs, tests, and maintains scalable data solutions for structured and unstructured data to support reporting, analytics, ML and AI
Assists with research and building of proof of concepts to test out theories recommended by Senior and Lead Data Engineers
Collaborates and contributes in identifying project risks, design mitigation plans, develops estimates
Contributes to the design, development of data-pipelines, and feature engineering of data solutions.
Established processes or methods are still relied on however Data Engineer will be required to come up with creative solutions to problems. Senior teammates are still needed to provide oversight on solutions to complex problems.
Basic Qualifications Required - Experience, Skills, And Knowledge
Bachelor of Science in a related field preferred or working towards it
A total minimum of 5 years of design and development experience.
At least one year of experience in leading design and advising team members.
Environmental / Physical Demands
This position requires incumbents to regularly sit at a desk and operate standard office equipment such as a computer and phone.
Employee is occasionally required to stand and walk, lift, carry and/or move up to 10 pounds.
Specific vision abilities required by this job include close vision and the ability to adjust focus.
Must be able to talk and hear.
Required to use hands to finger, handle, or feel objects, tools, or controls.
Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
Preferred Experience, Skills, And Knowledge
Knowledge of CRISP-DM methodology relevant to Data Engineering i.e.: Data preparation and Deployment.
Knowledge in Big Data technologies, concepts, and their applications for data processing.
Advanced knowledge of Business Intelligence, Data Warehousing.
Knowledge in fundamentals of Machine Learning and Artificial Intelligence using Microsoft technologies.
Performance tuning and code optimization in SQL.
Data profiling and dimension modeling techniques and creation of logical and physical data models.
Experience working with job scheduling tools.
Experience with SQL, Python, Linux Shell Scripting or MS Power Shell, Spark, NoSQL
Experience with Cloud Service Models: PaaS, IaaS, SaaS
