Location
Hyderabad | India
Job description
Job Summary:
- Under close supervision, supports the design, build, test and maintenance of data pipelines at big data scale. Assists with updating data from multiple data sources. Other functionalities under close supervision include working on batch processing of collected data and matching its format to the stored data, making sure that the data is ready to be processed and analyzed. Assists with keeping the ecosystem and the pipeline optimized and efficient, troubleshooting standard performance, data related problems and providing L3 support.
- The Data Engineer I plays a pivotal role within Dataworks, focused on supporting the delivery of engineering innovation within Dataworks, helping to execute the delivery of key business initiatives. She / he acts as a universal translator between IT, business, software engineers and data scientists, collaborating with these multi-disciplinary teams. The Data Engineer I will contribute to the adherence of technical standards for data engineering, including the selection and refinements of foundational technical components. She / he will work on those aspects of the Dataworks platform that govern the ingestion, transformation, and pipelining of data assets, both to end users within FedEx and into data products and services that may be externally facing. Day-to-day, she / he will be deeply involved in code development and small-scale deployments.
Essential Job Duties & Responsibilities:
- Understanding in depth both the business and technical problems Dataworks aims to solve
- Building tools, platforms and pipelines to enable teams to clearly and cleanly analyze data, build models and drive decisions
- Scaling up from laptop-scale to cluster scale problems, in terms of both infrastructure and problem structure and technique
- Delivering tangible value very rapidly, collaborating with diverse teams of varying backgrounds and disciplines
- Following best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases
- Interacting with junior technologists from the broader enterprise and outside of FedEx (partner ecosystems and customers) to create synergies and ensure smooth deployments to downstream operational systems
Skill / Knowledge Considered a Plus:
- Technical background in computer science, software engineering, database systems, distributed systems
- Familiarity with distributed and cloud environments and some understanding of how to balance computational considerations with theoretical properties.
- Detailed knowledge of the Microsoft Azure tooling for large-scale data engineering efforts and deployments is highly preferred.
- Some experience with designing and deploying large scale technical solutions, which deliver tangible, ongoing value.
- Direct experience having built and deployed robust, complex production systems that implement modern, data scientific methods at scale.
- Ability to context-switch, to provide support to dispersed teams which may need an expert hacker to unblock an especially challenging technical obstacle, and to work through problems as they are still being defined.
- Demonstrated ability to deliver technical projects with a team, often working under tight time constraints to deliver value.
- An engineering mindset, willing to make rapid, pragmatic decisions to improve performance, accelerate progress or magnify impact.
- Comfort with working with distributed teams on code-based deliverables, using version control systems and code reviews.
- Ability to conduct data analysis, investigation, and lineage studies to document and enhance data quality and access.
- Use of agile and devops practices for project and software management including continuous integration and continuous delivery
- Demonstrated expertise working with some of the following common languages and tools:
- Spark (Scala and PySpark), HDFS, Kafka and other high volume data tools
- SQL and NoSQL storage tools, such as MySQL, Postgres, Cassandra, MongoDB and Elasticsearch Pandas, Scikit-Learn, Matplotlib, TensorFlow, Jupyter and other Python data tools
Must Have:
- Knowledge of Data integration concepts
- Knowledge programming languages
- 1-3 yrs. exp. with knowledge on ETL tool Ab Initio, UNIX, SQL
Good to have:
- Work with internal and external subject matter experts to select relevant source of information and explore abundance of data available in FedEx to identify and solve business problems.
- Ability to manipulate and analyze data with a scientific approach in order to determine the best and most optimal solution.
- Concepts and basics of SAFe / Agile methodology
Minimum Qualifications:
- Bachelors degree in information systems, Computer Science or a quantitative discipline such as Mathematics or Engineering and/or equivalent formal training or work experience.
- Basic knowledge in data engineering and machine learning frameworks including design, development and implementation of highly complex systems and data pipelines. Basic knowledge in Information Systems including design, development and implementation of large batch or online transaction-based systems. Experience as a junior member of multi-functional project teams.
- Strong oral and written communication skills. A related advanced degree may offset the related experience requirements.
Job tags
Salary
