Location
Pune | India
Job description
- As a Data Engineer, you will design and develop highly scalable, reliable, and self-heal data engineering pipelines to process large volumes of data across disparate data sources effectively addressing functional and non-functional concerns while adhering to well-established engineering concepts, as well as data quality system to identify data issues proactively, rather than reactively
- On the other side, you will be supported for growth by the management and TMF Business Academy to stay ahead of the technology by researching and learning new technologies and developing innovative solutions for complex problems in data engineering to serve the AI and Data Science needs of the future
Key Responsibilities
- Work closely with the Analytics and Business teams to understand Business requirements
- Work closely with Data architect to design and document data models, source to target mapping
- Create Physical data models
- Propose design solutions for data integration challenges
- Develop POCs for proposing new solutions and technologies
- Maintain and manage master data
- Design and develop a data quality system to identify data issues proactively, rather than reactively
- Develop and promote best practices, reference architecture, and engineering standards in data engineering
- Automate pipeline deployments through CI/CD processes
- Review designs and code of junior developers and share feedback to improve the quality of deliverables
- Automate pipeline execution and monitoring
- Manage code repository and ensure timely code check-ins
- Work collaboratively as a member of the development team to build a best-in-class data platform in an agile environment
- Identify areas of improvement and propose solutions
- Ensure process compliance
- Provide Daily / Weekly updates on the progress of work
Ideally, you meet the following criteria:
- Bachelor of Engineering in Computer Science, Information Systems or equivalent
- Experience with Azure is a must-have, at least 3+ years
- Certifications in Big Data related technologies and/or Microsoft certifications will be considered a plus
- Deep understanding of Data Structures and algorithms
- Experience with relational SQL and NoSQL databases
- Experience with stream-processing systems: Storm, Spark-Streaming, etc
- Experience in building operational ETL data pipelines from a number of disparate sources
- Strong development experience with SSIS / ADF / DataBricks
- Experience with Spark and other distributed processing frameworks
- Understanding of code management, CI/CD concepts and their application in the Big-data universe
- Experience in implementing solutions for Master Data Management
- Experience with Data Cataloging tools
- Experience in Agile development methodologies
- Have a passion for solving problems with data
- Willing to learn new skills and technologies
Whats in it for you
Pathways for career development
- Work with colleagues and clients around the world on interesting and challenging work.
- We provide internal career opportunities, so you can take your career further within TMF.
- Continuous development is supported through global learning opportunities from the TMF Business Academy.
Making an impact
- You'll be helping us to make the world a simpler place to do business for our clients.
- Through our corporate social responsibility program, you'll also be making a difference in the communities where we work.
A supportive environment
- Strong feedback culture to help build an engaging workplace.
- Our inclusive work environment allows you to work from our offices around the world, as well as from home, helping you find the right work-life balance to perform at your best.
Job tags
Salary
