logo

JobNob

Your Career. Our Passion.

PySpark Tech Lead - Big Data


TD Newton & Associates


Location

India | India


Job description

Technical Skills :- Extensive experience working on software engineering projects and SDLC- Have worked on various development project as developer and/or lead developer.- Capability to understand and comprehend the solution and create a development framework where all the developers will be contributing.- Should be able to create a Test driven development strategy with contribution from other developers and QAs (focusing on the solution, which is data engineering/Big data in our case) to achieve maximum code coverage .- Should be work closely with dev ops team and able to design CI/CD Strategy to orchestrate the team activities.- Proficient in the python programming language and exposure of other scripting language and Database would be advantage.- Comfortable with code review and provide support when needed- Working experience in implementing Data lake, ETL, Virtualization, Containerization, Orchestration etc.- Working experience on PySpark and/or Spark based solution.- Good understanding of various aspects and best practices of Big Data solutions.- Good understanding of on-prem and cloud based solutions. - Good experience working with various big data file formats like - parquet/delta/orc etc.- Ability to guide junior team members in team and deliver the ask. (ref:hirist.tech)


Job tags



Salary

All rights reserved