Airtel Digital
Location
Gurgaon | India
Job description
We are looking for a Lead Data Engineer who will be working on AWS based modern data stack built for doing data exploration from first principles and powering Hyper-Personalization use cases in multiple properties like Wynk Music, Xstream App and Stick. You will also be responsible for building and maintaining ingestion and data processing systems handling data at large scale ( ~4B events, 200 TB Hadoop Cluster, Concurrency of 500K users)
Job location: Gurugram/Bangalore Responsibilities: ● Build scalable pipelines for data analysis and transformation ● Designing platforms as consumable data services ● Come up with efficient, modular and scalable solutions for business and technical problems ● Build near-real-time (low latency) infrastructure for segmentation, personalized recommendations, reporting, etc. ● Build and execute data mining and modelling activities using agile development techniques ● Participate in code reviews to assess overall code quality and flexibility ● Drive for continuous improvement in software and development process within an agile development team ● Solve problems in robust and creative ways and demonstrate solid verbal, interpersonal and written communication skills ● Handle TBs of data volume daily on Wynk Music/XStream Data platform ● Build pipelines to transform streaming data under a fixed SLA with 99.99% uptime ● Prototype creative solutions quickly by developing minimum viable products and work with seniors and peers in crafting and implementing the technical vision of the team ● The incumbent will be in the driving seat for innovation and delivery in various exciting tech opportunities, from powering business analytics pipelines to machine learning applications. Desired Profile: 1- Should have 3+ years experience in data engineering background 2- Should have good hands-on experience in distributed data systems and At Least 2 years experience in Spark computation system is mandatory. 3- Should have 2+ years coding experience either in Scala, PySpark or Java. 4- At-least one cloud technology stack exposure with 2+ years experience 6- Good to have if have exposure to handle large and unstructured data setsJob tags
Salary
