Tech lead - Python web scraping/crawling
Location
Bangalore | India
Job description
Company Overview
Gmware is a leading provider of customized tech and operations solutions to small, medium, and large enterprises. Our technology is being used by industry leaders worldwide. With a team of 11-50 employees, we are committed to helping businesses grow and thrive. Visit our website at to learn more about us and our solutions.
Job Overview
We are seeking a highly skilled Tech Lead with expertise in Python Web Scraping/crawling. As a Tech Lead, you will be responsible for leading a team and driving the development of web scraping and crawling projects. You will work closely with stakeholders to understand their requirements and design effective scraping solutions. The ideal candidate should have 7 to 10 years of experience in web scraping and a strong proficiency in Python programming language.
Qualifications and Skills
- 7 to 10 years of experience in web scraping and crawling
- Strong proficiency in Python programming language
- In-depth knowledge of web scraping techniques and best practices
- Experience with popular web scraping frameworks such as Scrapy and Beautiful Soup
- Proficiency in data cleaning and validation techniques
- Familiarity with data extraction from different sources including websites, APIs, and databases
- Ability to handle large data sets and perform data analysis
- Excellent problem-solving and troubleshooting skills
- Good communication and leadership skills
- Ability to work effectively in a team environment
- Familiarity with data privacy and security regulations
Roles and Responsibilities
- Lead a team of developers to design and implement web scraping and crawling projects
- Collaborate with stakeholders to define project requirements and deliverables
- Develop scraping scripts using Python programming language and popular frameworks like Scrapy and Beautiful Soup
- Efficiently extract data from various sources and ensure data quality through cleaning and validation processes
- Optimize and monitor scraping processes to ensure efficiency and accuracy
- Stay up to date with the latest web scraping trends and technologies
- Mentor and provide technical guidance to junior team members
- Collaborate with cross-functional teams to integrate scraped data into existing systems
- Ensure compliance with data privacy and security regulations
Job tags
Salary
