You will join our team of world-class experts developing the AlphaSense platform
The team is right at the very core of what we do and responsible for implementing cutting-edge technology for scalable, distributed processing of millions of documents
We are seeking a highly skilled Senior Software Engineer to join our dynamic team responsible for building and maintaining data ingestion systems at scale
As a key member of our team, you will play a crucial role in designing, implementing and optimizing robust solutions for ingesting millions of documents per month, including the addition of multimedia content such as audio and video from the public web
What you'll Do:
Design, develop, and maintain scalable data ingestion pipelines to process large volumes of documents efficiently and reliably.
Collaborate with cross-functional teams to understand requirements and translate them into technical specifications and system designs.
Implement best practices for data ingestion, storage, and processing to ensure high performance, scalability, and reliability.
Integrate multimedia content ingestion capabilities into existing data pipelines, leveraging appropriate technologies and APIs to retrieve and process audio and video files from the public web.
Optimize data ingestion workflows to improve throughput, reduce latency, and minimize resource utilization.
Monitor system performance, troubleshoot issues, and implement solutions to address bottlenecks and ensure smooth operation.
Stay up-to-date with emerging technologies and industry trends related to data ingestion and multimedia processing, and propose innovative solutions to enhance our capabilities.
Work closely with Product Management to translate product requirements into software architectures
Follow the engineering processes, DevOps practices, and trends inside the company (monitoring, alerting, performance optimization, integration testing, design documentation) and make sure the teams improve related knowledge as well
Candidate Requirements:
Must-Have
Bachelors or Masters degree in Computer Science, Engineering, or a related field.
Minimum 4 years of experience in Software Development with 3+ years of proficiency in Python
Experience working with multimedia content and familiarity with relevant technologies and protocols for audio and video processing (eg, FFmpeg, HLS, MPEG-DASH).
Experience in structured/unstructured data extraction through text processing, audio processing, regular expressions etc
Experience working in a cross-functional distributed team environment
Excellent verbal and written communication skills with the ability to share thoughts and ideas, concisely and persuasively.
Nice to have
Proficiency in programming languages such as Java, or GO, and experience with related frameworks and libraries for data processing and manipulation.
Knowledge of web scraping techniques and APIs for retrieving multimedia content from the public web is a plus.
Working knowledge of any cloud service provider(preferably AWS)
Familiar with IAC technologies such as Terraform and Crossplane.
Experience with workflow orchestration tools like Airflow, Prefect
Exposure NoSQL solutions like MongoDB, DynamoDB, etc
