Location
San Francisco, CA | United States
Job description
Company
Archipelo is building a code security platform that gives organizations the ability to verify the authenticity and provenance of code within their software development lifecycle. They are solving a painful problem that affects every software developer on the planet: ensuring software security, authenticity, integrity, and compliance - by providing the context for how the code was created.
They ensure that secure coding best practices are implemented proactively at the earliest stages of the SDLC, from research and design to development and deployment.
Description
Right now, they are seeking a Senior Data Engineer to lead technology development on the frontier of code discovery and developer productivity.
Tasks: - Implement data engineering best practices
- Design and develop data systems for machine learning
- Write real-time pipelines that execute complex operations on incoming data
- Experiment in ways that accelerate prototyping and maximize resource utilization
- Ensure pipelines work quickly, focus on fast single node performance and leverage horizontal scaling
- Manage our data pipeline, including scheduling, dataflow programming, SQL and data labelling
- Orchestrate the operation of clusters of commodity machines
- Review code, mentor other engineers and support the data team
- Attract, recruit and retain top engineering and scientific talent
Must-have skills:
- Expertise in microservices and cloud computing—across cloud platforms
- Proficient with distributed systems and the coordination of high volume independent commodity machines into complete, functional systems to handle diverse workloads
- Minimum 6+ years of professional data engineering experience
- Expertise with ETL
- Expertise with Python
- Proven expertise and leadership as a world-class senior data engineer
Nice-to-have skills:
- Bachelor’s or Master’s degree in computer science/engineering, mathematics, physics, or other related technical fields with equivalent practical experience
- Experience with Natural Language Processing and Understanding (NLP & NLU)
- Knowledge of math, probability, statistics and algorithms
- Experience with Golang
- Familiarity with machine learning frameworks (like Keras or PyTorch)
- Ability to run a CPU and IO profiler to figure out where to optimize the pipeline
- Advanced working knowledge of information retrieval and search technologies and have set up and used open-source search systems to query and understand data
- Experience with most of the following technologies:
- ElasticSearch, Solr and Lucene
- Machine learning infrastructure
- Kubernetes, Docker, Terraform
- Deep learning, GNNs
- CircleCI, GitHub Actions, Jenkins
- Graph databases
Benefits:
- A strong remote work culture that includes group activities and local gatherings
- Competitive salary & equity packages
- Unlimited vacation and sick leave
Interview process:
- Intro call with Toughbyte
- Call with Head of Operations
- Culture fit interview with a Founder
- Technical interview with Data Engineer
- Technical interview with Development Team Lead
- Culture fit interview with a second Founder
Job tags
Salary
