Technical Lead, Data Platform
August 11
🏢 In-office - Bay Area
Description
• The Data Platform team owns the data, infrastructure and tooling for the data we use to develop a learned driver. • You will be part of a growing group focussed on discovering the best data recipe for driving, exploring how far data can push autonomous driving performance. • You will be working across functions with machine learning research engineers, virtual world simulation engineers, robotics engineers and safety drivers to ingest, enrich and visualise thousands of hours of driving data. • Examples Projects: • Data governance tooling to control and audit access to data, as well as visualise what data is available and how it was created (data lineage). • Quality control and validation of datasets e.g. removing examples of bad driving. • Labelling, enrichment and augmentation of data at scale using thousands of GPUs simultaneously. • Orchestration of data processing and machine learning workloads by building out infrastructure for running Flyte and notebook environments (e.g. Google collab) at scale.
Requirements
• 5+ years of professional experience in Software Engineering • Proficiency in Spark and Kubernetes. • Experience building reliable data pipelines to handle large data sets. • Experience working with concurrent, parallel and distributed computing. • Experience with cloud infrastructure (AWS, Azure and/or GCP). • Knowledge of software engineering practices - what makes code reusable and extensible. • Passion for infrastructure: building internal tooling and frameworks. • Experience working closely with users, shaping data to fit their needs .
Apply Now
