Principal Data Engineer

March 15

Apply Now

Receive Emails with Similar Jobs

Spotter

WebsiteLinkedInAll Job Openings

Spotter empowers top YouTube creators to accelerate their business and unleash their full creative potential by giving them access to the capital, knowledge, and community they need to succeed at scale$1. .$1

51 - 200

Description

• Are you ready to help lead the charge in shaping the data-driven future of Spotter? • In search of an exceptional Principal Data Engineer • Develop and maintain scalable data pipelines for acquisition and transformation of large datasets • Troubleshoot issues directly with internal data consumers • Automate pipeline runs with scheduling and orchestration tools • Work with big data technologies in the cloud (AWS) • Lead development of projects involving other team members and act as a mentor • Actively participate in team discussions about technology/architecture/solutions for new projects

Requirements

• Bachelor’s degree, preferably in Computer Science or Computer Information Systems • 6+ years of software engineering experience • 5+ years of data engineering experience with Apache Spark or Apache Flink • 4+ years of experience running software and services in the cloud • Proficiency in working with DataFrame APIs (Pandas and Spark) • Proficiency using Python, Scala, etc. with modern data optimized file formats • Proficiency with SQL on RDBMS and data warehouse solutions like Redshift • Hands on experience with Data Lake technologies like Delta Lake and Iceberg • Experience supporting ML/AI projects • Additional Valued Skills: Experience with YouTube APIs, AWS Glue metastore, Data-Mesh approaches, data cataloging, data lineage, and data governance tools

Benefits

• Medical and vision insurance covered up to 100% • Dental insurance • 401(k) matching • Stock options • Complimentary gym access • Autonomy and upward mobility • Diverse, equitable, and inclusive culture