ML Engineer - LLM Evaluation
August 27
🏡 Remote – Anywhere in California
Description
• Own LLM evaluation processes and methods with a focus on generating benchmarks representative of real-world usage and safety vulnerabilities. • Generate high quality synthetic data, curate labels, and conduct rigorous benchmarking. • Deliver robust, scalable, and reproducible production code. • Push the envelope by developing methods for benchmarking that revamps how we assess the best LLMs for harmlessness and helpfulness. Your research will directly empower our customers to more feasibly deploy safe and responsible LLMs. • Co-author papers, patents, and presentations with our research team by integrating other members’ work with your vertical.
Requirements
• Domain knowledge in LLM evaluation and data curation techniques. • Extensive experience in designing and implementing LLM benchmarking, extending previous methods. Comfortability with leading end-to-end projects. • Adaptability and flexibility. In both the academic and startup world, a new finding in the community may necessitate an abrupt shift in focus. You must be able to learn, implement, and extend state-of-the-art research. • Preferred: past research or projects in benchmarking LLMs.
Apply Now
