Research Engineer, ML Systems (All Industry Levels)

Responsibilities

• check_circle Write efficient Triton kernels and tune them for our specific models and hardware
• check_circle Develop prefix-aware routing algorithms to improve serving cache hit rate
• check_circle Train and distill LLMs to improve latency while preserving accuracy and engagements
• check_circle Build an efficient and scalable distributed RLHF stack powering the model innovations
• check_circle Develop systems for efficient multimodal (image gen/video gen) model training & inference
• check_circle "All Industry Levels": at least PhD (or equivalent) research experience
• check_circle Write clear and clean production system code
• check_circle Strong understanding of modern machine learning techniques (reinforcement learning, transformers, etc)
• check_circle Track record of exceptional research or creative ML systems projects
• check_circle Comfortable writing model development code (PyTorch) for either training or inference

Preferred qualifications

• Experience training large models in a distributed setting utilizing PyTorch distributed, DeepSpeed, Megatron.
• Experience working with GPUs & collectives (training, serving, debugging) and writing kernels (Triton, CUDA, CUTLASS).
• Experience with LLM inference systems and literature such as vLLM and FlashAttention.
• Familiarity with ML deployment and orchestration (Kubernetes, Docker, cloud)
• Publications in relevant academic journals or conferences in the field of machine learning and systems

About the company