Senior ML Engineer (Token Factory)
Actively Hiring
Full-time Posted 5 months ago
Role overview
- check_circle Inference Optimization: Identifying LLM inference bottlenecks to drive production speedups. Squeezing the maximum performance for a wide range of LLM architectures at scale (e.g., GPT-OSS, Kimi K2.5, DeepSeek V3.1/V3.2, GLM-5).
- check_circle Inference engines support: Implement novel speculative decoding architectures, optimise components of various LLM designs (dense/MoE, autoregressive/parallel), and contribute to open-source inference engines.
- check_circle Low Precision Training & Inference: Design and productionise low-precision (FP8, NVFP4/MXFP4) training and inference pipelines with measurable gains in throughput and cost-efficiency.
- check_circle A profound understanding of theoretical foundations of machine learning and transformer architecture.
- check_circle Experience profiling GPU workloads using Nsight, PyTorch profiler, or similar tools
- check_circle Understanding of GPU memory hierarchy and compute/memory tradeoffs
- check_circle Familiarity with important ideas in LLM space, such as MHA, RoPE, KV-cache, Flash Attention, and quantisation
- check_circle Understanding of performance aspects of large neural network training (sharding strategies, custom kernels, hardware features etc.)
- check_circle Strong software engineering skills (we mostly use Python)
- check_circle Deep experience with modern deep learning frameworks
- check_circle Proficiency in contemporary software engineering approaches, including CI/CD, version control and unit testing
- check_circle Strong communication and leadership abilities
Preferred qualifications
- Experience working with open-source inference engines (vLLM, SGLang, TensorRT-LLM), including contributions
- Experience with kernel languages or DSLs such as Triton, Cute, CUTLASS, CUDA
- A track record of building and delivering products (not necessarily ML-related) in a dynamic startup-like environment.
- Strong engineering skills, including experience in developing large distributed systems or high-load web services.
- Open-source projects that showcase your engineering prowess
- Excellent command of the English language, alongside superior writing, articulation, and communication skills.
Benefits
- check_circle Competitive salary and comprehensive benefits package.
- check_circle Opportunities for professional growth within Nebius.
- check_circle Flexible working arrangements.
- check_circle A dynamic and collaborative work environment that values initiative and innovation.
Tags & Focus Areas
Ai Machine Learning