Role overview
- Inference Optimization: Identifying LLM inference bottlenecks to drive production speedups. Squeezing the maximum performance for a wide range of LLM architectures at scale (e.g., GPT-OSS, Kimi K2.5, DeepSeek V3.1/V3.2, GLM-5).
- Inference engines support: Implement novel speculative decoding architectures, optimise components of various LLM designs (dense/MoE, autoregressive/parallel), and contribute to open-source inference engines.
- Low Precision Training & Inference: Design and productionise low-precision (FP8, NVFP4/MXFP4) training and inference pipelines with measurable gains in throughput and cost-efficiency.
- A profound understanding of theoretical foundations of machine learning and transformer architecture.
- Experience profiling GPU workloads using Nsight, PyTorch profiler, or similar tools
- Understanding of GPU memory hierarchy and compute/memory tradeoffs
- Familiarity with important ideas in LLM space, such as MHA, RoPE, KV-cache, Flash Attention, and quantisation
- Understanding of performance aspects of large neural network training (sharding strategies, custom kernels, hardware features etc.)
- Strong software engineering skills (we mostly use Python)
- Deep experience with modern deep learning frameworks
- Proficiency in contemporary software engineering approaches, including CI/CD, version control and unit testing
- Strong communication and leadership abilities
Preferred qualifications
- Experience working with open-source inference engines (vLLM, SGLang, TensorRT-LLM), including contributions
- Experience with kernel languages or DSLs such as Triton, Cute, CUTLASS, CUDA
- A track record of building and delivering products (not necessarily ML-related) in a dynamic startup-like environment.
- Strong engineering skills, including experience in developing large distributed systems or high-load web services.
- Open-source projects that showcase your engineering prowess
- Excellent command of the English language, alongside superior writing, articulation, and communication skills.
Benefits
- Competitive salary and comprehensive benefits package.
- Opportunities for professional growth within Nebius.
- Flexible working arrangements.
- A dynamic and collaborative work environment that values initiative and innovation.
Tags & focus areas
Used for matching and alerts on DevFound Ai Machine Learning