Senior ML Engineer (Token Factory)

Role overview

• Inference Optimization: Identifying LLM inference bottlenecks to drive production speedups. Squeezing the maximum performance for a wide range of LLM architectures at scale (e.g., GPT-OSS, Kimi K2.5, DeepSeek V3.1/V3.2, GLM-5).
• Inference engines support: Implement novel speculative decoding architectures, optimise components of various LLM designs (dense/MoE, autoregressive/parallel), and contribute to open-source inference engines.
• Low Precision Training & Inference: Design and productionise low-precision (FP8, NVFP4/MXFP4) training and inference pipelines with measurable gains in throughput and cost-efficiency.
• A profound understanding of theoretical foundations of machine learning and transformer architecture.
• Experience profiling GPU workloads using Nsight, PyTorch profiler, or similar tools
• Understanding of GPU memory hierarchy and compute/memory tradeoffs
• Familiarity with important ideas in LLM space, such as MHA, RoPE, KV-cache, Flash Attention, and quantisation
• Understanding of performance aspects of large neural network training (sharding strategies, custom kernels, hardware features etc.)
• Strong software engineering skills (we mostly use Python)
• Deep experience with modern deep learning frameworks
• Proficiency in contemporary software engineering approaches, including CI/CD, version control and unit testing
• Strong communication and leadership abilities