Note: The job is a remote job and is open to candidates in USA. NVIDIA is at the forefront of the generative AI revolution, particularly focusing on optimizing generative AI models for maximal inference efficiency. They are seeking a Deep Learning Software Engineer to develop and scale up automated inference and deployment solutions, working on state-of-the-art generative AI models and enhancing NVIDIA's software platforms.
Responsibilities
- Train, develop, and deploy state-of-the generative AI models like LLMs and diffusion models using NVIDIA's AI software stack.
- Leverage and build upon the torch 2.0 ecosystem (TorchDynamo, torch.export, torch.compile, etc...) to analyze and extract standardized model graph representation from arbitrary torch models for our automated deployment solution.
- Develop high-performance optimization techniques for inference, such as automated model sharding techniques (e.g. tensor parallelism, sequence parallelism), efficient attention kernels with kv-caching, and more.
- Collaborate with teams across NVIDIA to use performant kernel implementations within our automated deployment solution.
- Analyze and profile GPU kernel-level performance to identify hardware and software optimization opportunities.
- Continuously innovate on the inference performance to ensure NVIDIA's inference software solutions (TRT, TRT-LLM, TRT Model Optimizer) can maintain and increase its leadership in the market.
- Play a pivotal role in architecting and designing a modular and scalable software platform to provide an excellent user experience with broad model support and optimization techniques to increase adoption.
Skills
- Experience in Deep Learning.
- Excellent software design skills, including debugging, performance analysis, and test design.
- Strong proficiency in Python, PyTorch, and related ML tools (e.g. HuggingFace).
- Strong algorithms and programming fundamentals.
- Contributions to PyTorch, JAX, or other Machine Learning Frameworks.
- Knowledge of GPU architecture and compilation stack, and capability of understanding and debugging end-to-end performance.
- Familiarity with NVIDIA's deep learning SDKs such as TensorRT.
- Experience in writing high-performance GPU kernels for machine learning workloads in frameworks such as CUDA, CUTLASS, or Triton.
Education Requirements
- Masters, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related field.
Benefits
- Equity
- Benefits
Company Overview
- NVIDIA is a computing platform company operating at the intersection of graphics, HPC, and AI. It was founded in 1993, and is headquartered in Santa Clara, California, USA, with a workforce of 10001+ employees. Its website is https://www.nvidia.com.
Company H1B Sponsorship
- NVIDIA has a track record of offering H1B sponsorships, with 1418 in 2025, 1356 in 2024, 976 in 2023, 835 in 2022, 601 in 2021, 529 in 2020. Please note that this does not guarantee sponsorship for this specific role.
Tags & focus areas
Used for matching and alerts on DevFound Remote Engineer Intern Entry Level Dev Pytorch Fulltime