NVIDIA/Model-Optimizer

[](https://nvidia.github.io/Model-Optimizer) [](https://pypi.org/project/nvidia-modelopt/) [](./LICENSE) Documentation | Roadmap | Announcement Blogs NVIDIA Model Optimizer (referred to as Model Optimizer, or ModelOpt) is a library comprising state-of-the-art model optimization techniques including quantization, pruning, Neural Architecture Search (NAS), distillation, speculative decoding and sparsity to accelerate models. [Input] Model Optimizer currently supports inputs of a Hugging Face, PyTorch or ONNX model. [Optimize] Model Optimizer provides Python APIs for users to easily compose the above model optimization techniques and export an optimized quantized checkpoint. Model Optimizer is also integrated with NVIDIA Megatron-Bridge, Megatron-LM and Hugging Face Accelerate for training required inference optimization techniques. [Export for deployment] Seamlessly integrated within the NVIDIA AI software ecosystem, the quantized checkpoint generated from Model Optimizer is ready for deployment in downstream inference frameworks like SGLang, TensorRT-LLM, TensorRT, or vLLM. The unified Hugging Face export API now supports both transformers and diffusers models. Previous News To install stable release packages for…