Skip to content

gptq

from Orchestra-Research/AI-research-SKILLs

Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or

v1.0.0MIT
451
Lines
1,422
Words
17
Code Blocks

Languages

bashpython
10-optimization/gptq/SKILL.md