knowledge-distillation

from Orchestra-Research/AI-research-SKILLs

Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source