This video showcases how one can perform supervised fine-tuning (SFT for short) on an open-source large language model. Supervised fine-tuning, also called instruction tuning, takes a base model that has been pre-trained on billions of tokens from the web, and turns it into a useful chatbot by training on human-written instructions and corresponding completions.
The video uses the following notebook: https://github.com/NielsRogge/Transfo....
The notebook is an annotated version of the scripts of the Alignment Handbook by Hugging Face: https://github.com/huggingface/alignm....
In terms of hardware, an RTX 4090 NVIDIA GPU was leveraged on Runpod, which has 24GB of RAM: https://www.runpod.io/serverless-gpu.
