04 — Training¶
This guide explains every option in the 🚀 Training tab. You don't need to understand all of them — the defaults are carefully chosen to work well for most people.
The Basic Workflow¶
- Upload your data (📂 Data tab)
- Choose a base model
- Choose a training mode
- Choose a preset (or adjust manually)
- Click ▶ Start Training
- Watch the loss curve go down
- Test in the 💬 Inference tab
Choosing a Base Model¶
The base model is the pre-trained AI that you'll be teaching. Think of it as hiring a smart employee — you're not creating intelligence from scratch, you're specialising existing intelligence.
Model List¶
| Model | VRAM needed | Speed | Quality | Good for |
|---|---|---|---|---|
distilgpt2 |
< 1 GB | Very fast | Low | Testing only |
gpt2 |
1 GB | Fast | Low | Quick experiments |
facebook/opt-125m |
1 GB | Fast | Low | Quick experiments |
facebook/opt-350m |
2 GB | Medium | Medium | Small tasks |
EleutherAI/pythia-70m |
< 1 GB | Very fast | Low | Testing |
EleutherAI/pythia-160m |
1 GB | Fast | Low | Quick experiments |
TinyLlama/TinyLlama-1.1B-Chat-v1.0 |
4 GB | Medium | Good | Most tasks |
mistralai/Mistral-7B-v0.1 |
16 GB | Slow | Excellent | Production use |
Auto-recommendation: The tool automatically suggests the best model for your hardware based on available VRAM. You'll see this recommendation pre-selected when you open the Training tab.
You can also type any model ID from HuggingFace Hub into the "Or enter any HuggingFace model ID" box. For example: meta-llama/Llama-2-7b-hf.
Training Mode¶
SFT — Supervised Fine-Tuning (Default)¶
The standard mode. You show the model many examples of the behaviour you want, and it learns to replicate it.
Use this when: - You want the model to answer questions in a specific style - You're teaching domain-specific knowledge - You're building a chatbot with a particular personality
Required data columns: instruction + output, or text
DPO — Direct Preference Optimization¶
A more advanced technique. Instead of just showing correct answers, you show pairs of answers where one is better than the other. The model learns to prefer the good style.
Use this when: - You want to align the model with human preferences - You have existing SFT output and want to improve quality - You want to make the model safer or more helpful
Required data columns: prompt, chosen, rejected
DPO Beta: Controls how strongly the model penalises bad responses. Default (0.1) works well for most cases. Higher values make the model more conservative.
Vision-language models¶
When the chat data has images (see 03 — Data Preparation), SFT switches to vision-language fine-tuning automatically — no extra setting. Use a vision-language model such as Qwen/Qwen2.5-VL-3B-Instruct (Custom model field / --model).
- The model's processor turns images into image tokens; the loss covers the assistant's answer only.
- LoRA (any variant), QLoRA on a GPU and Full fine-tuning are supported; Prefix/Prompt Tuning, IA3, Unsloth, packing and DPO are not.
- Sequences are not truncated (cutting image tokens would break the batch), so Max Sequence Length doesn't apply — keep conversations reasonably short.
- The processor is saved next to the adapter and the model card is tagged
image-text-to-text. - Needs
torchvisionmatching yourtorch(see 01 — Installation). - The Inference and Evaluation tabs are text-only for now; load a vision adapter in Python as the model card shows.
Training Presets¶
Presets are shortcuts that set the number of epochs and learning rate for you.
| Preset | Epochs | Learning Rate | Time | Use when |
|---|---|---|---|---|
| Quick (1 epoch) | 1 | 5×10⁻⁴ | Minutes | Testing, first runs |
| Balanced (3 epochs) | 3 | 2×10⁻⁴ | 30–60 min | Most real projects |
| Accurate (5 epochs) | 5 | 1×10⁻⁴ | 1–2 hours | Production models |
| Advanced | Custom | Custom | Depends | Expert tuning |
What is an epoch? One epoch means the model has seen every example in your dataset once. More epochs = more learning, but too many causes overfitting (the model memorises instead of generalising).
PEFT Methods (Parameter-Efficient Fine-Tuning)¶
Fine-tuning a large model means updating billions of parameters — which requires enormous VRAM. PEFT methods work around this by only updating a small fraction of the parameters.
| Method | VRAM usage | Speed | Quality | Use when |
|---|---|---|---|---|
| Auto | Varies | — | — | Let the tool decide (recommended) |
| LoRA | Low | Fast | Excellent | Default choice for most users |
| QLoRA Enhanced | Very low | Medium | Very good | Limited VRAM (< 8 GB) |
| Full Fine-tuning | Very high | Slow | Best possible | You have a lot of VRAM (> 40 GB) |
| Prefix Tuning | Very low | Fast | Good | Extremely limited hardware |
| Prompt Tuning | Minimal | Very fast | Moderate | Fastest possible training |
| IA3 | Very low | Fast | Good | Smallest adapter (~0.01% of the weights, ≈600k for a 7B Llama); use a higher learning rate (1e-3–3e-3) |
LoRA — The Recommended Choice¶
LoRA (Low-Rank Adaptation) is the most popular PEFT method. It inserts small trainable matrices into the model and only updates those. The result is: - 90%+ less VRAM than full fine-tuning - Only slightly worse quality - Very fast training
LoRA Settings (in Advanced Hyperparameters): - LoRA Rank (default: 8) — Higher = more capacity to learn, more VRAM. Try 16 or 32 for better quality. - LoRA Alpha (default: 16) — Usually set to 2× the rank. Controls the strength of the adaptation. - LoRA variant (default: LoRA) - LoRA — the standard method. - rsLoRA — scales by alpha/√rank instead of alpha/rank, so higher ranks (32–128) train more steadily. - DoRA — learns the size and the direction of each weight change separately; often better at low rank, somewhat slower to train. Unsloth doesn't support DoRA, so it is skipped (with a note) when DoRA is chosen. A DoRA adapter can't be compared with its base model in the Evaluation tab (PEFT cannot switch DoRA off per request).
LoRA is applied to every linear layer of the transformer blocks — attention and MLP (q/k/v/o_proj, gate/up/down_proj, or c_attn/c_fc/c_proj on GPT-2) — as current practice recommends; the output layer is left alone.
QLoRA Enhanced — For Low-VRAM Machines¶
QLoRA Enhanced uses 4-bit quantisation (NF4 format) to load the model in a fraction of the VRAM, then applies LoRA on top. This lets you fine-tune a 7B model on a GPU with just 8 GB of VRAM.
How to enable: Select
QLoRA Enhancedfrom the PEFT Method radio buttons. The NF4 settings are applied automatically.
Additional Training Options¶
Unsloth (🚀 2–5× faster)¶
Unsloth is a library that dramatically speeds up training and reduces VRAM usage. If you installed it during setup, check this box for every training run. There's no downside.
Unsloth is only compatible with LoRA training on NVIDIA GPUs.
Smart Chat Template (💬 recommended ON)¶
Automatically formats your data using the correct chat format for the model you chose (e.g. Llama 3 uses <|start_header_id|> headers, Qwen uses ChatML <|im_start|> tags, Mistral uses [INST] blocks). This improves output quality significantly for chat models.
Flash Attention 2 (⚡ for modern GPUs)¶
A highly optimised attention algorithm that's 2–3× faster and uses less VRAM. Requires: - An NVIDIA GPU with Ampere architecture or newer (RTX 3000 series or later) - bfloat16 precision support
System Prompt¶
The text the model will use as its "identity" during inference. For example:
You are a friendly customer support agent for Acme Corp. Always be polite and helpful.
Heretic Mode 🔓¶
See 10 — Advanced Usage for a full explanation. In short: removes built-in safety restrictions from the model after training.
Advanced Hyperparameters¶
Click ⚙️ Advanced hyperparameters to expand these options. You only need these if you want precise control.
| Parameter | Default | What it does |
|---|---|---|
| Learning Rate | 2×10⁻⁴ | How fast the model updates its weights. Too high → unstable training. Too low → slow learning. |
| Epochs | 3 | How many times the model sees the full dataset |
| Batch Size | 2 | How many examples processed at once. Higher = faster but uses more VRAM. |
| Gradient Accumulation | 4 | Simulates a larger batch size without using extra VRAM. |
| Max Sequence Length | 256 | Maximum number of tokens per example. Longer = more VRAM. |
| Warmup Steps | 100 | Slowly ramps up the learning rate at the start to avoid unstable early training. |
| Early Stopping Patience | 3 | Stops training early if the model stops improving. 0 = disabled. |
| LR Scheduler | cosine | How the learning rate changes over time. cosine is best for most cases. |
| Gradient Checkpointing | OFF | Saves VRAM at the cost of ~20% slower training. Enable if you're running out of VRAM. |
| Resume from checkpoint | OFF | Continue a previous training run that was interrupted. |
Reading the Training Log¶
The log on the right side shows:
🚀 Loading model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
✅ Model loaded (PEFT: LoRA, rank=8)
📊 Dataset: 450 examples → 360 train / 90 eval
🏃 Epoch 1/3 — Step 50/113 — Train loss: 1.847 — Eval loss: 1.923
🏃 Epoch 2/3 — Step 100/113 — Train loss: 1.432 — Eval loss: 1.541
🏃 Epoch 3/3 — Step 113/113 — Train loss: 1.201 — Eval loss: 1.318
✅ Training complete! Time: 0:18:42
📁 Model saved to: runs/my-run
What to watch for: - Train loss decreasing — the model is learning ✅ - Eval loss decreasing too — it's generalising, not just memorising ✅ - Eval loss rising while train loss falls — overfitting. Reduce epochs or get more data. ⚠️
The 📉 Loss Curve below the log plots these values visually.
Runs, resuming and reproducibility¶
- Every training run is saved in its own folder:
runs/<run name>/(Docker:/app/models/<run name>/). Leave Run name empty for an automaticdate-time-modename. - To continue a run, enter its name and tick Resume from last checkpoint. Using an existing name without Resume is refused, so a run is never overwritten by accident.
- Each run folder contains
run_config.yaml: the base model, every setting, the seed, a fingerprint (row count + SHA-256) of the exact data used, and the library versions. The same seed with the same data and settings reproduces the same weights. Replay a run from the CLI withpython main.py train --config runs/<name>/run_config.yaml --data <file>. - Experiment tracking: pick a tracker (Trackio, Weights & Biases, MLflow or TensorBoard) to log loss curves and settings. Only trackers installed in your environment are listed, e.g.
pip install trackiothentrackio showto open its dashboard.
What the model is trained on¶
- Instruction data (
instruction/output): the loss is computed on the response only — the model is not trained to reproduce your prompts — and an end-of-sequence token is added, so the fine-tuned model learns when to stop. - Plain text (
text): the model is trained on the whole text. - Chat data (
messages): the model is trained on the last assistant reply, using the model's chat template. - Eval split (default 0.1) holds that share of rows out to measure eval loss; set it to 0 to train on everything.
- Tokens per example are reported at the start of training, with a warning if examples are longer than Max Sequence Length (they get truncated). Max Sequence Length applies to DPO too.
- Mixed precision is chosen automatically: bfloat16 on GPUs that support it, float16 on older GPUs, full precision on CPU.
- 📦 Sequence packing (optional): packs short examples into full-length sequences, which is much faster on short data. It needs ⚡ Flash Attention 2 on a CUDA GPU; without it the setting is skipped, because packed examples would leak into each other.
- Long sequences (optional, SFT on a CUDA GPU; skipped with a note otherwise):
- 🧠 Activation offloading keeps activations in CPU RAM between the forward and backward pass — much less GPU memory for long sequences, somewhat slower. Combine with gradient checkpointing for the longest inputs.
- 📏 Padding-free batches concatenate each batch without padding, so no memory or compute is spent on pad tokens. Needs ⚡ Flash Attention 2 (other attention would mix the examples).
Embedding Models (search / RAG)¶
The 🔎 Embeddings tab fine-tunes an embedding model — the model a search engine or a RAG pipeline uses to find the passages that answer a question. A general-purpose embedding model often misses domain vocabulary (product names, internal terms, legal or medical language); a short fine-tune on your own (question, passage) pairs fixes much of that.
Data (CSV / JSON / JSONL):
| Columns | Pairs used |
|---|---|
anchor, positive (optional negative) |
as given |
instruction, context — the JSONL from create training data from your documents |
question ↔ the passage it was written from (best for RAG) |
instruction, output / prompt, completion / chats |
question ↔ answer |
Settings:
- Embedding model — pick a suggestion or type any Hub id or folder. all-MiniLM-L6-v2
(22M, English) trains in minutes on a CPU; multilingual-e5-small and bge-m3 cover other
languages; Qwen3-Embedding-0.6B is stronger and wants a GPU.
- Method — Full fine-tuning (default; embedding models are small) or LoRA for large
models. LoRA is merged into the model before saving, so the result is a normal model.
- Batch size — bigger is better: every other passage in the batch serves as a wrong answer
for each question (in-batch negatives, MultipleNegativesRankingLoss).
- Matryoshka (on) — also trains the first 512 / 256 / 128 / 64 dimensions, so you can store
shorter vectors later (truncate_dim) and keep most of the quality.
- Mine hard negatives — before training, the base model finds, for each question, a
passage that looks similar but is not its answer (passages scoring within 5 % of the right
one are skipped as likely duplicates). Harder examples, usually better results.
- Held-out share — pairs kept aside to measure retrieval (with fewer than 5 held-out
pairs, the training pairs are used and the status says so).
- Query prompt — a prefix some models expect on queries (query: for E5). Empty uses the
model's own; it is saved with the model.
Result: the status shows retrieval quality before → after — NDCG@10 (ranking quality),
MRR@10 (how high the right passage ranks) and Recall@10 (how often it is in the top 10)
— each held-out question searched among all passages. The model is saved to
runs/<run name>/ with run_config.yaml and a model card, ready for push to the Hub or any
tool that loads sentence-transformers models:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("runs/my-embedder")
scores = model.similarity(
model.encode(["How do I reset my password?"]),
model.encode(["To reset your password, open Settings …"]),
)
Needs sentence-transformers (in requirements.txt; or pip install -e ".[embedding]" from the repo folder).
From the CLI: python main.py embed --data pairs.jsonl --output ./my-embedder.
Stopping Training Early¶
Click ⏹ Stop at any time. The model will finish the current step, save what it has, and stop. Your partially-trained model is still usable.
Clearing GPU Memory¶
If you get out-of-memory errors or want to free up VRAM between runs, click 🧹 Clear GPU Cache. This releases any cached tensors without restarting the app.
Next Step¶
→ 05 — RLHF Pipeline: Advanced alignment with Reward Models, GRPO, ORPO and KTO.
→ 06 — Inference: Test your trained model.