Fine-tuning
Fine-tune VRAM calculator
Inference VRAM is the easy half — fine-tuning eats 5–20× more. We model the four real cost lines (weights, gradients, optimizer state, activations) for QLoRA, LoRA, and full fine-tunes.
Setup
Method
VRAM needed
6.1 GBSmallest GPU class that fits: RTX 4060 Ti 16 GB
- Weights3.73 GB
- Gradients0.07 GB
- Optimizer state0.15 GB
- Activations1.18 GB
Tips to reduce VRAM
- QLoRA is 8–10× lighter than full fine-tune for ~95 % of the quality
- Gradient checkpointing trades ~30 % VRAM for ~25 % slower steps
- Reduce batch size to 1 + use gradient accumulation
- AdamW 8-bit (bitsandbytes) cuts optimizer state 4×
- Shorter sequences during training, then ramp up
Estimates ±20 %. Real numbers depend on framework (Unsloth saves 30–60 %), gradient checkpointing, and CPU-offload settings. Inference VRAM for this model → · Rent a GPU big enough →