L
learnllm
Back to Catalog
Pro Track7.5 Hours•Level: Advanced•Updated: October 2026

Post-Training: LoRA, QLoRA & Direct Preference Optimization

Take full ownership of model weights. Master low-rank matrix decomposition, 4-bit normal float quantization, ultra-efficient Unsloth training loops, and preference alignment via DPO without RL reward models.

Preview Free Lesson

Curriculum Outcomes & Engineering Deliverables

Decompose full parameter updates W + delta_W into low-rank matrices B * A
Quantize base models into 4-bit NF4 with double quantization via bitsandbytes
Train Llama-3 and DeepSeek models on single consumer GPUs using Unsloth
Implement DPO loss equations to steer models towards custom style and safety

Included Engineering Deliverables

Full 4-bit NormalFloat QLoRA training pipeline script
Custom autograd fused cross-entropy Triton loss module
Direct Preference Optimization (DPO) training harness
Synthetic CoT dataset curation and LLM-as-a-judge filter scripts

Curriculum Modules

3 Modules • 5 Lessons
Module 012 Lessons

Low-Rank Adaptation (LoRA) Theory

The mathematical derivation that made fine-tuning 100x cheaper.

Module 022 Lessons

High-Efficiency Training Loops

Accelerate training with Unsloth and custom autograd kernels.

Module 031 Lessons

Direct Preference Optimization (DPO)

Align models with human preferences without training complex reward models.

Hardware & GPU Compute

Baseline GPUNVIDIA RTX 3090, 4090 or A100 (80GB)
VRAM Requirement16GB - 24GB VRAM minimum for 8B models
Recommended Configuration1x NVIDIA RTX 4090 (24GB) or RunPod 1x A100 (80GB) for 70B

Prerequisites & Tech Stack

Stack Components
UnslothBitsAndBytesPEFTTRLDeepSpeedPyTorch
Knowledge Prerequisites
  • PyTorch fundamentals
  • Understanding of backpropagation
  • LLM Foundations