Fine-Tune an LLM for Your Codebase with Unsloth

Fine-tuning lets you adapt a general-purpose model to your specific coding style, internal APIs, and domain knowledge. With Unsloth, you can do this on a consumer GPU in under 2 hours.

---

When Should You Fine-Tune?

Fine-tune when:

• The model doesn't know your internal APIs or naming conventions
• You want consistent code style that matches your existing codebase
• You're building a coding assistant for a specialized domain (embedded systems, biotech DSLs, etc.)
• You process thousands of similar tasks and want faster/cheaper inference

Don't fine-tune when:

• Prompt engineering + RAG can solve the problem (it usually can)
• Your codebase changes rapidly (the model will be out of date quickly)
• You have less than 100 good training examples

---

Prerequisites

• Python 3.10+
• NVIDIA GPU with 16GB+ VRAM (or Google Colab T4/A100)
• Dataset of code examples in your style

---

Step 1: Install Unsloth

# Install Unsloth with CUDA support
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps trl peft accelerate bitsandbytes

---

Step 2: Prepare Your Dataset

Your dataset should be a collection of instruction-response pairs. Format:

# dataset.jsonl
{"instruction": "Write a Prisma query to get all active users with their posts",
 "response": "const users = await prisma.user.findMany({\n  where: { isActive: true },\n  include: { posts: { where: { published: true } } }\n});"}

{"instruction": "Create a Next.js server action to update user profile",
 "response": "'use server';\n\nexport async function updateProfile(data: FormData) {\n  const session = await auth();\n  if (!session?.user) throw new Error('Unauthorized');\n  ...\n}"}

Minimum dataset size: 100 examples (500+ for meaningful improvement)

Collecting Training Data

import json
import os

examples = []

# Method 1: Extract from your git history
# git log --all --diff-filter=A -- "*.ts" | collect commits
# where the commit message describes what was added

# Method 2: Use your existing codebase + Claude to generate descriptions
# For each function, ask Claude to write the instruction that would generate it

with open('dataset.jsonl', 'w') as f:
    for example in examples:
        f.write(json.dumps(example) + '\n')

---

Step 3: Load the Base Model

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/llama-3-8b-Instruct",  # or "unsloth/deepseek-coder-7b"
    max_seq_length = 4096,
    dtype = None,          # Auto-detect Float16/BFloat16
    load_in_4bit = True,   # Use 4-bit quantization (reduces VRAM by ~50%)
)

Available base models:

• unsloth/llama-3-8b-Instruct — Best general coding (4.7GB VRAM with 4-bit)
• unsloth/deepseek-coder-7b — Best for code-specific tasks
• unsloth/mistral-7b-instruct-v0.3 — Fast and efficient

---

Step 4: Add LoRA Adapters

LoRA (Low-Rank Adaptation) fine-tunes only a small fraction of the model's weights:

model = FastLanguageModel.get_peft_model(
    model,
    r = 16,              # LoRA rank — higher = more capacity, more memory
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                       "gate_proj", "up_proj", "down_proj"],
    lora_alpha = 16,
    lora_dropout = 0,
    bias = "none",
    use_gradient_checkpointing = "unsloth",  # 30% less VRAM
    random_state = 42,
)

---

Step 5: Train

from trl import SFTTrainer
from transformers import TrainingArguments
from datasets import load_dataset

dataset = load_dataset("json", data_files="dataset.jsonl", split="train")

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    dataset_text_field = "text",
    max_seq_length = 4096,
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        max_steps = 200,          # For quick experiments; use 1000+ for production
        learning_rate = 2e-4,
        fp16 = not torch.cuda.is_bf16_supported(),
        bf16 = torch.cuda.is_bf16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        output_dir = "outputs",
    ),
)

trainer.train()

Training time: ~20 min on T4, ~8 min on A100

---

Step 6: Evaluate Your Model

FastLanguageModel.for_inference(model)

inputs = tokenizer(
    ["Write a TypeScript function to validate email format"],
    return_tensors = "pt"
).to("cuda")

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))

Compare the output to what the base model (without fine-tuning) would produce.

---

Step 7: Save and Deploy

# Save as LoRA adapter (small, ~50MB)
model.save_pretrained("my-coder-lora")
tokenizer.save_pretrained("my-coder-lora")

# Or merge and save as full model (for deployment)
model.save_pretrained_merged("my-coder-merged", tokenizer)

# Or export to GGUF for Ollama
model.save_pretrained_gguf("my-coder", tokenizer, quantization_method = "q4_k_m")

Load in Ollama

# Create Modelfile
cat > Modelfile << EOF
FROM ./my-coder.Q4_K_M.gguf
SYSTEM "You are a TypeScript expert who follows [company] coding conventions."
EOF

ollama create my-coder -f Modelfile
ollama run my-coder

---

Troubleshooting

Out of VRAM: Reduce per_device_train_batch_size to 1, or use a smaller model (7B vs 13B).

Model doesn't improve: Your dataset may be too small or lack diversity. Aim for at least 500 varied examples.

Output diverges from base model: Reduce LoRA rank (r=8 instead of r=16) or add more regularization.

---

Next Steps

• Explore Axolotl for more advanced training configurations
• Use H2O LLM Studio for a no-code fine-tuning UI
• Check our Model Training tools list for the full ecosystem