What's the difference between fine-tuning and prompt engineering?

Fine-tuning vs Prompt Engineering

Both adapt an LLM to a specific task, but they differ fundamentally in cost, flexibility, and when to use them.

Prompt Engineering

What it is: Crafting the input text (system prompt, instructions, examples) to steer the model's output — no weight changes.

pythonCopy
from openai import OpenAI
client = OpenAI()

system_prompt = '''You are a senior Python code reviewer.
When given code, you:
1. Identify bugs and security issues
2. Suggest improvements with explanations
3. Rate code quality 1-10
Always respond in this format: Issues: ... Improvements: ... Rating: X/10'''

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": "Review this: def add(a,b): return a+b"}
    ]
)

Fine-tuning

What it is: Updating model weights on a task-specific dataset to permanently change the model's behaviour.

pythonCopy
from transformers import TrainingArguments
from trl import SFTTrainer
from datasets import load_dataset

dataset = load_dataset("json", data_files="training_data.jsonl")

trainer = SFTTrainer(
    model=model,  # Base or instruction-tuned model
    args=TrainingArguments(
        output_dir="./fine-tuned-model",
        num_train_epochs=3,
        per_device_train_batch_size=4,
        learning_rate=2e-4,
    ),
    train_dataset=dataset["train"],
    dataset_text_field="text",
)
trainer.train()

Decision Framework

When to Choose Each

Choose prompt engineering first if:

• You can get good results with a well-crafted system prompt
• Task changes frequently
• Budget is limited

Choose fine-tuning if:

1. You need consistent format/style the prompt alone can't achieve
2. You have proprietary domain knowledge not in the base model
3. You want to compress a long system prompt into model weights (faster inference)
4. You need style/format consistency at scale

Best practice in production: Start with prompt engineering. Fine-tune only when prompt engineering plateaus. Always measure accuracy improvement before committing to a fine-tuning pipeline.