What is an AI detector?

What is an AI Detector?

An AI detector is a tool that attempts to determine whether a piece of text, image, or code was generated by an AI system (like ChatGPT, Claude, or DALL-E) or created by a human.

Types of AI Detectors

How Text AI Detectors Work

Method 1: Perplexity Analysis

LLMs generate text that is statistically "expected" — low perplexity means the model predicted each token with high confidence. AI-written text tends to have lower perplexity than human text.

pythonCopy
import torch
from transformers import GPT2LMHeadModel, GPT2TokenizerFast

def compute_perplexity(text: str) -> float:
    '''Lower perplexity may indicate AI-written text'''
    tokenizer = GPT2TokenizerFast.from_pretrained("gpt2")
    model = GPT2LMHeadModel.from_pretrained("gpt2")
    model.eval()

    encodings = tokenizer(text, return_tensors="pt")
    input_ids = encodings.input_ids

    with torch.no_grad():
        outputs = model(input_ids, labels=input_ids)
        loss = outputs.loss

    return torch.exp(loss).item()

human_text = "Yesterday I went to the market and bought some fresh tomatoes."
ai_text = "In the contemporary landscape, the acquisition of fresh produce..."

print(f"Human perplexity: {compute_perplexity(human_text):.2f}")  # Higher
print(f"AI perplexity: {compute_perplexity(ai_text):.2f}")        # Lower

Method 2: Burstiness Analysis

Human writing varies in sentence length and complexity (high burstiness). AI tends to produce more uniform, consistent sentence structures (low burstiness).

Method 3: Classifier-Based Detection

Train a binary classifier on AI-generated vs human-written text:

pythonCopy
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Pre-trained AI text classifier
model_name = "Hello-SimpleAI/chatgpt-detector-roberta"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

def detect_ai_text(text: str) -> dict:
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
    with torch.no_grad():
        logits = model(**inputs).logits
    probs = torch.softmax(logits, dim=-1)
    return {
        "human_probability": probs[0][0].item(),
        "ai_probability": probs[0][1].item()
    }

result = detect_ai_text("The mitochondria is the powerhouse of the cell.")
print(f"AI probability: {result['ai_probability']:.2%}")

Reliability and Limitations

False Positive Problem

Research shows AI detectors have false positive rates of 10-20% for human-written text — meaning 1 in 5-10 legitimate human essays might be flagged as AI-written. This is why most academic institutions are cautious about relying solely on AI detectors.

Key Commercial Detectors

Bottom line: AI detectors are useful signals but not definitive evidence — they should not be used as sole proof of academic dishonesty or policy violations.