Explain type hints and why they're important in production Gen AI code.

Type Hints in Python

Type hints (introduced in Python 3.5+) annotate variables, function parameters, and return values with their expected types. They're checked by static analysis tools like

mypy

Basic Syntax

pythonCopy
# Variables
name: str = "Alice"
count: int = 0
embeddings: list[float] = []

# Functions
def embed_text(text: str, model: str = "text-embedding-3-small") -> list[float]:
    ...

def search_docs(query: str, k: int = 5) -> list[dict[str, str]]:
    ...

Advanced Type Hints for Gen AI Code

pythonCopy
from typing import Optional, Union, TypedDict, Callable, Generator
from collections.abc import Iterator

# Optional — value can be None
def get_answer(question: str, context: Optional[str] = None) -> str:
    ...

# Union — one of several types (use | in Python 3.10+)
def process(input: str | list[str]) -> list[str]:
    if isinstance(input, str):
        return [input]
    return input

# TypedDict — typed dictionary structure
class Document(TypedDict):
    id: str
    content: str
    embedding: list[float]
    metadata: dict[str, str]

# Callable — function type
def apply_transform(
    documents: list[str],
    transform: Callable[[str], str]
) -> list[str]:
    return [transform(doc) for doc in documents]

# Generator for streaming
def stream_tokens(prompt: str) -> Generator[str, None, None]:
    for token in llm_stream(prompt):
        yield token

Real-World Production Example

pythonCopy
from typing import Optional, TypedDict
from enum import Enum

class Difficulty(str, Enum):
    EASY = "easy"
    MEDIUM = "medium"
    HARD = "hard"

class RAGResult(TypedDict):
    answer: str
    sources: list[str]
    confidence: float
    latency_ms: int

class RAGPipeline:
    def __init__(
        self,
        embedding_model: str,
        llm_model: str,
        top_k: int = 5,
        temperature: float = 0.0,
    ) -> None:
        self.embedding_model = embedding_model
        self.llm_model = llm_model
        self.top_k = top_k
        self.temperature = temperature

    def query(
        self,
        question: str,
        user_id: Optional[str] = None,
        filter_metadata: Optional[dict[str, str]] = None,
    ) -> RAGResult:
        ...

Why Type Hints Matter in Production Gen AI

mypy

Running Type Checks

bashCopy
# Install mypy
pip install mypy

# Check a file
mypy src/rag_pipeline.py

# Check strict mode (recommended for new projects)
mypy --strict src/

Common Gen AI Type Patterns

pythonCopy
from typing import TypeVar, Generic

T = TypeVar("T")

# Generic wrapper for typed responses
class LLMResponse(Generic[T]):
    data: T
    raw_text: str
    usage: dict[str, int]

# Type alias for embeddings
Embedding = list[float]
EmbeddingBatch = list[Embedding]

def batch_embed(texts: list[str]) -> EmbeddingBatch:
    ...

Best practice: Add type hints to all public interfaces (function signatures) from the start. Use

textCopy

mypy --strict

in CI to catch type errors before they reach production.