What is Deep Learning in AI?

What is Deep Learning in AI?

Deep Learning is a subset of Machine Learning that uses artificial neural networks with many layers (hence "deep") to learn hierarchical representations from raw data.

Why "Deep"?

The "depth" refers to many stacked layers of neurons, each learning increasingly abstract features:

textCopy
Input → [Layer 1: edges] → [Layer 2: shapes] → [Layer 3: faces] → Output
         (low-level)          (mid-level)          (high-level)

Neural Network Basics

pythonCopy
import torch
import torch.nn as nn

class SimpleNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.layers = nn.Sequential(
            nn.Linear(784, 256),   # Layer 1: 784 inputs → 256 neurons
            nn.ReLU(),             # Activation function
            nn.Linear(256, 128),   # Layer 2: 256 → 128
            nn.ReLU(),
            nn.Linear(128, 10),    # Output: 10 classes
        )

    def forward(self, x):
        return self.layers(x)

model = SimpleNN()

Key Architectures

How Training Works

1. Forward pass — input flows through layers, produces prediction
2. Loss calculation — compare prediction to ground truth
3. Backpropagation — calculate gradients of loss w.r.t. weights
4. Weight update — optimizer adjusts weights to reduce loss
5. Repeat — thousands of iterations over the dataset

Deep Learning vs Classic ML

Why Deep Learning Powers Gen AI

• LLMs (GPT-4, Claude) = Transformer deep learning on text
• DALL-E, Stable Diffusion = Diffusion + Transformer on images
• Whisper = Transformer on audio spectrograms
• AlphaFold = Transformer on protein sequences

Key Concepts to Know