From Functions to Neural Networks

What if I told you that everything we know about the world - every rule, every pattern, every piece of logic - can be described by functions? This chapter starts with the most basic question: what IS a function? Then we'll journey from simple mathematical relationships to the neural networks that power modern AI.

What Are Functions?

Let's start with the fundamental concept:

1. Functions Describe the World: A function is simply a way to describe relationships. Functions capture the logic and knowledge of how things work in our world
2. Input → Output Mapping: Functions take something in (input) and give something back (output) based on a rule or pattern
3. Examples Everywhere: From calculating triangle areas to predicting weather, functions describe cause-and-effect relationships
4. The AI Connection: What if we could learn these functions automatically from data instead of writing them manually?

From Simple Rules to Complex Patterns

Understanding how we move from basic mathematical relationships to intelligent systems:

1. We Start with Simple Math: Basic functions like f(x) = wx + b describe straight-line relationships
2. Real World is Complex: Most relationships aren't straight lines - they're curved, twisted, and hard to describe with simple equations
3. The Learning Problem: Sometimes we can't see through the complexity to find the exact function, so we need to guess and adjust
4. Neural Networks as Function Learners: These systems learn to approximate complex functions by adjusting parameters automatically

Mathematical Foundation

Let's trace the journey from basic functions to neural networks step by step:

// Starting point: Simple geometric relationship
// Triangle area: base × height ÷ 2
area_triangle(base, height) = (base × height) / 2

// Linear function: straight line relationship  
f(x) = wx + b

// The guessing game: What if we don't know w and b?
// We have some (x, y) coordinate pairs and need to find the line
// Example points: (2, 2), (3, 2.5), (4, 3), (5, 3.5)
// We adjust w and b until our function fits the data

// When one function isn't enough:
// Some patterns can't be described by straight lines
// We need curves, which means we need non-linear functions

// Adding activation functions for non-linearity:
f(x) = g(wx + b)
// where g() could be ReLU: g(z) = max(0, z)

// Multiple inputs: real world has many factors
f(x₁, x₂) = g(w₁x₁ + w₂x₂ + b)

// When we can't find the perfect function:
// We find one that's "close enough" to the real answer
// This is the core idea behind neural networks

Core Components

1. Input (X): The information we feed into our function - could be numbers, images, text
2. Weights (W): The parameters we adjust to make our function fit the data better
3. Bias (b): An offset that helps our function be more flexible
4. Activation Function (g): The non-linear transformation that lets us learn curved relationships
5. Output (Y): The result our function produces - our best guess at the answer

The Journey to Neural Networks

How do we go from f(x) = wx + b to systems that understand language and recognize images?

The Linear Limitation Problem

The Issue:
Linear functions like f(x) = wx + b can only draw straight lines. But most real-world relationships are curved and complex.

The Solution - Add Non-linearity:

• Introduce activation functions: f(x) = g(wx + b)
• Popular choices: ReLU (cuts off negative values), sigmoid (S-shaped curve)
• Now our function can bend and curve to fit complex patterns

The Multi-Input Reality

When there are multiple inputs, input layer increases:

Real World Complexity:
Most decisions depend on multiple factors. A simple example: predicting house prices depends on size, location, age, etc.

// Single input: oversimplified
price = f(size) = w × size + b

// Multiple inputs: more realistic  
price = f(size, location, age) = w₁×size + w₂×location + w₃×age + b

// Matrix form for efficiency:
price = f(X) = W·X + b
where X = [size, location, age]ᵀ and W = [w₁, w₂, w₃]

Stacking Functions: Deep Networks

Add more active function layer because:

g(w₁x₁ + w₂x₂ + b) represents a hidden layer — and we can stack many of them to build deep networks.

This structure is called a neural network, and passing inputs through its layers is known as forward propagation.

The Power of Layers:
When one function layer isn't enough, we stack them. Each layer transforms the data a little bit, and together they can learn incredibly complex patterns.

// Single layer network
output = g(W₁·input + b₁)

// Two layer network (one hidden layer)
hidden = g(W₁·input + b₁)
output = g(W₂·hidden + b₂)

// Deep network: many layers of transformation
input → layer₁ → layer₂ → ... → layerₙ → output

// Each layer learns different aspects:
// - Early layers: simple patterns (edges in images, common words)
// - Later layers: complex concepts (objects, meanings)

---

At its core, machine learning is about learning functions that describe the world. From simple rules like f(x) = wx + b to neural networks that translate languages or recognize faces, it's all about approximating complex relationships in data. It all clicks when you see that every model is just a function figuring out patterns—from basic math to powerful AI, it’s the same idea at heart. It's all functions—just deeper, smarter, and more flexible ones.