Exponential Functions

An exponential function has the variable in the exponent: f(x) = b^x. The base b is a positive constant ≠ 1. This single shape underlies population growth, radioactive decay, compound interest, and most probability distributions in ML.

Behavior at a glance

• f(x) = 2^x: increasing, passes through (0, 1), asymptote at y = 0 from above.
• f(x) = (1/2)^x: decreasing (decay), same passes-through-(0, 1) and same asymptote.
• For any positive base ≠ 1: f(0) = 1 always (anything to the 0 is 1).

Solving exponential equations by matching bases

When both sides can be rewritten with the same base, the exponents must be equal.

2^x = 32
2^x = 2^5
x = 5

3^(x + 1) = 27
3^(x + 1) = 3³
x + 1 = 3
x = 2

If you can’t easily match bases, you’ll need logarithms (next week).

The natural base e ≈ 2.71828…

Why e and not 10 or 2? Because e^x is its own derivative: d/dx [e^x] = e^x. This makes calculus dramatically simpler, and it’s why nearly every continuous probability distribution and ML activation function uses e.

Compound interest — the prototypical exponential

A = P · (1 + r/n)^(nt)

P = principal, r = annual rate, n = compounding periods per year, t = years. As n → ∞, this becomes A = P · e^(rt) (continuous compounding). The number e appears naturally as the limit of compounding.

ML connection — sigmoid, softmax, and Gaussian

Sigmoid (logistic regression’s activation):

σ(x) = 1 / (1 + e^(−x))

Maps any real number to (0, 1) — perfect for outputting a probability.

Softmax (multi-class classifier):

softmax(x_i) = e^(x_i) / Σ e^(x_j)

Turns a vector of real numbers into a probability distribution.

Gaussian (normal distribution):

f(x) = (1 / (σ√(2π))) · e^(−(x − μ)² / (2σ²))

Used everywhere — anomaly detection, Gaussian processes, Bayesian inference.

All three rely on the structural fact that e^x is positive, smooth, and grows fast — and its derivative is itself, which makes gradient descent through them analytically clean.