6. Systems of 3 Variables & Intro to Matrices

Systems of 3 Variables & Intro to Matrices

When you go from 2 to 3 variables, the geometric picture changes from "lines in a plane" to "planes in space." Three planes generally meet at a single point, but they can also miss each other entirely or share a line.

Solving a 3-variable system by elimination

Strategy: eliminate one variable using two of the equations, then eliminate the same variable using a different pair. You're left with a 2-variable system, which you already know how to solve.

Example:

Add (1) and (2): $2x + 2z = 8$. Add (1) and (3): $2x + 2y = 6$. Subtract (2) from (1): $2y = 4 \Rightarrow y = 2$. Substitute back: $x = 1$, $z = 3$.

The pattern is mechanical, but the bookkeeping gets tedious fast. Matrices give us a cleaner way to track it.

Matrices: the bookkeeping device

A matrix is a rectangular grid of numbers. The system above can be written as:

| 1  1  1 | | x |   | 6 |
| 1 −1  1 | | y | = | 2 |
| 1  1 −1 | | z |   | 0 |

This is $A\mathbf{x} = \mathbf{b}$. The system's solution is $\mathbf{x} = A^{-1}\mathbf{b}$ when $A$ is invertible.

Gaussian elimination

The algorithm that powers every linear-algebra library:

1. Write the augmented matrix $[A \mid \mathbf{b}]$.
2. Use row operations to convert $A$ into upper-triangular form (zeros below the diagonal).
3. Back-substitute to read off the solution.

Allowed row operations:

• Swap two rows.
• Scale a row by a nonzero constant.
• Add a multiple of one row to another row.

These operations preserve the solution set.

Worked example: Gaussian elimination on the same system

Start from the augmented matrix:

| 1   1   1 | 6 |
| 1  −1   1 | 2 |
| 1   1  −1 | 0 |

R2 ← R2 − R1 (zero out the first entry of row 2):

| 1   1   1 |  6 |
| 0  −2   0 | −4 |
| 1   1  −1 |  0 |

R3 ← R3 − R1 (zero out the first entry of row 3):

| 1   1   1 |  6 |
| 0  −2   0 | −4 |
| 0   0  −2 | −6 |

The matrix is now upper-triangular. Back-substitute from the bottom:

• Row 3: −2z = −6 → z = 3.
• Row 2: −2y = −4 → y = 2.
• Row 1: x + y + z = 6 → x + 2 + 3 = 6 → x = 1.

Solution (x, y, z) = (1, 2, 3), matching the elimination pass above.

The identity matrix

$I$ has 1s on the diagonal and 0s elsewhere. It plays the same role for matrix multiplication that $1$ plays for ordinary multiplication: $A \cdot I = A$ for any matrix $A$. Multiplying a matrix by its inverse gives $I$.

Why ML uses matrices, not loops

Every neural network layer is $Z = WX + b$: matrix multiplication plus a bias. A numerical computing library's matrix-multiply call runs in optimized C code (BLAS / LAPACK). The same operation written as an interpreted for loop over rows would be orders of magnitude slower. Matrix notation isn't just compact; it unlocks vectorized hardware acceleration.