Fourier Series as Basis Functions

Take AI Test

by Marcus Math

🔍

## Fourier Series as Basis Functions (Vectors) in Function Space

The Fourier Series can be used as a set of Basis Functions to represent any function. Given some well behaved function $f(x)$ with $x \in [-\pi,\pi]$, we can represent it as a sum of cosines and sines of varying frequencies, specifically:

$$ f(x) = \frac{a_0}{2} + \sum_{n=1}^\infty \left[ a_n \cos(nx) + b_n \sin(nx) \right] $$

From the formula, we can see that $n$ is an integer $n \in \mathbb{Z}$ and $n \geq 1$ in the sum, meaning that we have countably infinite sines and cosines we can add to reconstruct a function $f(x)$. Additionally, note that the frequencies of cosine and sine are integer frequencies and not decimal.

It's worth being precise about where the series *really* starts. Conceptually the Fourier basis starts at $n = 0$. The leading term $\frac{a_0}{2}$ is exactly the frequency zero cosine, $\cos(0x) = 1$ (and as we'll see, it is simply the average value of $f$). The reason we pull it out front and start the sum at $n = 1$ is that the $n = 0$ sine term contributes nothing, since $\sin(0x) = \sin(0) = 0$ is the zero function. The zero function can never belong to a linearly independent set (it is a nontrivial way of writing the zero vector), so $\sin(0x)$ cannot be a basis function. That is why only $\cos(0x)$ survives at $n = 0$, giving us the single constant term and a sum that runs over $n \geq 1$ for both sines and cosines.

### What does "well behaved" mean?

It has been stated that the Fourier Series only works on "well behaved" functions, but what does this mean? We are going to show that this is for the Hilbert Space $L^2[-\pi,\pi]$, which is the space of all measurable functions $f: [-\pi,\pi] \rightarrow \mathbb{C}$ that are square integrable, i.e.

$$ \int_{-\pi}^\pi |f(x)|^2 dx < \infty $$

This vector space has some very nice properties that we can work with. Firstly, it's a vector space over the functions, and in particular it has closure under addition: if $f$ and $g$ are square integrable, then so is $f + g$. Let's see why, because it's a nice little argument. The thing we know is finite is $\int|f|^2$ and $\int|g|^2$ separately, so the goal is to bound $|f+g|^2$ using only $|f|^2$ and $|g|^2$. Start from the (pointwise) triangle inequality,

$$ |f(x) + g(x)| \leq |f(x)| + |g(x)|, $$

and square both sides:

$$ |f+g|^2 \leq \big(|f| + |g|\big)^2 = |f|^2 + 2|f||g| + |g|^2. $$

The only awkward piece is the cross term $2|f||g|$, which tangles $f$ and $g$ together. We can split it apart with $2ab \leq a^2 + b^2$ (just a rearrangement of $(a-b)^2 \geq 0$), which gives $2|f||g| \leq |f|^2 + |g|^2$. Substituting that back leaves us with the clean bound

$$ |f+g|^2 \leq 2|f|^2 + 2|g|^2. $$

Now integrate both sides over $[-\pi,\pi]$:

$$ \int_{-\pi}^{\pi} |f+g|^2\, dx \leq 2\int_{-\pi}^{\pi} |f|^2\, dx + 2\int_{-\pi}^{\pi} |g|^2\, dx < \infty, $$

so $f + g$ is square integrable whenever $f$ and $g$ are, and closure holds. (Sharpening this same idea from the loose constant $2$ down to its tightest form gives Minkowski's inequality $||f+g||_2 \leq ||f||_2 + ||g||_2$, which is exactly the triangle inequality for the $L^2$ norm.) Furthermore, we have that the inner product of this space is defined as

$$ \langle f,g \rangle = \int_{-\pi}^\pi f(x)\,\overline{g(x)}\, dx $$

with a norm then defined by

$$ ||f||_2 = \left( \int_{-\pi}^\pi |f(x)|^2 dx \right)^{\frac{1}{2}} $$

The bar denotes complex conjugation. All of the basis functions we use below ($\cos(nx)$, $\sin(nx)$, and the constant) are real valued, so for our purposes the conjugate does nothing and the inner product reduces to $\int_{-\pi}^\pi f(x) g(x)\, dx$.

What this means intuitively is that within the $L^2[-\pi, \pi]$ space we can treat functions as vectors, and can find the distance between two functions along with all properties we inherit from inner product spaces. See [Metric Space vs Inner Product Space](https://finetuneinterview.com/lessons/metric-space-vs-inner-product-space) for more details on Inner Product Spaces.

Additionally, the $L^2[-\pi,\pi]$ space is complete, meaning that every Cauchy Sequence of its elements (functions) converges to an element (function) within the space. Since this is a complete inner product space, this is a **Hilbert Space**, which has very nice properties similar to spaces like $\mathbb{R}^n$.

Completeness is exactly what lets the infinite Fourier sum make sense. The partial sums of the Fourier series of an $f \in L^2[-\pi,\pi]$ form a Cauchy sequence in the $L^2$ norm, so completeness guarantees they converge to some element of $L^2[-\pi,\pi]$. (That the limit is in fact $f$ itself is the spanning/completeness statement we prove at the end.)

### Fourier Series as Basis Functions in $L^2[-\pi, \pi]$

Similar to how the vectors $[1,0,...0]^T$, $[0,1,...0]^T$,... are the standard basis in $\mathbb{R}^n$, we have that the Fourier functions are similarly **basis functions** in $L^2[-\pi, \pi]$. We will assume here that this statement is true, and then prove it in the next subsection.

One important caveat before the analogy. In an infinite dimensional Hilbert space, "basis" does **not** mean a Hamel basis, where every element is a *finite* combination of basis vectors. It means a **complete orthonormal system** (an orthonormal Schauder basis): the basis functions are pairwise orthogonal, each has norm one, and their *closed* span is all of $L^2[-\pi,\pi]$. That is the sense in which we'll show the Fourier system is a basis. If you'd like us to go deeper into Hilbert spaces and the general theory of orthonormal bases, leave a suggestion in Lessons.

To see the power of the Fourier Series to approximate functions, see the PyApplet below.

Now, here is the subtle point that makes the $\mathbb{R}^n$ analogy work. In $\mathbb{R}^n$ the standard basis vectors are **orthonormal** (they are orthogonal *and* each has length one), which is exactly why "the coordinate equals the inner product." Our raw trig functions are orthogonal but **not** normalized:

$$ ||1||_2^2 = \int_{-\pi}^\pi 1\, dx = 2\pi, \qquad ||\cos(nx)||_2^2 = ||\sin(nx)||_2^2 = \int_{-\pi}^\pi \cos^2(nx)\, dx = \pi \quad (n \geq 1) $$

So to make the analogy exact, we normalize. Define the orthonormal system

$$ \phi_0(x) = \frac{1}{\sqrt{2\pi}}, \qquad \phi_n^c(x) = \frac{1}{\sqrt{\pi}}\cos(nx), \qquad \phi_n^s(x) = \frac{1}{\sqrt{\pi}}\sin(nx) \quad (n \geq 1) $$

These each have norm one, and (as we prove below) they are pairwise orthogonal. With an orthonormal basis, reconstructing a vector is exactly the projection formula. Take for example $\mathbb{R}^n$ with the standard basis

$$
\mathbf{e}_1 = \begin{pmatrix} 1 \\ 0 \\ \vdots \\ 0 \end{pmatrix}, \quad
\mathbf{e}_2 = \begin{pmatrix} 0 \\ 1 \\ \vdots \\ 0 \end{pmatrix}, \quad
\dots, \quad
\mathbf{e}_n = \begin{pmatrix} 0 \\ 0 \\ \vdots \\ 1 \end{pmatrix}.
$$

If we wanted to represent some vector $v \in \mathbb{R}^n$ by this standard basis, we typically do it in the following way:

$$ v = \sum_{i=1}^n \langle v,e_i \rangle e_i $$

Which for $\mathbb{R}^3$ and the concrete vector $v = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}$ becomes:

$$
v = \langle v, \mathbf{e}_1 \rangle \mathbf{e}_1
  + \langle v, \mathbf{e}_2 \rangle \mathbf{e}_2
  + \langle v, \mathbf{e}_3 \rangle \mathbf{e}_3
$$
$$
v = 1 \cdot \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}
  + 2 \cdot \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix}
  + 3 \cdot \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}
$$

For our (orthonormal) Fourier basis the identical projection statement holds:

$$ f(x) = \langle f, \phi_0 \rangle \phi_0 + \sum_{n=1}^\infty \left[ \langle f, \phi_n^c \rangle \phi_n^c + \langle f, \phi_n^s \rangle \phi_n^s \right] $$

This is the clean analogue of $v = \sum \langle v, e_i\rangle e_i$: each coordinate is genuinely the inner product of $f$ onto a unit norm basis function.

If we now expand the normalizations and collect constants, we recover the classical Fourier form stated at the top:

$$ f(x) = \frac{a_0}{2} + \sum_{n=1}^\infty \left[ a_n \cos(nx) + b_n \sin(nx) \right] $$

where the coefficients carry the $\tfrac{1}{\pi}$ factor that comes from removing the normalization:

$$ a_n = \frac{1}{\pi}\langle f, \cos(nx) \rangle = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x) \cos(nx)\, dx \quad (n \geq 0) $$
$$ b_n = \frac{1}{\pi}\langle f, \sin(nx) \rangle = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x) \sin(nx)\, dx \quad (n \geq 1) $$

In particular $\frac{a_0}{2} = \frac{1}{2\pi}\int_{-\pi}^\pi f(x)\, dx$, which is exactly the average value of $f$ over $[-\pi,\pi]$ (the constant term is the DC component). The key correction to keep in mind is that the coefficient is the inner product *onto the normalized basis function*, or equivalently the raw inner product divided by $\pi$ (or $2\pi$ for the constant term). Forgetting the $\tfrac1\pi$ is the most common slip here.

### Proof showing Fourier Series are Basis Functions in $L^2[-\pi, \pi]$

To prove that a set of vectors in a Hilbert space is an (orthonormal) basis we need to show two things:
1. The set is orthogonal, and since each vector is nonzero, this gives Linear Independence for free.
2. The set is Spanning, meaning its *closed* span is the whole space (this is called **completeness**).

A quick word on why orthogonality buys us linear independence. If $\{v_i\}$ is orthogonal with each $v_i \neq 0$ and $\sum_i c_i v_i = 0$, then taking the inner product of both sides with a fixed $v_j$ gives $c_j ||v_j||^2 = 0$, hence $c_j = 0$. Every coefficient must vanish, so the set is linearly independent. This is why the entire linear independence argument below reduces to *computing inner products and showing they're zero*.

#### Orthogonality (Linear Independence) of the Fourier System

We will do a Proof By Induction on the highest frequency $N$ included. Let

$$ B_N = \{\,1\,\} \cup \{\, \cos(nx),\, \sin(nx) \;:\; 1 \leq n \leq N \,\} $$

and we will show every $B_N$ is an orthogonal set. Since orthogonality is a property of pairs, if every finite $B_N$ is orthogonal then the whole infinite system is orthogonal. To show orthogonality we show the relevant inner products are 0:

$$ \langle f,g \rangle = \int_{-\pi}^{\pi} f(x)g(x)\,dx = 0 $$

To help us, we use the following inner products (integrals) without proof (proof left to the reader):

$$
Eq.\ 1\quad \langle \cos(kx), \cos(mx) \rangle = \int_{-\pi}^{\pi} \cos(kx) \cos(mx) \, dx =
\begin{cases}
2\pi & k = m = 0 \\
\pi & k = m \neq 0 \\
0 & \text{otherwise}
\end{cases}
$$

$$
Eq.\ 2\quad \langle \sin(kx),\cos(mx) \rangle = \int_{-\pi}^{\pi} \sin(kx) \cos(mx) \, dx = 0 \quad \forall\, k,m
$$

$$
Eq.\ 3\quad \langle \sin(kx), \sin(mx) \rangle = \int_{-\pi}^{\pi} \sin(kx) \sin(mx) \, dx =
\begin{cases}
\pi & k = m \neq 0 \\
0 & \text{otherwise}
\end{cases}
$$

$$
Eq.\ 4\quad \int_{-\pi}^{\pi} \sin(mx)\,dx = 0 \qquad\qquad Eq.\ 5\quad \int_{-\pi}^{\pi} \cos(mx)\,dx = 0 \quad (m \neq 0)
$$

(Note that $\sin(0x) \equiv 0$ is the zero function, so it is **not** a basis element. The sine functions only start at frequency $n=1$, and the only frequency zero basis function is the constant $\cos(0x) = 1$.)

**Base Case**

The smallest system is $B_0 = \{1\}$, a single nonzero function. A set containing a single nonzero vector is trivially orthogonal (there are no distinct pairs to check) and therefore linearly independent. This is our (corrected) base case.

**Inductive Step**

Assume the set $B_n$ is orthogonal. To extend to $B_{n+1}$ we add two new functions and must check each is orthogonal to everything already in $B_n$ (and to each other):
1. Add $\cos((n+1)x)$
2. Add $\sin((n+1)x)$

#### Case 1: $\cos((n+1)x)$

We take the inner product of $\cos((n+1)x)$ with every element of $B_n$. The elements of $B_n$ are $\cos(kx)$ for $0 \le k \le n$ (with $k=0$ giving the constant $1$) and $\sin(kx)$ for $1 \le k \le n$.

Against the cosines:
$$ \langle \cos(kx), \cos((n+1)x) \rangle = \int_{-\pi}^{\pi} \cos(kx)\cos((n+1)x)\, dx $$
Using Eq. 1, since $k \neq n+1$ for every $k \in [0,n]$, this inner product is 0. (This includes $k = 0$: $\langle 1, \cos((n+1)x)\rangle = \int_{-\pi}^\pi \cos((n+1)x)\,dx = 0$ by Eq. 5, since $n+1 \neq 0$.)
$$ \langle \cos(kx), \cos((n+1)x) \rangle = 0 \quad \forall\, k \in [0,n] $$

Against the sines:
$$ \langle \sin(kx), \cos((n+1)x) \rangle = \int_{-\pi}^{\pi} \sin(kx)\cos((n+1)x)\, dx = 0 \quad \forall\, k \in [1,n] $$
which is immediate from Eq. 2.

So every inner product of $\cos((n+1)x)$ against $B_n$ is 0, showing $\cos((n+1)x)$ is orthogonal to the previous set.

#### Case 2: $\sin((n+1)x)$

Same technique. Against the cosines (including the constant):
$$ \langle \cos(kx), \sin((n+1)x) \rangle = \int_{-\pi}^{\pi} \cos(kx)\sin((n+1)x)\, dx = 0 \quad \forall\, k \in [0,n] $$
which is again immediate from Eq. 2.

Against the sines:
$$ \langle \sin(kx), \sin((n+1)x) \rangle = \int_{-\pi}^{\pi} \sin(kx)\sin((n+1)x)\, dx $$
Using a product to sum identity, restate as
$$ \int_{-\pi}^{\pi} \sin(kx)\sin((n+1)x)\, dx = \frac{1}{2}\int_{-\pi}^{\pi} \cos\big((k-(n{+}1))x\big)\,dx - \frac{1}{2}\int_{-\pi}^{\pi} \cos\big((k+(n{+}1))x\big)\,dx $$
Since $1 \le k \le n$, both $k - (n{+}1) \neq 0$ and $k + (n{+}1) \neq 0$, so by Eq. 5 each integral is 0. (Equivalently, this is just Eq. 3 with $k \neq n+1$.) Thus
$$ \langle \sin(kx), \sin((n+1)x) \rangle = 0 \quad \forall\, k \in [1,n] $$

Finally, the two new functions are orthogonal to each other:
$$ \langle \cos((n+1)x), \sin((n+1)x) \rangle = 0 $$
by Eq. 2.

Having verified both cases, $B_{n+1}$ is orthogonal. By induction every $B_N$ is orthogonal, so the full infinite system

$$ \{\,1\,\} \cup \{\, \cos(nx),\, \sin(nx) \,\}_{n \geq 1} $$

is orthogonal. Since each of these functions is nonzero, the orthogonality lemma above tells us the system is **linearly independent**.

#### Spanning (Completeness) of the Fourier System — via Fejér's Theorem

Orthogonality alone is not enough: we also need the system to be **complete**, meaning its closed span is *all* of $L^2[-\pi,\pi]$. We prove this by the contrapositive:

*If every Fourier coefficient of $f$ is zero, then $f$ must be the zero function.*

i.e. if $\langle f, \cos(nx) \rangle = 0$ for all $n \geq 0$ and $\langle f, \sin(nx) \rangle = 0$ for all $n \geq 1$, then $f = 0$. This is exactly the statement that no nonzero $f$ can sit orthogonal to the whole system, so the system's closed span leaves nothing out (it spans).

We'll use **Fejér's theorem**, which gives a cleaner and more constructive route than the Stone Weierstrass approach, because it converges directly in $L^2$ and so handles general (not necessarily continuous) $L^2$ functions without an extra density step.

**Proof**

Assume $\langle f, \cos(nx) \rangle = 0$ for all integers $n \geq 0$ and $\langle f, \sin(nx) \rangle = 0$ for all integers $n \geq 1$. Then every coefficient $a_n, b_n$ of $f$ is zero, so every partial sum of the Fourier series vanishes:

$$ S_N(f)(x) = \frac{a_0}{2} + \sum_{n=1}^{N}\big[ a_n \cos(nx) + b_n \sin(nx) \big] = 0 \quad \text{for every } N. $$

Now form the **Fejér means**, the Cesàro (running) averages of the partial sums:

$$ \sigma_N(f) = \frac{1}{N+1}\sum_{k=0}^{N} S_k(f). $$

Since each $S_k(f) = 0$, we have $\sigma_N(f) = 0$ for every $N$.

The reason we average the partial sums rather than use them raw is that the Fejér means are a convolution with the **Fejér kernel** $F_N$ (see **Appendix**), which is a *nonnegative approximate identity*. This is precisely what makes them well behaved where the raw partial sums need not be. Fejér's theorem states that for any $f \in L^2[-\pi,\pi]$ (extended $2\pi$ periodically), the Fejér means converge to $f$ in the $L^2$ norm:

$$ ||\sigma_N(f) - f||_2 \longrightarrow 0 \quad \text{as } N \to \infty. $$

But $\sigma_N(f) = 0$ for all $N$, so $||\sigma_N(f) - f||_2 = ||f||_2$ is a fixed number independent of $N$. The only way a constant can tend to $0$ is to be $0$, hence

$$ ||f||_2 = 0 \quad \rightarrow \quad f = 0 \text{ almost everywhere in } [-\pi,\pi]. $$

So the only function orthogonal to every Fourier basis function is the zero function. The Fourier system is therefore complete, so it spans $L^2[-\pi,\pi]$.

Since we showed that the Fourier system is both orthogonal (hence linearly independent) and spanning, we have shown that it is a **basis** in $L^2[-\pi,\pi]$. And after the normalization from earlier, it is in fact an **orthonormal basis**, which is exactly what makes the projection formula $f = \sum \langle f, \phi \rangle \phi$ hold.

---
## Appendix
### The Fejér Kernel

The Fejér means can be written as a convolution $\sigma_N(f) = f * F_N$, where the Fejér kernel is

$$ F_N(t) = \frac{1}{N+1}\left( \frac{\sin\!\big(\tfrac{(N+1)t}{2}\big)}{\sin\!\big(\tfrac{t}{2}\big)} \right)^{\!2}. $$

It has three properties that together make it an **approximate identity**, and these are what drive the convergence used above:

1. **Nonnegativity:** $F_N(t) \geq 0$ for all $t$ (unlike the Dirichlet kernel of the raw partial sums, which changes sign).
2. **Unit mass:** $\dfrac{1}{2\pi}\displaystyle\int_{-\pi}^{\pi} F_N(t)\, dt = 1$.
3. **Concentration:** for any fixed $\delta > 0$, the mass of $F_N$ outside $(-\delta, \delta)$ tends to $0$ as $N \to \infty$; the kernel piles up at the origin.

A standard fact in analysis is that convolving against an approximate identity converges to the original function in $L^p$ norm for $1 \le p < \infty$. Applying this with $p = 2$ gives $||\sigma_N(f) - f||_2 \to 0$ for every $f \in L^2[-\pi,\pi]$, which is the form of Fejér's theorem we used. (If you'd like a full treatment of approximate identities and convergence in Hilbert spaces, leave a suggestion in Lessons.)