Metric Space vs Inner Product Space Take AI Test by Marcus Math 🔍 Many ideas in math begin with geometric intuition, points can have distance, vectors can have length, and two directions can meet at an angle. In common Euclidean space, these ideas feel strongly interconnected. The distance between two points, the length of a vector, and the angle between two vectors are connected from the same familiar geometry. However, as we want to represent a richer and more diverse set of problems these concepts separate. We may want to talk about distance of elements without having notion of length or angles. Perhaps, we want a structure rich enough to measure both length and direction. This is where the metric space and inner product space concepts enter. A **metric space** is constructed around the core idea of distance. This construction enables us to measure the distance between two elements in a set, sequence convergence, and a formal definition for continuity. An **inner product space** begins with a way to multiply vectors (the fundamental unit from the inherited *vector space* this is defined in) producing geometric information including - lengths - angles - orthogonality The key relation between these two mathematical objects is that *Every inner product space gives rise to a metric **however** not every metric space comes from an inner product.* In this blog we are going to compare these two spaces and define key properties of each. ## Metric Spaces A metric space is a set of elements together with a metric function that allows us to measure the distance between any two elements. The formal definition is the following: Let $X$ be a set, and let $d: X \times X \rightarrow \mathbb{R}$ be the distance/metric function such that $\forall x,y,z \in X$ we have the properties: 1. Non-negative Distance: $d(x,y) \geq 0$ 2. Identity of Indiscernibles: $d(x,y) = 0$ if and only if $x=y$ - A point has distance 0 from itself - Distinct points have positive distance from each other 3. Symmetry: $d(x,y) = d(y,x)$ - The distance between two elements doesn't change with respect to the ordering of the input arguments 4. Triangle Inequality: $d(x,z) \leq d(x,y) + d(y,z)$ The **Metric Space** is defined as the pair $(X,d)$ such that $X$ is a set of elements and for all elements $x,y,z \in X$ satisfy the four metric axioms. ### Examples Let's give some examples of what a metric space is to motivate and see how this construction is useful! #### Metric Space 1: $d(x,y) := |x-y|$ on the reals $\mathbb{R}$ For this example, we define the metric space such that the pair $(X,d)$ is defined as: $$ (X,d) = (\mathbb{R}, d(x,y) := |x-y|\ \forall x,y \in \mathbb{R})$$ Let's show how this pair satisfies the conditions needed for a metric space. 1. We can see that $|x-y|$ is always greater than or equal to zero. Simply imagine (or plot) the absolute value function $f(z) = |z|$ and take $z = x-y$. 2. If $x=y$, we get $|x-x| = 0$. The vice versa can be seen easily. 3. We can see that $|x-y| = |y-x|$ since we're taking the absolute value of the difference between them so if $z = x-y$ we get that $|z| = |-z|$. 4. Finally, we get to have some fun! The triangle inequality for this: $$ d(x,z) \leq d(x,y) + d(y,z)$$ is the following: $$ |x-z| \leq |x-y| + |y-z|$$ Let $ a = x - y $ and $ b = y - z $. Then $ x - z = a + b $, and we need to show $ |a + b| \leq |a| + |b| $. Consider two cases: **Case 1**: $ a + b \geq 0 $. Then $ |a + b| = a + b $. Since $ a \leq |a| $ and $ b \leq |b| $, we have $ a + b \leq |a| + |b|. $ **Case 2**: $ a + b < 0 $. Then $ |a + b| = -(a + b) = (-a) + (-b) $. Since $ -a \leq |a| $ and $ -b \leq |b| $, we again get $ (-a) + (-b) \leq |a| + |b|. $ In both cases, $ |a + b| \leq |a| + |b| $, which completes the proof. Note, this may look extremely familiar if you read my blogs on the [L1 Regularizer](https://www.noteblogdoku.com/blog/l1-lasso-regularization-sparsity) where we regularize a Loss Function such that: $$ L(w) = || y + Xw||^2 + \lambda \sum |w_i| $$ We can see that the distance function $d(x,y) = |x-y|$ is the distance function induced by the L1 Norm in one dimension (real line). It is extremely satisfying to see how concepts connect together in mathematics! #### Metric Space 2: The discrete (trivial) metric on set $X$ The discrete metric on set $X$ is defined as $$d(x, y) = \begin{cases} 0 & \text{if } x = y \\ 1 & \text{if } x \neq y \end{cases}$$ Where basically the distance between any two elements is 1 and the distance between an element and itself is 0. <img src="/media/images/20260531_230722_Screenshot_2026-05-31_at_4.06.18_PM.png" alt="Screenshot 2026 05 31 at 4.06.18 PM" style="max-width: min(400px,80%); height: auto;"> <center> $\textbf{Figure 1}$: Discrete Metric over the set $\{A,B,C,D\} = X$ </center> We can see that this easily satisfies the first three properties for a distance function for a metric space. For the triangle inequality, we get that the only possible values are 0 and 1. The worst case is $1 \leq 1+1$ and other cases are trivial from this one. This distance function is really neat since it works on **any** set $X$. #### Metric Space 3: Continuous Functions on the interval $[a,b]$, i.e. $C([a,b],\mathbb{R})$ with Supremum Metric Define the metric on $C([a,b],\mathbb{R})$ (real continuous functions on interval $a,b$ inclusive) to be: $$ d(f,g) := sup_{x \in [a,b]} |f(x) - g(x)| $$ Note that $d(f,g)$ is finite as $|f(x) - g(x)|$ is a continuous function on a closed bounded interval $[a,b]$ bounding the metric. The first three requirements for a metric space are trivial to see from this definition. For the triangle inequality we have: $$ d(f,g) = sup_{x \in [a,b]} |f(x) - g(x)| $$ $$ d(f,g) = sup_{x \in [a,b]} |f(x) - h(x) + h(x) - g(x)| $$ $$ d(f,g) \leq sup_{x \in [a,b]} (|f(x) - h(x)| + |h(x) - g(x)|) $$ $$ d(f,g) \leq sup_{x \in [a,b]} (|f(x) - h(x)|) + sup_{x \in [a,b]} (|h(x) - g(x)|)$$ $$ sup_{x \in [a,b]} (|f(x) - h(x)|) + sup_{x \in [a,b]} (|h(x) - g(x)|) = d(f,h) + d(h,g)$$ Thus we get $$ d(f,g) \leq d(f,h) + d(h,g) $$ <img src="/media/images/20260601_003641_Screenshot_2026-05-31_at_5.35.28_PM.png" alt="Screenshot 2026 05 31 at 5.35.28 PM" style="max-width: min(650px,80%); height: auto;"> <center> $\textbf{Figure 2}$: Metric Space for the Set $C([a,b],\mathbb{R})$ </center> This metric space is particularly important because it naturally corresponds to uniform convergence of functions. It serves as a foundational setting for many areas of mathematics, including: - **Fourier Series**, where one studies the uniform approximation of continuous functions by trigonometric polynomials; - **Calculus of Variations**, where functionals (such as energy or length) are defined and optimized over spaces of continuous functions; - **Ordinary Differential Equations**, especially through the Picard–Lindelöf theorem and contraction mapping arguments; - **Approximation Theory**, via results like the Stone–Weierstrass and Weierstrass Approximation Theorems. Additionally, its completeness makes it one of the most important examples of an infinite-dimensional Banach space, playing a central role in functional analysis and the rigorous study of convergence, continuity, and operators on function spaces. ## Inner Product Spaces An Inner Product Space is a vector space where you can multiply two vectors to get a scalar. It consists of a vector space $V$ with an inner product $\langle x,y \rangle$ such that $x,y \in V$. First we need to state the formal definition of a vector space so we can define an inner product space. The formal definition of a **Vector Space** is the following: --- Let $ \mathbb{F} $ be a field (most commonly $ \mathbb{F} = \mathbb{R} $ or $ \mathbb{F} = \mathbb{C} $). A vector space over $ \mathbb{F} $ is a set $ V $ equipped with two operations: Vector addition: $ +: V \times V \to V $ Scalar multiplication: $ \cdot: \mathbb{F} \times V \to V $ such that the following 10 axioms hold for all vectors $ \mathbf{u}, \mathbf{v}, \mathbf{w} \in V $ and all scalars $ \alpha, \beta \in \mathbb{F} $: Axioms for Vector Addition Associativity: $ (\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w}) $ Commutativity: $ \mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u} $ Identity element: There exists a zero vector $ \mathbf{0} \in V $ such that $ \mathbf{u} + \mathbf{0} = \mathbf{u} $ Inverse element: For every $ \mathbf{u} \in V $, there exists $ -\mathbf{u} \in V $ such that $ \mathbf{u} + (-\mathbf{u}) = \mathbf{0} $ Axioms for Scalar Multiplication Distributivity (scalar over vectors): $ \alpha(\mathbf{u} + \mathbf{v}) = \alpha\mathbf{u} + \alpha\mathbf{v} $ Distributivity (vectors over scalars): $ (\alpha + \beta)\mathbf{u} = \alpha\mathbf{u} + \beta\mathbf{u} $ Compatibility: $ \alpha(\beta\mathbf{u}) = (\alpha\beta)\mathbf{u} $ Scalar identity: $1 \cdot \mathbf{u} = \mathbf{u}$ Additional Axioms Closure under addition: $ \mathbf{u} + \mathbf{v} \in V $ Closure under scalar multiplication: $ \alpha\mathbf{u} \in V $ --- Fantastic! Now we can give the formal definition of an inner product space, which builds off vector spaces. The formal definition of an **Inner Product Space** is the following: --- The Inner Product on a vector space $V$ (over $\mathbb{R}$ or $\mathbb{C}$ is a function $\langle \cdot, \cdot \rangle : V \times V \rightarrow \mathbb{K}$ ($\mathbb{K}$ scalar field, usually just $\mathbb{R}, \mathbb{C}$ satisfying 1. Positive Definiteness; $\langle x,x \rangle \geq 0$ & $\langle x,x \rangle = 0 \leftrightarrow x = 0$ 2. Linearity of the First Argument: $\langle ax + by, z \rangle = a \langle x,z \rangle + b \langle y,z \rangle$ 3. Conjugate Symmetry: $\langle x,y \rangle = \bar{\langle y,x \rangle}$ Note: we also have conjugate-linearity in the second argument: $\langle x, ay + bz \rangle = \bar{a}\langle x,y \rangle + \bar{b}\langle x,z \rangle $ Of important note, **if you have an inner product you can always construct a metric from it:** $$ d(x,y) := \sqrt{\langle x-y, x-y \rangle}$$ We can see that $d$ satisfies all metric axioms, with the Triangle Inequality following from the Cauchy-Schwartz Inequality (see **Appendix**). Because of this special property, we can also state that **Inner Product Spaces are special Metric Spaces with Vector-Space geometry**. --- Of special note, Inner Products give us lengths, angles, orthogonality, projections, and distances. - Length (norm) is given by $||u|| = \sqrt{\langle u,u \rangle}$ - Distance is given by $d(u,v) = ||u-v|| = \sqrt{\langle u-v,u-v \rangle}$ - Angle $ cos\ \theta = \frac{\langle u,v \rangle}{||u||||v||}$ - Orthogonality $u \perp v \leftrightarrow \langle u,v \rangle = 0$ - Projection, orthogonal projection of $u$ onto $v$: $proj_v u = \frac{\langle u,v \rangle}{\langle v,v \rangle} v$ The inner product enables us to use these properties in abstract vector spaces. Even in infinite dimensional spaces like $C([a,b], \mathbb{R})$ we can define an inner product such as $\langle f,g \rangle = \int_a^b f(x)g(x) dx$. This is really cool since we can abstract the notion of vector beyond a typical space (like $\mathbb{R}^3$) to things like functions, and can even describe what the length of a function is! The standard $ L^2 $ inner product $ \langle f, g \rangle = \int_a^b f(x)g(x)\, dx $ turns $ C([a,b]) $ into a (pre-)Hilbert space. This allows us to study orthogonal polynomials, Fourier series, etc. ### Examples Let's give some examples of what an inner product space is to motivate and see how this construction is useful! #### Inner Product Space 1: $\mathbb{R}^3$ This is a classic example, the $\mathbb{R}^3$ is a ubiquitous vector space. The inner product is defined as the following for $x,y \in \mathbb{R}^3$ $$ \langle x,y \rangle = \sum_i^3 x_i y_i $$ Since this is such a well known example, I will not prove all three properties. #### Inner Product Space 2: The Space of Continuous Functions with the $L^2$ Inner Product Let $C([a,b])$ be the vector space of all continuous real-valued functions on the closed interval $[a,b]$. We define the inner product on this space as $$ \langle f,g \rangle = \int_a^b f(x)g(x) dx $$ Turning $C([a,b])$ into an inner product space. This inner product space is very useful and has the following geometric concepts we can apply to continuous functions: - Length/Norm $||f||_2 = \sqrt{\langle f,f \rangle} = \sqrt{\int_a^b |f(x)|^2 dx}$ - Distance $d(f,g) = \sqrt{\int_a^b [f(x)-g(x)]^2 \, dx} $ - Orthogonality $ f \perp g \iff \langle f,g \rangle = 0 $ - Angle $ \cos \theta = \frac{\langle f,g \rangle}{\|f\|_2 \|g\|_2} $ - How similar are two functions? Now we have a way to measure! - Projection $ \text{proj}_g f = \frac{\langle f,g \rangle}{\langle g,g \rangle} g $ - Best approximation of one function by another <img src="/media/images/20260601_012129_Screenshot_2026-05-31_at_6.17.10_PM.png" alt="Screenshot 2026 05 31 at 6.17.10 PM" style="max-width: min(650px,80%); height: auto;"> <center> $\textbf{Figure 3}$: $L^2$ Inner Product on $C([a,b])$ enabling us to use Functions as Vectors </center> This example is very important since it opens up a whole new way of using functions and thinking of them as infinite dimensional vectors. 1. Hilbert Spaces When we complete $C([a,b])$ with respect to this norm, we get the famous $L^2[a,b]$ space, one of the most important Hilbert Spaces (see **Appendix**) in mathematics! 2. Fourier Analysis Using this inner product, we can build orthogonal bases and expand functions as infinite series. These bases include - Sines & Cosines - Legendre polynomials - Etc. 3. Applications include - Signal processing - Quantum mechanics (wave functions) - Partial Differential Equations (heat equation, wave equation) 4. Bridges Discrete & Continuous Generalizes the dot product from $\mathbb{R}^n$ to functions. We can see how $ \langle f,g \rangle = \int_a^b f(x)g(x) dx $ is really the infinite dimensional version of the dot product for finite vectors $ \langle u,v \rangle = \sum_i^n u_i v_i$ An example of two orthogonal functions is $sin(x)$ and $cos(x)$ (thanks Fourier!). For the interval $[-\pi, \pi]$, we can see $$ \langle sin(x), cos(x) \rangle = \int_{-\pi}^{\pi} sin(x)cos(x) dx = 0$$ Which is a fundamental reason why we can use Fourier Series as basis functions (linear independence). ## Inner Product Space vs Metric Space | Feature | Metric Space | Inner Product Space | |----------------------------------|-------------------------------|----------------------------------------------| | **Basic object** | Set of points | Vector space | | **Main structure** | Distance function $(d(x,y))$ | Inner product $\langle x, y \rangle$ | | Can measure distance? | Yes | Yes, indirectly | | Can measure length? | Not necessarily | Yes | | Can measure angle? | No, not in general | Yes | | Can talk about orthogonality? | No, not in general | Yes | | Needs vector addition? | No | Yes | | Needs scalar multiplication? | No | Yes | | **More general?** | Yes | No, more specialized | The key difference between **Metric Spaces** and **Inner Product Spaces** is - Metric Spaces: Space where we can measure distances between every pair of two elements $d(x,y) \forall x,y \in X$ for metric space $(X,d)$ - Inner Product Spaces: Space where we inherit all properties of vector spaces & can compare vectors to get length, distance, orthogonality, angles, and projections. Furthermore, **Inner Product Spaces are special Metric Spaces with Vector-Space geometry**. --- ## Appendix ### What does $d: X \times X \rightarrow \mathbb{R}$ mean? This is standard mathematical notation that means $d$ is a function that takes two elements form the set $X$ where $x,y \in X$ and outputs a real number. $d$ the name of the function, $X \times X$ is the domain, which is the Cartesian product of the set $X$ with itself. This is the set of all possible ordered pairs $(x,y)$ where $x,y \in X$. $\mathbb{R}$ is the codomain which is the set of real numbers. So the output of $d(x,y)$ is always a real number. ### Triangle Inequality showing Metrics can always be constructed from Inner Products We need to prove: $$d(x, z) \leq d(x, y) + d(y, z) \quad \text{for all } x,y,z \in V$$ That is: $$\|x - z\| \leq \|x - y\| + \|y - z\|$$ Proof (Step by Step): Step 1: Reduce to norm triangle inequality Note that: $$x - z = (x - y) + (y - z)$$ So it is enough to prove the triangle inequality for the norm: $$\|u + v\| \leq \|u\| + \|v\| \quad \forall u,v \in V$$ Step 2: Use the inner product definition Expand the square of the norm: $$\|u + v\|^2 = \langle u+v, u+v \rangle = \langle u,u \rangle + 2\text{Re}\langle u,v \rangle + \langle v,v \rangle = \|u\|^2 + 2\text{Re}\langle u,v \rangle + \|v\|^2$$ Step 3: Apply Cauchy-Schwarz Inequality By Cauchy-Schwarz: $$|\langle u, v \rangle| \leq \|u\| \|v\|$$ So: $$\text{Re}\langle u, v \rangle \leq |\langle u, v \rangle| \leq \|u\| \|v\|$$ Therefore: $$\|u + v\|^2 \leq \|u\|^2 + 2\|u\|\|v\| + \|v\|^2 = (\|u\| + \|v\|)^2$$ Step 4: Take square roots Since the square root is increasing: $$\|u + v\| \leq \|u\| + \|v\|$$ This proves the norm triangle inequality, and therefore the metric triangle inequality. ### What's a Hilbert Space? Upcoming blog is a work in progress! Many ideas in math begin with geometric intuition, points can have distance, vectors can have length, and two directions can meet at an angle. In common Euclidean space, these ideas feel strongly interconnected. The distance between two points, the length of a vector, and the angle between two vectors are connected from the same familiar geometry. However, as we want to represent a richer and more diverse set of problems these concepts separate. We may want to talk about distance of elements without having notion of length or angles. Perhaps, we want a structure rich enough to measure both length and direction. This is where the metric space and inner product space concepts enter. A **metric space** is constructed around the core idea of distance. This construction enables us to measure the distance between two elements in a set, sequence convergence, and a formal definition for continuity. An **inner product space** begins with a way to multiply vectors (the fundamental unit from the inherited *vector space* this is defined in) producing geometric information including - lengths - angles - orthogonality The key relation between these two mathematical objects is that *Every inner product space gives rise to a metric **however** not every metric space comes from an inner product.* In this blog we are going to compare these two spaces and define key properties of each. ## Metric Spaces A metric space is a set of elements together with a metric function that allows us to measure the distance between any two elements. The formal definition is the following: Let $X$ be a set, and let $d: X \times X \rightarrow \mathbb{R}$ be the distance/metric function such that $\forall x,y,z \in X$ we have the properties: 1. Non-negative Distance: $d(x,y) \geq 0$ 2. Identity of Indiscernibles: $d(x,y) = 0$ if and only if $x=y$ - A point has distance 0 from itself - Distinct points have positive distance from each other 3. Symmetry: $d(x,y) = d(y,x)$ - The distance between two elements doesn't change with respect to the ordering of the input arguments 4. Triangle Inequality: $d(x,z) \leq d(x,y) + d(y,z)$ The **Metric Space** is defined as the pair $(X,d)$ such that $X$ is a set of elements and for all elements $x,y,z \in X$ satisfy the four metric axioms. ### Examples Let's give some examples of what a metric space is to motivate and see how this construction is useful! #### Metric Space 1: $d(x,y) := |x-y|$ on the reals $\mathbb{R}$ For this example, we define the metric space such that the pair $(X,d)$ is defined as: $$ (X,d) = (\mathbb{R}, d(x,y) := |x-y|\ \forall x,y \in \mathbb{R})$$ Let's show how this pair satisfies the conditions needed for a metric space. 1. We can see that $|x-y|$ is always greater than or equal to zero. Simply imagine (or plot) the absolute value function $f(z) = |z|$ and take $z = x-y$. 2. If $x=y$, we get $|x-x| = 0$. The vice versa can be seen easily. 3. We can see that $|x-y| = |y-x|$ since we're taking the absolute value of the difference between them so if $z = x-y$ we get that $|z| = |-z|$. 4. Finally, we get to have some fun! The triangle inequality for this: $$ d(x,z) \leq d(x,y) + d(y,z)$$ is the following: $$ |x-z| \leq |x-y| + |y-z|$$ Let $ a = x - y $ and $ b = y - z $. Then $ x - z = a + b $, and we need to show $ |a + b| \leq |a| + |b| $. Consider two cases: **Case 1**: $ a + b \geq 0 $. Then $ |a + b| = a + b $. Since $ a \leq |a| $ and $ b \leq |b| $, we have $ a + b \leq |a| + |b|. $ **Case 2**: $ a + b < 0 $. Then $ |a + b| = -(a + b) = (-a) + (-b) $. Since $ -a \leq |a| $ and $ -b \leq |b| $, we again get $ (-a) + (-b) \leq |a| + |b|. $ In both cases, $ |a + b| \leq |a| + |b| $, which completes the proof. Note, this may look extremely familiar if you read my blogs on the [L1 Regularizer](https://www.noteblogdoku.com/blog/l1-lasso-regularization-sparsity) where we regularize a Loss Function such that: $$ L(w) = || y + Xw||^2 + \lambda \sum |w_i| $$ We can see that the distance function $d(x,y) = |x-y|$ is the distance function induced by the L1 Norm in one dimension (real line). It is extremely satisfying to see how concepts connect together in mathematics! #### Metric Space 2: The discrete (trivial) metric on set $X$ The discrete metric on set $X$ is defined as $$d(x, y) = \begin{cases} 0 & \text{if } x = y \\ 1 & \text{if } x \neq y \end{cases}$$ Where basically the distance between any two elements is 1 and the distance between an element and itself is 0. <img src="/media/images/20260531_230722_Screenshot_2026-05-31_at_4.06.18_PM.png" alt="Screenshot 2026 05 31 at 4.06.18 PM" style="max-width: min(400px,80%); height: auto;"> <center> $\textbf{Figure 1}$: Discrete Metric over the set $\{A,B,C,D\} = X$ </center> We can see that this easily satisfies the first three properties for a distance function for a metric space. For the triangle inequality, we get that the only possible values are 0 and 1. The worst case is $1 \leq 1+1$ and other cases are trivial from this one. This distance function is really neat since it works on **any** set $X$. #### Metric Space 3: Continuous Functions on the interval $[a,b]$, i.e. $C([a,b],\mathbb{R})$ with Supremum Metric Define the metric on $C([a,b],\mathbb{R})$ (real continuous functions on interval $a,b$ inclusive) to be: $$ d(f,g) := sup_{x \in [a,b]} |f(x) - g(x)| $$ Note that $d(f,g)$ is finite as $|f(x) - g(x)|$ is a continuous function on a closed bounded interval $[a,b]$ bounding the metric. The first three requirements for a metric space are trivial to see from this definition. For the triangle inequality we have: $$ d(f,g) = sup_{x \in [a,b]} |f(x) - g(x)| $$ $$ d(f,g) = sup_{x \in [a,b]} |f(x) - h(x) + h(x) - g(x)| $$ $$ d(f,g) \leq sup_{x \in [a,b]} (|f(x) - h(x)| + |h(x) - g(x)|) $$ $$ d(f,g) \leq sup_{x \in [a,b]} (|f(x) - h(x)|) + sup_{x \in [a,b]} (|h(x) - g(x)|)$$ $$ sup_{x \in [a,b]} (|f(x) - h(x)|) + sup_{x \in [a,b]} (|h(x) - g(x)|) = d(f,h) + d(h,g)$$ Thus we get $$ d(f,g) \leq d(f,h) + d(h,g) $$ <img src="/media/images/20260601_003641_Screenshot_2026-05-31_at_5.35.28_PM.png" alt="Screenshot 2026 05 31 at 5.35.28 PM" style="max-width: min(650px,80%); height: auto;"> <center> $\textbf{Figure 2}$: Metric Space for the Set $C([a,b],\mathbb{R})$ </center> This metric space is particularly important because it naturally corresponds to uniform convergence of functions. It serves as a foundational setting for many areas of mathematics, including: - **Fourier Series**, where one studies the uniform approximation of continuous functions by trigonometric polynomials; - **Calculus of Variations**, where functionals (such as energy or length) are defined and optimized over spaces of continuous functions; - **Ordinary Differential Equations**, especially through the Picard–Lindelöf theorem and contraction mapping arguments; - **Approximation Theory**, via results like the Stone–Weierstrass and Weierstrass Approximation Theorems. Additionally, its completeness makes it one of the most important examples of an infinite-dimensional Banach space, playing a central role in functional analysis and the rigorous study of convergence, continuity, and operators on function spaces. ## Inner Product Spaces An Inner Product Space is a vector space where you can multiply two vectors to get a scalar. It consists of a vector space $V$ with an inner product $\langle x,y \rangle$ such that $x,y \in V$. First we need to state the formal definition of a vector space so we can define an inner product space. The formal definition of a **Vector Space** is the following: --- Let $ \mathbb{F} $ be a field (most commonly $ \mathbb{F} = \mathbb{R} $ or $ \mathbb{F} = \mathbb{C} $). A vector space over $ \mathbb{F} $ is a set $ V $ equipped with two operations: Vector addition: $ +: V \times V \to V $ Scalar multiplication: $ \cdot: \mathbb{F} \times V \to V $ such that the following 10 axioms hold for all vectors $ \mathbf{u}, \mathbf{v}, \mathbf{w} \in V $ and all scalars $ \alpha, \beta \in \mathbb{F} $: Axioms for Vector Addition Associativity: $ (\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w}) $ Commutativity: $ \mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u} $ Identity element: There exists a zero vector $ \mathbf{0} \in V $ such that $ \mathbf{u} + \mathbf{0} = \mathbf{u} $ Inverse element: For every $ \mathbf{u} \in V $, there exists $ -\mathbf{u} \in V $ such that $ \mathbf{u} + (-\mathbf{u}) = \mathbf{0} $ Axioms for Scalar Multiplication Distributivity (scalar over vectors): $ \alpha(\mathbf{u} + \mathbf{v}) = \alpha\mathbf{u} + \alpha\mathbf{v} $ Distributivity (vectors over scalars): $ (\alpha + \beta)\mathbf{u} = \alpha\mathbf{u} + \beta\mathbf{u} $ Compatibility: $ \alpha(\beta\mathbf{u}) = (\alpha\beta)\mathbf{u} $ Scalar identity: $1 \cdot \mathbf{u} = \mathbf{u}$ Additional Axioms Closure under addition: $ \mathbf{u} + \mathbf{v} \in V $ Closure under scalar multiplication: $ \alpha\mathbf{u} \in V $ --- Fantastic! Now we can give the formal definition of an inner product space, which builds off vector spaces. The formal definition of an **Inner Product Space** is the following: --- The Inner Product on a vector space $V$ (over $\mathbb{R}$ or $\mathbb{C}$ is a function $\langle \cdot, \cdot \rangle : V \times V \rightarrow \mathbb{K}$ ($\mathbb{K}$ scalar field, usually just $\mathbb{R}, \mathbb{C}$ satisfying 1. Positive Definiteness; $\langle x,x \rangle \geq 0$ & $\langle x,x \rangle = 0 \leftrightarrow x = 0$ 2. Linearity of the First Argument: $\langle ax + by, z \rangle = a \langle x,z \rangle + b \langle y,z \rangle$ 3. Conjugate Symmetry: $\langle x,y \rangle = \bar{\langle y,x \rangle}$ Note: we also have conjugate-linearity in the second argument: $\langle x, ay + bz \rangle = \bar{a}\langle x,y \rangle + \bar{b}\langle x,z \rangle $ Of important note, **if you have an inner product you can always construct a metric from it:** $$ d(x,y) := \sqrt{\langle x-y, x-y \rangle}$$ We can see that $d$ satisfies all metric axioms, with the Triangle Inequality following from the Cauchy-Schwartz Inequality (see **Appendix**). Because of this special property, we can also state that **Inner Product Spaces are special Metric Spaces with Vector-Space geometry**. --- Of special note, Inner Products give us lengths, angles, orthogonality, projections, and distances. - Length (norm) is given by $||u|| = \sqrt{\langle u,u \rangle}$ - Distance is given by $d(u,v) = ||u-v|| = \sqrt{\langle u-v,u-v \rangle}$ - Angle $ cos\ \theta = \frac{\langle u,v \rangle}{||u||||v||}$ - Orthogonality $u \perp v \leftrightarrow \langle u,v \rangle = 0$ - Projection, orthogonal projection of $u$ onto $v$: $proj_v u = \frac{\langle u,v \rangle}{\langle v,v \rangle} v$ The inner product enables us to use these properties in abstract vector spaces. Even in infinite dimensional spaces like $C([a,b], \mathbb{R})$ we can define an inner product such as $\langle f,g \rangle = \int_a^b f(x)g(x) dx$. This is really cool since we can abstract the notion of vector beyond a typical space (like $\mathbb{R}^3$) to things like functions, and can even describe what the length of a function is! The standard $ L^2 $ inner product $ \langle f, g \rangle = \int_a^b f(x)g(x)\, dx $ turns $ C([a,b]) $ into a (pre-)Hilbert space. This allows us to study orthogonal polynomials, Fourier series, etc. ### Examples Let's give some examples of what an inner product space is to motivate and see how this construction is useful! #### Inner Product Space 1: $\mathbb{R}^3$ This is a classic example, the $\mathbb{R}^3$ is a ubiquitous vector space. The inner product is defined as the following for $x,y \in \mathbb{R}^3$ $$ \langle x,y \rangle = \sum_i^3 x_i y_i $$ Since this is such a well known example, I will not prove all three properties. #### Inner Product Space 2: The Space of Continuous Functions with the $L^2$ Inner Product Let $C([a,b])$ be the vector space of all continuous real-valued functions on the closed interval $[a,b]$. We define the inner product on this space as $$ \langle f,g \rangle = \int_a^b f(x)g(x) dx $$ Turning $C([a,b])$ into an inner product space. This inner product space is very useful and has the following geometric concepts we can apply to continuous functions: - Length/Norm $||f||_2 = \sqrt{\langle f,f \rangle} = \sqrt{\int_a^b |f(x)|^2 dx}$ - Distance $d(f,g) = \sqrt{\int_a^b [f(x)-g(x)]^2 \, dx} $ - Orthogonality $ f \perp g \iff \langle f,g \rangle = 0 $ - Angle $ \cos \theta = \frac{\langle f,g \rangle}{\|f\|_2 \|g\|_2} $ - How similar are two functions? Now we have a way to measure! - Projection $ \text{proj}_g f = \frac{\langle f,g \rangle}{\langle g,g \rangle} g $ - Best approximation of one function by another <img src="/media/images/20260601_012129_Screenshot_2026-05-31_at_6.17.10_PM.png" alt="Screenshot 2026 05 31 at 6.17.10 PM" style="max-width: min(650px,80%); height: auto;"> <center> $\textbf{Figure 3}$: $L^2$ Inner Product on $C([a,b])$ enabling us to use Functions as Vectors </center> This example is very important since it opens up a whole new way of using functions and thinking of them as infinite dimensional vectors. 1. Hilbert Spaces When we complete $C([a,b])$ with respect to this norm, we get the famous $L^2[a,b]$ space, one of the most important Hilbert Spaces (see **Appendix**) in mathematics! 2. Fourier Analysis Using this inner product, we can build orthogonal bases and expand functions as infinite series. These bases include - Sines & Cosines - Legendre polynomials - Etc. 3. Applications include - Signal processing - Quantum mechanics (wave functions) - Partial Differential Equations (heat equation, wave equation) 4. Bridges Discrete & Continuous Generalizes the dot product from $\mathbb{R}^n$ to functions. We can see how $ \langle f,g \rangle = \int_a^b f(x)g(x) dx $ is really the infinite dimensional version of the dot product for finite vectors $ \langle u,v \rangle = \sum_i^n u_i v_i$ An example of two orthogonal functions is $sin(x)$ and $cos(x)$ (thanks Fourier!). For the interval $[-\pi, \pi]$, we can see $$ \langle sin(x), cos(x) \rangle = \int_{-\pi}^{\pi} sin(x)cos(x) dx = 0$$ Which is a fundamental reason why we can use Fourier Series as basis functions (linear independence). ## Inner Product Space vs Metric Space | Feature | Metric Space | Inner Product Space | |----------------------------------|-------------------------------|----------------------------------------------| | **Basic object** | Set of points | Vector space | | **Main structure** | Distance function $(d(x,y))$ | Inner product $\langle x, y \rangle$ | | Can measure distance? | Yes | Yes, indirectly | | Can measure length? | Not necessarily | Yes | | Can measure angle? | No, not in general | Yes | | Can talk about orthogonality? | No, not in general | Yes | | Needs vector addition? | No | Yes | | Needs scalar multiplication? | No | Yes | | **More general?** | Yes | No, more specialized | The key difference between **Metric Spaces** and **Inner Product Spaces** is - Metric Spaces: Space where we can measure distances between every pair of two elements $d(x,y) \forall x,y \in X$ for metric space $(X,d)$ - Inner Product Spaces: Space where we inherit all properties of vector spaces & can compare vectors to get length, distance, orthogonality, angles, and projections. Furthermore, **Inner Product Spaces are special Metric Spaces with Vector-Space geometry**. --- ## Appendix ### What does $d: X \times X \rightarrow \mathbb{R}$ mean? This is standard mathematical notation that means $d$ is a function that takes two elements form the set $X$ where $x,y \in X$ and outputs a real number. $d$ the name of the function, $X \times X$ is the domain, which is the Cartesian product of the set $X$ with itself. This is the set of all possible ordered pairs $(x,y)$ where $x,y \in X$. $\mathbb{R}$ is the codomain which is the set of real numbers. So the output of $d(x,y)$ is always a real number. ### Triangle Inequality showing Metrics can always be constructed from Inner Products We need to prove: $$d(x, z) \leq d(x, y) + d(y, z) \quad \text{for all } x,y,z \in V$$ That is: $$\|x - z\| \leq \|x - y\| + \|y - z\|$$ Proof (Step by Step): Step 1: Reduce to norm triangle inequality Note that: $$x - z = (x - y) + (y - z)$$ So it is enough to prove the triangle inequality for the norm: $$\|u + v\| \leq \|u\| + \|v\| \quad \forall u,v \in V$$ Step 2: Use the inner product definition Expand the square of the norm: $$\|u + v\|^2 = \langle u+v, u+v \rangle = \langle u,u \rangle + 2\text{Re}\langle u,v \rangle + \langle v,v \rangle = \|u\|^2 + 2\text{Re}\langle u,v \rangle + \|v\|^2$$ Step 3: Apply Cauchy-Schwarz Inequality By Cauchy-Schwarz: $$|\langle u, v \rangle| \leq \|u\| \|v\|$$ So: $$\text{Re}\langle u, v \rangle \leq |\langle u, v \rangle| \leq \|u\| \|v\|$$ Therefore: $$\|u + v\|^2 \leq \|u\|^2 + 2\|u\|\|v\| + \|v\|^2 = (\|u\| + \|v\|)^2$$ Step 4: Take square roots Since the square root is increasing: $$\|u + v\| \leq \|u\| + \|v\|$$ This proves the norm triangle inequality, and therefore the metric triangle inequality. ### What's a Hilbert Space? Upcoming blog is a work in progress! Comments (0) Please log in to comment. No comments yet. Be the first to comment! ← Back to Lessons
Comments (0)
Please log in to comment.
No comments yet. Be the first to comment!