Vectors and Linear Combinations

TL;DR

• Definition of a vector
  • Vector in the narrow sense (Euclidean vector): a physical quantity that has both magnitude and direction
  • Vector in the broad, linear-algebraic sense: an element of a vector space
• Ways to represent vectors
  • Arrow representation: the vector’s magnitude is the length of the arrow, and its direction is the arrow’s direction. It is easy to visualize and intuitive, but it is difficult to represent higher-dimensional vectors (4D and above) or non-Euclidean vectors.
  • Component representation: place the tail of the vector at the origin of a coordinate space and express the vector by the coordinates of its head.
• Basic operations on vectors
  • Sum: $(a_1, a_2, \cdots, a_n) + (b_1, b_2, \cdots, b_n) := (a_1+b_1, a_2+b_2, \cdots, a_n+b_n)$
  • Scalar multiplication: $c(a_1, a_2, \cdots, a_n) := (ca_1, ca_2, \cdots, ca_n)$
• Linear combination of vectors
  • For finitely many vectors $\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_n$ and scalars $a_1, a_2, \dots, a_n$, a vector $\mathbf{v}$ satisfying $\mathbf{v} = a_1\mathbf{u}_1 + a_2\mathbf{u}_2 + \cdots + a_n\mathbf{u}_n$ is called a linear combination of $\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_n$.
  • The numbers $a_1, a_2, \dots, a_n$ are called the coefficients of this linear combination.
• Span
  • For a nonempty subset $S$ of a vector space $\mathbb{V}$, the set of all linear combinations formed from vectors in $S$, denoted $\mathrm{span}(S)$.
  • By definition, $\mathrm{span}(\emptyset) = \{0\}$.
  • For a subset $S$ of a vector space $\mathbb{V}$, if $\mathrm{span}(S) = \mathbb{V}$, then $S$ is said to generate (or span) $\mathbb{V}$.

Prerequisites

• Coordinate plane/coordinate space
• Field

What is a vector?

Vector in the narrow sense: Euclidean vector

Many physical quantities such as force, velocity, and acceleration carry not only magnitude but also directional information. A physical quantity that has both magnitude and direction is called a vector.

The definition above is the one used in mechanics in physics and in high-school-level mathematics. A vector in this geometric sense—“the magnitude and direction of a directed line segment,” grounded in physical intuition—is more precisely called a Euclidean vector.

Vector in the broad sense: an element of a vector space

In linear algebra, vectors are defined more broadly than Euclidean vectors, as an abstract algebraic structure:

Definition
A vector space (or linear space) $\mathbb{V}$ over a field $F$ is a set equipped with two operations, sum and scalar multiplication, satisfying the following eight axioms. Elements of the field $F$ are called scalars, and elements of the vector space $\mathbb{V}$ are called vectors.

• Sum: For any $\mathbf{x}, \mathbf{y} \in \mathbb{V}$, there exists a unique element $\mathbf{x} + \mathbf{y} \in \mathbb{V}$. We call $\mathbf{x} + \mathbf{y}$ the sum of $\mathbf{x}$ and $\mathbf{y}$.
• Scalar multiplication: For any $a \in F$ and $\mathbf{x} \in \mathbb{V}$, there exists a unique element $a\mathbf{x} \in \mathbb{V}$. In this case, $a\mathbf{x}$ is called the scalar multiple of $\mathbf{x}$.

1. For all $\mathbf{x},\mathbf{y} \in \mathbb{V}$, $\mathbf{x} + \mathbf{y} = \mathbf{y} + \mathbf{x}$. (commutativity of addition)
2. For all $\mathbf{x},\mathbf{y},\mathbf{z} \in \mathbb{V}$, $(\mathbf{x}+\mathbf{y})+\mathbf{z} = \mathbf{x}+(\mathbf{y}+\mathbf{z})$. (associativity of addition)
3. There exists $\mathbf{0} \in \mathbb{V}$ such that $\mathbf{x} + \mathbf{0} = \mathbf{x}$ for all $\mathbf{x} \in \mathbb{V}$. (zero vector, additive identity)
4. For each $\mathbf{x} \in \mathbb{V}$, there exists $\mathbf{y} \in \mathbb{V}$ such that $\mathbf{x} + \mathbf{y} = \mathbf{0}$. (additive inverse)
5. For each $\mathbf{x} \in \mathbb{V}$, $1\mathbf{x} = \mathbf{x}$. (multiplicative identity)
6. For all $a,b \in F$ and $\mathbf{x} \in \mathbb{V}$, $(ab)\mathbf{x} = a(b\mathbf{x})$. (associativity of scalar multiplication)
7. For all $a \in F$ and $\mathbf{x},\mathbf{y} \in \mathbb{V}$, $a(\mathbf{x}+\mathbf{y}) = a\mathbf{x} + a\mathbf{y}$. (distributivity of scalar multiplication over vector addition)
8. For all $a,b \in F$ and $\mathbf{x} \in \mathbb{V}$, $(a+b)\mathbf{x} = a\mathbf{x} + b\mathbf{x}$. (distributivity of scalar multiplication over field addition)

This definition of a vector in linear algebra encompasses a broader class than the previously mentioned Euclidean vector. You can verify that Euclidean vectors satisfy these eight properties.

The origin and development of vectors are closely tied to practical problems in physics—such as describing force, motion, rotation, and fields quantitatively. The concept was first introduced as Euclidean vectors to meet the physical need to mathematically express natural phenomena. Mathematics then generalized and systematized these physical ideas, establishing formal structures such as vector spaces, inner products, and exterior products, leading to today’s definition of vectors. In other words, vectors are concepts demanded by physics and formalized by mathematics—an interdisciplinary product developed through close interaction between the two communities, rather than a creation of pure mathematics alone.

The Euclidean vectors handled in classical mechanics can be expressed within a more general framework mathematically. Modern physics actively uses not only Euclidean vectors but also more abstract notions defined in mathematics—vector spaces, function spaces, etc.—and attaches physical meaning to them. Hence it is inappropriate to regard the two definitions of a vector as merely “the physical definition” and “the mathematical definition.”

We will defer a deeper dive into vector spaces and, for now, focus on Euclidean vectors—vectors in the narrow sense that admit geometric representation in coordinate spaces. Building intuition with Euclidean vectors first will be helpful when generalizing to other kinds of vectors later.

Ways to represent vectors

Arrow representation

This is the most common and most geometrically intuitive representation. The vector’s magnitude is represented by the length of an arrow, and its direction by the direction of the arrow.

Image credits

• Author: Wikipedia user Nguyenthephuc
• License: CC BY-SA 3.0

While intuitive, this arrow representation has clear limitations for higher-dimensional vectors (4D and above). Moreover, we will eventually need to handle non-Euclidean vectors that are not easily depicted geometrically, so it is important to become comfortable with the component representation described next.

Component representation

Regardless of where a vector is located, if its magnitude and direction are the same, we consider it the same vector. Therefore, given a coordinate space, if we fix the tail of the vector at the origin of that coordinate space, then an $n$-dimensional vector corresponds to an arbitrary point in $n$-dimensional space, and we can represent the vector by the coordinates of its head. This is called the component representation of a vector.

\[(a_1, a_2, \cdots, a_n) \in \mathbb{R}^n \text{ or } \mathbb{C}^n\]

Image credits

• Author: Wikimedia user Acdx
• License: CC BY-SA 3.0

Basic operations on vectors

The two basic operations on vectors are sum and scalar multiplication. Every vector operation can be expressed as a combination of these two.

Vector addition

The sum of two vectors is again a vector; its components are obtained by adding the corresponding components of the two vectors.

\[(a_1, a_2, \cdots, a_n) + (b_1, b_2, \cdots, b_n) := (a_1+b_1, a_2+b_2, \cdots, a_n+b_n)\]

Scalar multiplication of vectors

A vector can be scaled up or down by multiplying it by a scalar (a constant); the result is obtained by multiplying each component by that scalar.

\[c(a_1, a_2, \cdots, a_n) := (ca_1, ca_2, \cdots, ca_n)\]

Image credits

• Author: Wikipedia user Silly rabbit
• License: CC BY-SA 3.0

Linear combinations of vectors

Just as calculus starts from numbers $x$ and functions $f(x)$, linear algebra starts from vectors $\mathbf{v}, \mathbf{w}, \dots$ and their linear combinations $c\mathbf{v} + d\mathbf{w} + \cdots$. Every linear combination of vectors is built from the two basic operations above, sum and scalar multiplication.

Given finitely many vectors $\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_n$ and scalars $a_1, a_2, \dots, a_n$, a vector $\mathbf{v}$ satisfying

\[\mathbf{v} = a_1\mathbf{u}_1 + a_2\mathbf{u}_2 + \cdots + a_n\mathbf{u}_n\]

is called a linear combination of $\mathbf{u}_1, \mathbf{u}_2, \dots, \mathbf{u}_n$. The numbers $a_1, a_2, \dots, a_n$ are the coefficients of this linear combination.

Why are linear combinations important? Consider the following situation: $n$ vectors in $m$-dimensional space form the $n$ columns of an $m \times n$ matrix.

\[\begin{gather*} \mathbf{v}_1 = (a_{11}, a_{21}, \dots, a_{m1}), \\ \mathbf{v}_2 = (a_{12}, a_{22}, \dots, a_{m2}), \\ \vdots \\ \mathbf{v}_n = (a_{1n}, a_{2n}, \dots, a_{mn}) \\ \\ A = \Bigg[ \mathbf{v}_1 \quad \mathbf{v}_2 \quad \cdots \quad \mathbf{v}_n \Bigg] \end{gather*}\]

The key questions are:

1. Describe all possible linear combinations $Ax = x_1\mathbf{v}_1 + x_2\mathbf{v}_2 + \cdots + x_n\mathbf{v}_n$. What do they form?
2. Given a desired output vector $b$, find numbers $x_1, x_2, \dots, x_n$ such that $Ax = b$.

We will return to the second question later; for now, focus on the first. To simplify, consider the case of two nonzero 2D vectors ($m=2$, $n=2$).

The linear combination $c\mathbf{v} + d\mathbf{w}$

A vector $\mathbf{v}$ in 2D has two components. For any scalar $c$, the vector $c\mathbf{v}$ traces an infinitely long line through the origin in the $xy$-plane, parallel to the original vector $\mathbf{v}$.

If the given second vector $\mathbf{w}$ is not on this line (i.e., $\mathbf{v}$ and $\mathbf{w}$ are not parallel), then $d\mathbf{w}$ traces another line. Combining these two lines, we see that the linear combination $c\mathbf{v} + d\mathbf{w}$ fills a single plane that includes the origin.

Image credits

• Author: Wikimedia user Svjo
• License: CC BY-SA 4.0

Span

In this way, linear combinations of vectors form a vector space, a process called spanning.

Definition
For a nonempty subset $S$ of a vector space $\mathbb{V}$, the set of all linear combinations formed from vectors in $S$ is called the span of $S$ and is denoted by $\mathrm{span}(S)$. By definition, $\mathrm{span}(\emptyset) = \{0\}$.

Definition
For a subset $S$ of a vector space $\mathbb{V}$, if $\mathrm{span}(S) = \mathbb{V}$, then $S$ is said to generate (or span) $\mathbb{V}$.

Although we have not yet introduced concepts such as subspaces and bases, recalling this example will help you understand the concept of a vector space.