Lecture 7

Another common variant is known as strong induction. This allows you to assume all previous cases in your induction step, rather than just the last one.

It looks like the following:
Definition: [Strong induction] Let $P(n)$ be a statement that depends on a natural number $n$. Then if

1. $P(0)$ is true, and

2. for all $k$, if $P(0), P(1), \ldots P(k)$ are all true, then $P(k+1)$ is also true,

then $P(n)$ is true for all $n\in\mathbb{N}$.

Again, this can be viewed as a cleverly disguised version of ordinary induction: if we define the statement $Q(n)$ as follows: $\begin{aligned} Q(n) &= P(0)\wedge P(1)\wedge\cdots\wedge P(n-1)\wedge P(n)\\ &= \text{``all the statements $P(0),\ldots,P(n)$ are true'',}\end{aligned}$ then proving $Q(n)$ by ordinary induction works out to be effectively the same thing as proving $P(n)$ by strong induction.

Indeed, the base case $Q(0)$ is the same thing as the base case $P(0)$. The induction step $Q(k)\Rightarrow Q(k+1)$ looks like $P(0)\wedge\cdots\wedge P(k)\Rightarrow P(0)\wedge\cdots\wedge P(k)\wedge P(k+1).$

In order to prove this, we assume $P(0),\ldots,P(k)$ are all true and have to prove that $P(0),\ldots,P(k+1)$ are all true. But then all of these except the last are assumptions: what is left is to prove $P(k+1)$ assuming $P(0),\ldots,P(k)$, and that’s exactly the induction step of a strong induction.

Here’s an example of strong induction in practice. First we’ll need a definition:
Definition: The Fibonacci numbers $F_0, F_1, \ldots$ are defined by taking $F_0 =0$ and $F_1 = 1$, and then for $n\geq 2$ by $F_{n} = F_{n-1} + F_{n-2}.$

Now for the result in question:

Proposition

For all $n\in\mathbb{N}$ we have $F_n<2^n$.

Proof

Let $P(n)$ be the statement $F_n<2^n$. We know that $P(0)$ is true, since $F_0 = 0<1 = 2^0.$ We also know that $P(1)$ is true, since $F_1 = 1<2 = 2^1.$

Now we’ll show that for all $k\geq 1$ we have that if $P(0),\ldots,P(k)$ are all true, then $P(k+1)$ is true. So suppose that $P(0),\ldots, P(k)$ are all true.

We now have that $\begin{aligned} F_{k+1} &= F_k + F_{k-1}\\ &< 2^k + 2^{k-1}\qquad\text{(using $P(k)$ and $P(k-1)$)}\\ &< 2^k + 2^k\\ &= 2^{k+1},\end{aligned}$ which is exactly $P(k+1)$. This completes the induction step, and so finishes the proof.

That strong induction argument really had two base cases before the induction step.

So we proved $P(0)$, and we proved $P(0)\Rightarrow P(1)$ by proving $P(1)$, and then we proved $P(0)\wedge\cdots\wedge P(k)\Rightarrow P(k+1)$ by proving $P(k-1)\wedge P(k)\Rightarrow P(k+1)$.

I like to think that the proof was arranged according to the shape of the definition of the Fibonacci numbers: that definition has two base cases $F_0 = 0$ and $F_1 = 1$, and a step $F_{n+2} = F_{n+1} + F_n$. This is not a rare coincidence.

Why induction works

In this section we make a few comments on why induction works. They may be helpful in thinking about when you can and when you can’t generalise induction to other settings.

Let’s introduce a definition:
Definition: A set (of numbers) is well-ordered if every nonempty subset has a least element.

For now, our main use of that is to say this:
Definition: The well-ordering principle for $\mathbb{N}$ says that $\mathbb{N}$ is well-ordered.

This is a very special property of $\mathbb{N}$. The integers $\mathbb{Z}$, for example, are not well-ordered. Indeed, the subset $\mathbb{Z}\subset\mathbb{Z}$ of all integers does not have a least element: there is no least integer.

The main interest is this:

Theorem

The well-ordering principle for $\mathbb{N}$ and the principle of strong induction are equivalent.

Proof

We’ll show first that we can derive the principle of strong induction from the well-ordering principle.

So suppose we had a statement $P(n)$ for each $n\in\mathbb{N}$, and we had a base case (that $P(0)$ was true) and an induction step (that, for all $k\in\mathbb{N}$ if we have $P(i)$ for all $i<k$, we also have $P(k)$). We need to show that $P(n)$ is true for all $n$.

We might argue as follows. Let $A$ be the set of natural numbers $n$ for which $P(n)$ does not hold: $A = \left\{n\in\mathbb{N}\mid \neg P(n)\right\}.$ So $A$ is the set of “counterexamples”.

If $A$ has any elements at all, it has a smallest element $a$. But $a$ can’t be $0$, because we have $P(0)$. But $a$ can’t be bigger than $0$ either: because $a$ is minimal, we have $P(i)$ for all $i<a$. Hence we have $P(a)$ also, by the induction step. But that’s a contradiction: we assumed that $\neg P(a)$.

Hence $A$ doesn’t have any elements, which is the same as saying that $P(n)$ holds for all $n$.

Now we’ll show the other half of the equivalence: that we can derive the well-ordering principle from the principle of strong induction.

Let $Q(n)$ be the statement, “any subset of $\mathbb{N}$ which contains $n$ has a smallest element”. We’ll prove $Q(n)$ for all $n$ by strong induction.

Firstly, for a base case, we must prove $Q(0)$ (“any subset of $\mathbb{N}$ which contains $0$ has a smallest element”). This is clearly true, as $0$ is the smallest natural number of all, so any such subset has $0$ as its smallest element.

Now we must prove the induction step: we assume for some $k$ that $Q(i)$ is true for all $i<k$, and we prove that $Q(k)$ is true. Consider a subset $S\subset\mathbb{N}$ which contains $k$. If it contains some element $i<k$, then by $Q(i)$ it has a least element. If, however, it contains no element $i<k$, then $k$ is its least element: so in particular, it has a least element. This proves $Q(k)$.

Hence we have $Q(n)$ for all $n$ by strong induction. So if $S$ is a nonempty subset of $\mathbb{N}$, it has at least one element: call it $n$. But then by $Q(n)$, the set $S$ has a least element: this proves the well-ordering principle.

Part of the reason this is such good news is that there are other well-ordered sets, besides the natural numbers. Whenever you find a well-ordering, you get a notion of induction for free.

For example, consider the set of pairs $(m,n)$ of naturals, where we say that $(m,n)<(m',n')$ if $m<m'$, or if $m=m'$ and $n<n'$. (This is called the lexicographic ordering, because it’s inspired by the way that words in dictionaries are ordered).

It is not too hard to prove that that is well-ordered: any set of pairs of natural numbers has a “least” element with respect to this ordering. Hence we can do (strong) induction on pairs of naturals!