Lecture 7

Another common variant is known as strong induction. This allows you to assume all previous cases in your induction step, rather than just the last one.

It looks like the following:
Definition: [Strong induction] Let P(n)P(n) be a statement that depends on a natural number nn. Then if

  1. P(0)P(0) is true, and

  2. for all kk, if P(0),P(1),P(k)P(0), P(1), \ldots P(k) are all true, then P(k+1)P(k+1) is also true,

then P(n)P(n) is true for all nNn\in\mathbb{N}.

Again, this can be viewed as a cleverly disguised version of ordinary induction: if we define the statement Q(n)Q(n) as follows: Q(n)=P(0)P(1)P(n1)P(n)=“all the statements P(0),,P(n) are true”,\begin{aligned} Q(n) &= P(0)\wedge P(1)\wedge\cdots\wedge P(n-1)\wedge P(n)\\ &= \text{``all the statements $P(0),\ldots,P(n)$ are true'',}\end{aligned} then proving Q(n)Q(n) by ordinary induction works out to be effectively the same thing as proving P(n)P(n) by strong induction.

Indeed, the base case Q(0)Q(0) is the same thing as the base case P(0)P(0). The induction step Q(k)Q(k+1)Q(k)\Rightarrow Q(k+1) looks like P(0)P(k)P(0)P(k)P(k+1).P(0)\wedge\cdots\wedge P(k)\Rightarrow P(0)\wedge\cdots\wedge P(k)\wedge P(k+1).

In order to prove this, we assume P(0),,P(k)P(0),\ldots,P(k) are all true and have to prove that P(0),,P(k+1)P(0),\ldots,P(k+1) are all true. But then all of these except the last are assumptions: what is left is to prove P(k+1)P(k+1) assuming P(0),,P(k)P(0),\ldots,P(k), and that’s exactly the induction step of a strong induction.

Here’s an example of strong induction in practice. First we’ll need a definition:
Definition: The Fibonacci numbers F0,F1,F_0, F_1, \ldots are defined by taking F0=0F_0 =0 and F1=1F_1 = 1, and then for n2n\geq 2 by Fn=Fn1+Fn2.F_{n} = F_{n-1} + F_{n-2}.

Now for the result in question:


For all nNn\in\mathbb{N} we have Fn<2nF_n<2^n.


Let P(n)P(n) be the statement Fn<2nF_n<2^n. We know that P(0)P(0) is true, since F0=0<1=20.F_0 = 0<1 = 2^0. We also know that P(1)P(1) is true, since F1=1<2=21.F_1 = 1<2 = 2^1.

Now we’ll show that for all k1k\geq 1 we have that if P(0),,P(k)P(0),\ldots,P(k) are all true, then P(k+1)P(k+1) is true. So suppose that P(0),,P(k)P(0),\ldots, P(k) are all true.

We now have that Fk+1=Fk+Fk1<2k+2k1(using P(k) and P(k1))<2k+2k=2k+1,\begin{aligned} F_{k+1} &= F_k + F_{k-1}\\ &< 2^k + 2^{k-1}\qquad\text{(using $P(k)$ and $P(k-1)$)}\\ &< 2^k + 2^k\\ &= 2^{k+1},\end{aligned} which is exactly P(k+1)P(k+1). This completes the induction step, and so finishes the proof.

That strong induction argument really had two base cases before the induction step.

So we proved P(0)P(0), and we proved P(0)P(1)P(0)\Rightarrow P(1) by proving P(1)P(1), and then we proved P(0)P(k)P(k+1)P(0)\wedge\cdots\wedge P(k)\Rightarrow P(k+1) by proving P(k1)P(k)P(k+1)P(k-1)\wedge P(k)\Rightarrow P(k+1).

I like to think that the proof was arranged according to the shape of the definition of the Fibonacci numbers: that definition has two base cases F0=0F_0 = 0 and F1=1F_1 = 1, and a step Fn+2=Fn+1+FnF_{n+2} = F_{n+1} + F_n. This is not a rare coincidence.

Why induction works

In this section we make a few comments on why induction works. They may be helpful in thinking about when you can and when you can’t generalise induction to other settings.

Let’s introduce a definition:
Definition: A set (of numbers) is well-ordered if every nonempty subset has a least element.

For now, our main use of that is to say this:
Definition: The well-ordering principle for N\mathbb{N} says that N\mathbb{N} is well-ordered.

This is a very special property of N\mathbb{N}. The integers Z\mathbb{Z}, for example, are not well-ordered. Indeed, the subset ZZ\mathbb{Z}\subset\mathbb{Z} of all integers does not have a least element: there is no least integer.

The main interest is this:


The well-ordering principle for N\mathbb{N} and the principle of strong induction are equivalent.


We’ll show first that we can derive the principle of strong induction from the well-ordering principle.

So suppose we had a statement P(n)P(n) for each nNn\in\mathbb{N}, and we had a base case (that P(0)P(0) was true) and an induction step (that, for all kNk\in\mathbb{N} if we have P(i)P(i) for all i<ki<k, we also have P(k)P(k)). We need to show that P(n)P(n) is true for all nn.

We might argue as follows. Let AA be the set of natural numbers nn for which P(n)P(n) does not hold: A={nN¬P(n)}.A = \left\{n\in\mathbb{N}\mid \neg P(n)\right\}. So AA is the set of “counterexamples”.

If AA has any elements at all, it has a smallest element aa. But aa can’t be 00, because we have P(0)P(0). But aa can’t be bigger than 00 either: because aa is minimal, we have P(i)P(i) for all i<ai<a. Hence we have P(a)P(a) also, by the induction step. But that’s a contradiction: we assumed that ¬P(a)\neg P(a).

Hence AA doesn’t have any elements, which is the same as saying that P(n)P(n) holds for all nn.

Now we’ll show the other half of the equivalence: that we can derive the well-ordering principle from the principle of strong induction.

Let Q(n)Q(n) be the statement, “any subset of N\mathbb{N} which contains nn has a smallest element”. We’ll prove Q(n)Q(n) for all nn by strong induction.

Firstly, for a base case, we must prove Q(0)Q(0) (“any subset of N\mathbb{N} which contains 00 has a smallest element”). This is clearly true, as 00 is the smallest natural number of all, so any such subset has 00 as its smallest element.

Now we must prove the induction step: we assume for some kk that Q(i)Q(i) is true for all i<ki<k, and we prove that Q(k)Q(k) is true. Consider a subset SNS\subset\mathbb{N} which contains kk. If it contains some element i<ki<k, then by Q(i)Q(i) it has a least element. If, however, it contains no element i<ki<k, then kk is its least element: so in particular, it has a least element. This proves Q(k)Q(k).

Hence we have Q(n)Q(n) for all nn by strong induction. So if SS is a nonempty subset of N\mathbb{N}, it has at least one element: call it nn. But then by Q(n)Q(n), the set SS has a least element: this proves the well-ordering principle.

Part of the reason this is such good news is that there are other well-ordered sets, besides the natural numbers. Whenever you find a well-ordering, you get a notion of induction for free.

For example, consider the set of pairs (m,n)(m,n) of naturals, where we say that (m,n)<(m,n)(m,n)<(m',n') if m<mm<m', or if m=mm=m' and n<nn<n'. (This is called the lexicographic ordering, because it’s inspired by the way that words in dictionaries are ordered).

It is not too hard to prove that that is well-ordered: any set of pairs of natural numbers has a “least” element with respect to this ordering. Hence we can do (strong) induction on pairs of naturals!