# RecurrenceRelations

ClassS04CS141 | recent changes | Preferences

### Introduction

Recurrence relations are a tool for analyzing recursive algorithms.

#### MergeSort?

An example you may be familiar with is MergeSort? . A recursion diagram for mergesort looks like this:

Define T(n) to be the time taken to mergesort n elements (neglecting constant factors). Then T(n) satisfies the following:

T(1) = 1
T(n) = 2 T(n/2) + n (for n > 1)

The above relation between T(n) and T(n/2) is called a recurrence relation.

The relation is true because the time to mergesort an array with n elements equals the time to mergesort an array with n/2 elements, plus the time to mergesort another array with n/2 elements, plus n.

#### Binary Search

Similarly, a recurrence relation for the time to do a binary search on an array of N elements is:

T(1) = 1
T(N) = 1 + T(N/2).

That is, a binary search in an array of size N does constant work, then recurses on one of the two halves of the array (a problem of size N/2).

The recursion diagram in this case just looks like this:

```
N
|
N/2
|
N/4
|
...
|
4
|
2
|
1

```

In this case, the work associated with each node of the tree is 1, and the depth of the tree is log(N), so the total work done is O(log N).

#### Towers of Hanoi

The number of single-disc moves to transfer N discs in the TowersOfHanoi? problem satisfies:

T(N) = 2 T(N-1) + 1
T(1) = 1

The corresponding recursion diagram is a complete binary tree of depth N, and for each node, 1 disc is moved. Thus, the total number of moves made to transfer N discs is 2N-1.

### Solving recurrence relations

Consider these:

1. T(n) = 2 T(n/3) + n
2. T(n) = 3 T(n/3) + n
3. T(n) = 4 T(n/3) + n

1. recursion diagram is a complete binary tree of depth log3 n. The Ith level from the top has 2i nodes, each with work n/3i.

Total work is

n[ 1 + (2/3) + (2/3)2 + ... + (2/3)log3 n]
which is O(n).

2. same as above but each node has 3 children, so the Ith level from the top has 3i nodes.

Total work is

n[ 1 + (3/3) + (3/3)2 + ... + (3/3)log3 n]
which is O(n log n).

3. now each node has 4 children, so total work is

n[ 1 + (4/3) + (4/3)2 + ... + (4/3)log3 n]
which is O(n (4/3)log3 n) = O(n1+log3 4/3) = O(n1.26...) .

#### More

1. T(n) = 8 T(n/3) + n2
2. T(n) = 9 T(n/3) + n2
3. T(n) = 10 T(n/3) + n2

1. recursion diagram is of depth log3 n. Each node has 8 children. The Ith level from the top has 8i nodes, each with work (n/3i)2 = n2 / 9i .

Total work is

n2[ 1 + (8/9) + (8/9)2 + ... + (8/9)log3 n]
which is O(n2).

2. Total work is

n2[ 1 + (9/9) + (9/9)2 + ... + (9/9)log3 n]
which is O(n2 log n).

3. Total work is

n2[ 1 + (10/9) + (10/9)2 + ... + (10/9)log3 n]
which is O(n2 (10/9)log3 n) = O(n2.095...).

#### Repeated Substitution

T(n) = 1.5 T(n/3.1) + n0.5.

No recursion diagram!?

Try repeated substitution. Use the recurrence relation to replace T(n/3.1) with (1.5 T(n/3.12) + (n/3.1)0.5 :

T(n) = 1.5 T(n/3.1) + n0.5
= 1.5 [ (1.5 T(n/3.12) + (n/3.1)0.5 ] + n0.5 .
Then use the recurrence relation to expand T(n/3.12), and so on, simplifying as you go.

After i substitutions, get T(n) = (1.5)i T(n / 3.1i) + (1.5)i-1 [n/3.1i-1]0.5 + ... .

Taking i = \log3.1 n, after i substitutions the term left is T(1) (or at least T(O(1))), which we substitute using the base case of the recurrence relation. This gives us

T(n) = n0.5[ 1 + (1.5/3.10.5) + (1.5/3.10.5)2 + ... + (1.5/3.10.5){log1.5 n}]

Since 1.5 < 3.10.5, this is O(n0.5). (See GeometricSums .)

#### Proving bounds using induction

1. T(n) = T(n/3) + T(n/2) + n, T(1) = 1.

Can make a recursion diagram for this. Every node has two children, but depth is uneven, and work is not the same for each node on a level. Try guessing, then prove by induction.

Guess that T(n) ≤ 10 n. Attempt proof by induction:

Base case: T(1) ≤ 10 . (true)
Inductive step: T(n) = T(n/3) + T(n/2) + n ≤ 10(n/3) + 10(n/2) + n = (10/3+10/2 + 1) n = 9.33.. n ≤ 10 n. (verified)

What is the best constant that would work?

Guess that T(n) ≤ c n. Attempt proof by induction:

Base case: T(1) ≤ c . (need c ≥ 1 )
Inductive step: T(n) = T(n/3) + T(n/2) + n ≤ c(n/3) + c(n/2) + n = (c(1/3+1/2) + 1) n ≤ c n ?

Need c(1/3+1/2)+1 ≤ c . True iff c ≥ 6 .

Conclude T(n) ≤ 6 n .

### References

ClassS04CS141 | recent changes | Preferences