ClassS04CS141/Notes 05 03

Dynamic programming, continued

Subset Sum

The subset-sum problem is defined as follows:

You are given a set S of non-negative integers: {A[1], ..., A[n]} and a goal G (also a non-negative integer).

The question is whether, for any subset S' ⊆ S, the sum of the integers in S' is G.

For i=1,2,..,n and h=0,1,...,G, define

M[i,h] = true if there exists a subset of {A[1], A[2], ..., A[i]} that sums to h.

Claim: M satisfies the following recurrence:

M[0,0] = true

M[0,h] = false for h > 0

M[i,h] = ( M[i-1,h] or M[i-1,h-A[i]] )

This is because any subset S'' of {A[1],A[2],...,A[i]} summing to h either contains A[i] or doesn't. If the subset S'' doesn't contain A[i], then the subset is also a subset of {A[1],...,A[i-1]} summing to h. If the subset S'' does contain A[i], then removing A[i] yields a subset of {A[1],...,A[i-1]} that sums to h-A[i].

(This is similar to computing n choose k (C(n,k)), as discussed in the notes for the previous class [/Notes 04 30.]?)

This gives the following recursive algorithm:

 bool subset-sum(Array<int> A, int i, int h) {
    if (i == 0)  return (h == 0);
    else return subset-sum(A, i-1, h) || subset-sum(A, i-1, h-A[i]);
 }

As implemented, the running time is exponential (something like 2ⁱ). (What is the recursion tree?)

We can modify it to cache the answers:

 bool subset-sum(Array<int> A, int i, int h) {
    static Array<Array<bool> > M;

    if (M[i].exists(h)) return M[i][h];

    if (i == 0)  return (h == 0);
    else return (M[i][h] = (subset-sum(A, i-1, h) || subset-sum(A, i-1, h-A[i])));
 }

Now what does the recursion diagram look like?

Since there are at most n*G distinct subproblems, and the time spent in each call (not counting the recursion) is O(1), the total time is O(n*G).

Knapsack

The Knapsack problem is defined as follows:

You are given, for a collection of n items {1,2,...,n}, the cost cost[i] and value value[i] for each item i. Each cost and value is a non-negative integer.

You are also given a budget B (a non-negative integer).

The goal is to find the maximum value of any subset of the items having total cost at most B.

Formally, given a subset S of the items, define cost(S) = ∑_{i∈ S} cost[i] and define value(S) = ∑_{i∈ S} value[i]. The goal is to compute

max { value(S) : S ⊆ {1,2,...,n} and cost(S) ≤ B}.

Define V[i,b] = max { value(S) : S ⊆ {1,2,...,i} and cost(S) = b}. In words, V[i,b] is the maximum value of any subset having total cost exactly b and using only items in {1,2,...,i}.

Claim:

V[0,0] = 0

V[0,b] = -∞ for b ≠ 0

V[i,b] = max { V[i-1,b], V[i-1, b-cost[i]] + value[i] } for (i > 0)

Naive recursive algorithm takes exponential time (something like 2ⁱ).

Modifying the algorithm to cache answers gives an O(n*B)-time algorithm.