ClassS04CS141/Hwk1Soln

ClassS04CS141 | ClassS04CS141 | recent changes | Preferences

Showing revision 1
Difference (from revision 1 to revision 1) (minor diff)
(The revisions are identical or unavailable.)

1. Stack via shrinkable array.

Consider an Array class (as described for programming assignment 1 /Prog1) implemented by doubling the array size when necessary (as described in class wednesday week 1 --- /Notes 03 31).

Consider adding the following method (described in pseudo-code below) to that data structure:

set_size(int i) 
   max_referenced = i-1.
   // possibly resize the table:
   while (i >= table_size) grow()   --- double the size of the table until i < table_size.
   while (i < table_size/2) shrink()  --- cut the table in half until i >= table_size / 2.

The function grow() doubles the size of the table as described in class (by replacing the current array with an appropriately initialized array twice as large). The operation grow() takes time proportional to the current table_size.

The function shrink() decreases the size of the table to half its current size, by replacing the current array with an appropriately initialized array of half the size. This saves memory. The operation shrink() takes time proportional to the current table_size.

Now suppose we implement a Stack class using the Array class as follows:

template <class VALUE>
class Stack {
  Array<VALUE> array;

public:
  void push(const VALUE &v) {  array[array.size()] = v;  }
  VALUE top() const { if (array.size() > 0) return array[array.size()-1]; else exit(-1); }
  VALUE pop() 
    if (array.size() > 0) {
       VALUE v = top();
       array.set_size(array.size()-1);
       return v;
    } else exit(-1);
  }
}

1A. Show that with this implementation of the Stack class, there is a sequence of N push() and pop() operations that takes total time proportional to N2.


answer to 1A

One such sequence of pushes and pops is as follows:

Do N/4 pushes. Then, do just enough pushes (at most N/4) so that the grow() function is called. Immediately after the push that results in grow() being called, alternate pop() and push() operations until a total of N operations have been done.

Now, why does this make the implementation take time proportional to N2? Let S be the size of the table just before the first pop(). Let E be the number of elements in the table at that point. Just before the grow() operation before the first pop(), the table size was equal to E, and then the grow() operation doubled it. So, the new size, S, equals 2E, while the number of elements is E.

When the first pop() happens, the number of elements becomes E-1. Since the table size is 2E, the number of elements is now less than half the table size. Thus, shrink() is called, which cuts the size of the table in half. At this point, the number of elements is E-1 and the size of the table is E.

When the next push() happens, the number of elements becomes E again, so grow() is called. bringing the table size back to 2E, while the number of elements is E.

Thus, before and after each pop(),push() pair in the sequence, there are E elements in a table of size 2E. Also, each such pair does one grow and one shrink, and thus takes time proportional to E. Since there are at least N/4 such pairs in the sequence, and E≥ N/4) the total work done is at least proportional to \(N2.


1B. Now suppose that the set_size() method of the Array class is modified as follows:

set_size(int i) 
   max_referenced = i-1.
   while (i >= table_size) grow()
   while (i < table_size/4) shrink()  -- note that the "2" has been changed to "4"

Claim: if the Stack class is implemented with the Array class modified in this way, then any sequence of N push() and pop() operations on such a Stack takes O(N) time.

Prove this claim, or find a counter-example (a sequence of N push() and pop() operations taking more than O(N) time). Hint: if a set_size() call does grow or shrink the table, how many push() or pop() operations must have preceded the call since the previous resizing of the table by set_size()? </i>


answer to 1B

A grow() operation happens when #elements == table_size, and leaves #elements == table_size/2.

A shrink() operation happens when #elements == table_size/4 - 1, and leaves #elements == table_size/2 - 1.

Thus, after any grow() or shrink() operation, the number of elements is about half the table size. Before another grow() or shrink() operation to occur, the number of elements must reach either one quarter of the table size, or the full table size.

This means that any grow() or shrink() operation that leaves the table at some size T (and thus requiring O(T) work) must have been preceedeed by Ω(T) constant-time push or pop operations (push or pop operations not resulting in a shrink or grow).

Thus, the total work spent growing or shrinking is proportional to the number of constant-time push or pop operations. This is O(N).


2. Union-Find using parent pointers.

Recall the UNION-FIND data type discussed in class on Friday, week 1 (/Notes 04 02).

In that class we discussed a particular implementation of the data type (also described in Section 4.2.2 of the text), and showed that, for that implementation, the total time taken to support N UNION or FIND operations on M elements was O(N + M log M).

Now, consider the following alternate implementation (also outlined in section 4.2.3 of the text):

Each set is represented as a tree; the nodes of the tree represent the elements in that set. The tree edges are directed from each node to its parent node in the tree. The root of the tree has its parent pointer directed to itself.

FIND(e) is implemented by tracing the parent pointers from the node corresponding to e to the root of e's tree, and returning the name of the element stored at the root.

UNION(i,j) is implemented by taking the root node of i's tree and the root node of j's tree, and changing the parent pointer of the root of the smaller tree (the tree of the set with less elements) so that it points to the root of the other tree (thus making one tree out of two).

(See figure 4.8 of the text.)

Suppose the data structure is implemented as described above. (That is, with Union-by-size, but without path compression, as described in section 4.2.3.)

2A. Find a sequence of N UNION and FIND operations on M elements that takes time at least proportional to N log M.


This is not possible when N is very small in comparison to M. For example, when N = O(1) and M is large, since only O(N) elements can be involved in the sequence of operations, the total time will be O(N) = O(1), not Ω(log M) = Ω(N log M).

So, we answer the problem as best we can by making the assumption that M ≤ 2N. This is reasonable in some sense, because in N operations we can only touch at most 2N elements.

Let M' = min(M,N/3).

Starting with M' elements, do log(M') phases of unions:

  For i = 1,2,..., log(M') do
     do  M'/2i  unions, each joining two sets of size 2i.

This takes M' + M'/2 + M'/4 + ... + 4 + 2 + 1 < 2M' unions total. Each union joins two sets with the same tree structure, each phase increases the height (the distance from the root to the deepest leaf) of the underlying trees by 1. Thus, after the final phase, when only one tree is left, the height of that tree is Ω(log(M').

Next, do finds, each one on a leaf at maximum depth in the tree. Each such find takes time Ω(log(M'). Do enough finds so that the total number of operations is N. This is at least N-2M' finds. Thus, the total work is Ω((N-2M')*log M'.

Since M'=N/3, we have N - 2M' = Ω(N), so the total work is Ω(N log M').

Since we assumed M ≤ 2N, and we defined M' = min(M,N/3), it follows that M' ≥ M/6 = Ω(M). Thus, the total work is Ω(N log M).


2B. Prove that any sequence of N UNION and FIND operations on M elements takes time at most O(N log M).


Because the union operation makes the root of the smaller tree a child of the root of the larger tree, every time the depth of a node in its tree increases, it must be the case that the size of the set it is in at least doubles.

Thus, if a node is at some depth D, the size of the set it is in must be at least 2D. Since no set ever has size bigger than M, it follows that no node ever has depth bigger than log M.

Each Union operation takes constant time, and each Find operation takes time proportional to the depth of the node found. Since the depth is O(log M), it follows that time for any find operation is O(log M).

Since each of the N operations takes O(log M) time, the total time is O(N log M).


3. O-notation, sums

3A. Show that i=1n log(i) is O(n log n). Hint: It is enough to show that the sum is less than (c n log n) for some constant c.


The sum has n terms, each of which is at most log n. Thus, the sum is at most n log n = O(n log n).
3B. Show that the sum above is Ω(n log n). Hint: It is enough to show that the sum is at least (c' n log n) for some constant c' > 0.
The largest n/2 terms in the sum are each at least log(n/2). Thus, the sum is at least n/2 log(n/2).

Since n/2 log(n/2) = (n/2) log(n) - O(n/2) = Ω(n log n), it follows that the sum is Ω(n log n).


3C: Show that i=0n ic = Θ(nc+1). It is enough to show that the sum is both O(nc+1) and Ω(nc+1).
The sum has n terms, each of which is at most nc. Thus, the sum is at most nc+1 = O(nc+1).

The n/2 largest terms in the sum are each at least (n/2)c = nc/2c. Thus, the sum is at least (n/2) nc/2c. Since c is constant, this is Ω(nc+1).


4. Induction

4A. Prove by induction that every tree with N nodes has N-1 edges.


Base case: Any tree with 1 node has zero edges.

Induction step: Let T be a tree with N nodes where N > 1. The tree necessary has at least one node, say L, of degree 1. (Why? Every node has degree at least 1, because the tree is connected. And if every node had degree more than 1, we could find a cycle in it by starting at any vertex and walking until we encounter a vertex we'd already traversed.)

Removing L from the tree, and the edge adjacent to L, leaves a tree T'. (If we are being very careful, we need to argue that removing L does leave a tree. That is, a connected acyclic graph. We won't do that here.) By induction, we can assume that the tree T', which has N-1 nodes, has N-2 edges. Since T' has one less edge than T, the tree T must have (N-2)+1 = N-1 edges.


4B. What exactly is wrong with the following line of proof? Which step is flawed?
Claim: For every n=1,2,3,..., in any set of n people, all people in the set share the same name.

Proof: By induction on n. The base case is n=1. Clearly the claim is true when n=1: in any set containing just one person, all the people in the set have the same name (since the set has just one person).

For the inductive step, suppose n > 1. Consider any set of n people. Line the people up in a row. Consider people 1,2,..,n-1. Since these people form a group of n-1, we can assume by induction that they all share the same name, say, X. Now consider people 2,3,...,N. Since these people form a group of n-1, we can assume by induction that they all share the same name, say, Y. Now, since people 2,3,..,N-1 are in both groups, and each has name X as well as name Y, it must be that X = Y (each person has only one name). Therefore, all people 1,2,...,N share the same name.


The inductive step claims to prove that, for any N > 1, if any group of N-1 people share the same name, then any group of N people do.

The justification of the inductive step is sound for N > 2, because indeed the two groups of size N-1 described in the argument do overlap. But in the case N=2, the two groups don't overlap, and so the argument fails. Thus, all the argument proves is that, for any N>2, if any group of N-1 people share the same name, then any group of N people do.

In sum, the base case proves it for N=1, and the argument for the inductive step proves that:

if it is true for N=2, then it is true for N=3
if it is true for N=3, then it is true for N=4
if it is true for N=4, then it is true for N=5
and so on.

But the argument for the inductive step does not prove

if it is true for N=1, then it is true for N=2
which is necessary to prove if we are to conclude from the base case that it is true for N=2, N=3, and so on.


5. Recurrence relations

Describe the recurrence tree, and give the best big-O upper bounds you can for T(n), for each of the following recurrences. Explain why your big-O upper bounds are correct.

5A. T(n) = 3 T(n/2) + n2, T(1) = 0

5B. T(n) = 4 T(n/2) + n2, T(1) = 0

5C. T(n) = 5 T(n/2) + n2, T(1) = 0


All recurrence trees have depth log2 n. Each subproblem at the Kth level is of size n/2K and requires work n2/4K (except the leaves which require no work).

5A. In this case each node has 3 children, and the number of subproblems at the Kth level is 3K. Thus, the total work is

K=0..log2 n -1 3K n2/4K
= K=0..log2 n -1 (3/4)K n2

Since this sum is geometric, it is proportional to its largest term. Thus, T(n) = Θ(n2).

5B. In this case, each node has 4 children, and the number of subproblems at the Kth level is 4K. Thus, the total work is

K=0..log2 n -1 4K n2/4K
= Ω(n2 log n) .

5C. Each node has 5 children, total work is

K=0.. log2 n -1 5K n2/4K
= K=0.. log2 n -1 (5/4)K n2

Since this sum is geometric, it is proportional to its largest term. Thus,

T(n) = Θ(n2 (5/4)log2 n).
Using a\log b = b\log a, we can simplify this to
T(n) = Θ(n2+log2 (5/4)).



ClassS04CS141 | ClassS04CS141 | recent changes | Preferences
This page is read-only | View other revisions | View current revision
Edited April 29, 2004 12:45 pm by 66-215-205-98.riv-eres.charterpipeline.net (diff)
Search: