Consider an Array class (as described for programming assignment 1 /Prog1) implemented by doubling the array size when necessary (as described in class wednesday week 1 --- /Notes 03 31).
Consider adding the following method (described in pseudo-code below) to that data structure:
set_size(int i) max_referenced = i-1. // possibly resize the table: while (i >= table_size) grow() --- double the size of the table until i < table_size. while (i < table_size/2) shrink() --- cut the table in half until i >= table_size / 2.
The function grow() doubles the size of the table as described in class (by replacing the current array with an appropriately initialized array twice as large). The operation grow() takes time proportional to the current table_size.
The function shrink() decreases the size of the table to half its current size, by replacing the current array with an appropriately initialized array of half the size. This saves memory. The operation shrink() takes time proportional to the current table_size.
Now suppose we implement a Stack class using the Array class as follows:
template <class VALUE>
class Stack {
Array<VALUE> array;
public:
void push(const VALUE &v) { array[array.size()] = v; }
VALUE top() const { if (array.size() > 0) return array[array.size()-1]; else exit(-1); }
VALUE pop()
if (array.size() > 0) {
VALUE v = top();
array.set_size(array.size()-1);
return v;
} else exit(-1);
}
}
1A. Show that with this implementation of the Stack class, there is a sequence of N push() and pop() operations that takes total time proportional to N2.
One such sequence of pushes and pops is as follows:
Do N/4 pushes. Then, do just enough pushes (at most N/4) so that the grow() function is called. Immediately after the push that results in grow() being called, alternate pop() and push() operations until a total of N operations have been done.
Now, why does this make the implementation take time proportional to N2? Let S be the size of the table just before the first pop(). Let E be the number of elements in the table at that point. Just before the grow() operation before the first pop(), the table size was equal to E, and then the grow() operation doubled it. So, the new size, S, equals 2E, while the number of elements is E.
When the first pop() happens, the number of elements becomes E-1. Since the table size is 2E, the number of elements is now less than half the table size. Thus, shrink() is called, which cuts the size of the table in half. At this point, the number of elements is E-1 and the size of the table is E.
When the next push() happens, the number of elements becomes E again, so grow() is called. bringing the table size back to 2E, while the number of elements is E.
Thus, before and after each pop(),push() pair in the sequence, there are E elements in a table of size 2E. Also, each such pair does one grow and one shrink, and thus takes time proportional to E. Since there are at least N/4 such pairs in the sequence, and E≥ N/4) the total work done is at least proportional to \(N2.
set_size(int i) max_referenced = i-1. while (i >= table_size) grow() while (i < table_size/4) shrink() -- note that the "2" has been changed to "4"
Claim: if the Stack class is implemented with the Array class modified in this way, then any sequence of N push() and pop() operations on such a Stack takes O(N) time.
Prove this claim, or find a counter-example (a sequence of N push() and pop() operations taking more than O(N) time). Hint: if a set_size() call does grow or shrink the table, how many push() or pop() operations must have preceded the call since the previous resizing of the table by set_size()? </i>
A grow() operation happens when #elements == table_size, and leaves #elements == table_size/2.
A shrink() operation happens when #elements == table_size/4 - 1, and leaves #elements == table_size/2 - 1.
Thus, after any grow() or shrink() operation, the number of elements is about half the table size. Before another grow() or shrink() operation to occur, the number of elements must reach either one quarter of the table size, or the full table size.
This means that any grow() or shrink() operation that leaves the table at some size T (and thus requiring O(T) work) must have been preceedeed by Ω(T) constant-time push or pop operations (push or pop operations not resulting in a shrink or grow).
Thus, the total work spent growing or shrinking is proportional to the number of constant-time push or pop operations. This is O(N).
Recall the UNION-FIND data type discussed in class on Friday, week 1 (/Notes 04 02).
In that class we discussed a particular implementation of the data type (also described in Section 4.2.2 of the text), and showed that, for that implementation, the total time taken to support N UNION or FIND operations on M elements was O(N + M log M).
Now, consider the following alternate implementation (also outlined in section 4.2.3 of the text):
Each set is represented as a tree; the nodes of the tree represent the elements in that set. The tree edges are directed from each node to its parent node in the tree. The root of the tree has its parent pointer directed to itself.
FIND(e) is implemented by tracing the parent pointers from the node corresponding to e to the root of e's tree, and returning the name of the element stored at the root.
UNION(i,j) is implemented by taking the root node of i's tree and the root node of j's tree, and changing the parent pointer of the root of the smaller tree (the tree of the set with less elements) so that it points to the root of the other tree (thus making one tree out of two).
(See figure 4.8 of the text.)
Suppose the data structure is implemented as described above. (That is, with Union-by-size, but without path compression, as described in section 4.2.3.)
2A. Find a sequence of N UNION and FIND operations on M elements that takes time at least proportional to N log M.
So, we answer the problem as best we can by making the assumption that M ≤ 2N. This is reasonable in some sense, because in N operations we can only touch at most 2N elements.
Let M' = min(M,N/3).
Starting with M' elements, do log(M') phases of unions:
For i = 1,2,..., log(M') do
do M'/2i unions, each joining two sets of size 2i.
This takes M' + M'/2 + M'/4 + ... + 4 + 2 + 1 < 2M' unions total. Each union joins two sets with the same tree structure, each phase increases the height (the distance from the root to the deepest leaf) of the underlying trees by 1. Thus, after the final phase, when only one tree is left, the height of that tree is Ω(log(M').
Next, do finds, each one on a leaf at maximum depth in the tree. Each such find takes time Ω(log(M'). Do enough finds so that the total number of operations is N. This is at least N-2M' finds. Thus, the total work is Ω((N-2M')*log M'.
Since M'=N/3, we have N - 2M' = Ω(N), so the total work is Ω(N log M').
Since we assumed M ≤ 2N, and we defined M' = min(M,N/3), it follows that M' ≥ M/6 = Ω(M). Thus, the total work is Ω(N log M).
2B. Prove that any sequence of N UNION and FIND operations on M elements takes time at most O(N log M).
Thus, if a node is at some depth D, the size of the set it is in must be at least 2D. Since no set ever has size bigger than M, it follows that no node ever has depth bigger than log M.
Each Union operation takes constant time, and each Find operation takes time proportional to the depth of the node found. Since the depth is O(log M), it follows that time for any find operation is O(log M).
Since each of the N operations takes O(log M) time, the total time is O(N log M).
3A. Show that ∑i=1n log(i) is O(n log n). Hint: It is enough to show that the sum is less than (c n log n) for some constant c.
Since n/2 log(n/2) = (n/2) log(n) - O(n/2) = Ω(n log n), it follows that the sum is Ω(n log n).
The n/2 largest terms in the sum are each at least (n/2)c = nc/2c. Thus, the sum is at least (n/2) nc/2c. Since c is constant, this is Ω(nc+1).
4A. Prove by induction that every tree with N nodes has N-1 edges.
Induction step: Let T be a tree with N nodes where N > 1. The tree necessary has at least one node, say L, of degree 1. (Why? Every node has degree at least 1, because the tree is connected. And if every node had degree more than 1, we could find a cycle in it by starting at any vertex and walking until we encounter a vertex we'd already traversed.)
Removing L from the tree, and the edge adjacent to L, leaves a tree T'. (If we are being very careful, we need to argue that removing L does leave a tree. That is, a connected acyclic graph. We won't do that here.) By induction, we can assume that the tree T', which has N-1 nodes, has N-2 edges. Since T' has one less edge than T, the tree T must have (N-2)+1 = N-1 edges.
Proof: By induction on n. The base case is n=1. Clearly the claim is true when n=1: in any set containing just one person, all the people in the set have the same name (since the set has just one person).
For the inductive step, suppose n > 1. Consider any set of n people. Line the people up in a row. Consider people 1,2,..,n-1. Since these people form a group of n-1, we can assume by induction that they all share the same name, say, X. Now consider people 2,3,...,N. Since these people form a group of n-1, we can assume by induction that they all share the same name, say, Y. Now, since people 2,3,..,N-1 are in both groups, and each has name X as well as name Y, it must be that X = Y (each person has only one name). Therefore, all people 1,2,...,N share the same name.
The justification of the inductive step is sound for N > 2, because indeed the two groups of size N-1 described in the argument do overlap. But in the case N=2, the two groups don't overlap, and so the argument fails. Thus, all the argument proves is that, for any N>2, if any group of N-1 people share the same name, then any group of N people do.
In sum, the base case proves it for N=1, and the argument for the inductive step proves that:
But the argument for the inductive step does not prove
Describe the recurrence tree, and give the best big-O upper bounds you can for T(n), for each of the following recurrences. Explain why your big-O upper bounds are correct.
5A. T(n) = 3 T(n/2) + n2, T(1) = 0
5B. T(n) = 4 T(n/2) + n2, T(1) = 0
5C. T(n) = 5 T(n/2) + n2, T(1) = 0
5A. In this case each node has 3 children, and the number of subproblems at the Kth level is 3K. Thus, the total work is
Since this sum is geometric, it is proportional to its largest term. Thus, T(n) = Θ(n2).
5B. In this case, each node has 4 children, and the number of subproblems at the Kth level is 4K. Thus, the total work is
5C. Each node has 5 children, total work is
Since this sum is geometric, it is proportional to its largest term. Thus,