CS5800 Advanced Data Structures and Algorithms

Lectures

# Date Topics covered References
1 Jul 30, 09:00 Administrative details about the course and evaluation. Introduction to algorithm design - TA allocation and stable matchings as an example. [KT] - Chapter 1
[E] - Section 4.5
2 Jul 31, 08:00 Gale-Shapley algorithm to find a stable matching, example instances, proof of correctness. [KT] - Chapter 1
[E] - Section 4.5
3 Aug 2, 12:00 Introduction to asymptotic analysis of algorithms. Implementing the Gale-Shapley algorithm - choosing the best data structures to make the implementation efficient. [KT] - Sections 2.1-2.3
[CLRS] - Chapter 3
4 Aug 5, 10:00 Divide and conquer paradigm - Karatsuba's algorithm for integer multiplication. Solving recurrence relations - guess and verify, examples. [DPV] - Section 2.1
[E] - Section 1.9
[CLRS] - Section 4.3
5 Aug 6, 09:00 Solving recurrence relations - recursion tree method, Master theorem, examples. Strassen's algorithm for matrix multiplication (check the Wikipedia page).
Optional reading: The Akra-Bazzi method for solving recurrence relations.
[E] - Section 1.7
[CLRS] - Sections 4.2-4.5
6 Aug 7, 08:00 Quicksort - example instance, proof of correctness, analyzing the recurrence. Why is an average case analysis assuming a uniformly random input unsatisfactory? Moving to randomized quicksort. [E] - Section 1.5
[CLRS] - Sections 7.1, 7.2
T1 Aug 9, 12:00 Tutorial #1.
7 Aug 13, 09:00 Basic discrete probability - sample space, random variables, expectation.
Optional reading: St. Petersburg paradox
Basic notes on probability
8 Aug 14, 08:00 Linearity of expectation - examples, number of fixed-points of a random permutation, randomized algorithm for Max-3-SAT.
Setting up the analysis of randomized quicksort.
Basic notes on probability
[CLRS] - Section 7.4
9 Aug 16, 12:00 Analysis of the expected running time of randomized quicksort. Extending the idea of the partition algorithm for randomized selection. [CLRS] - Sections 7.4, 9.2
10 Aug 19, 10:00 Randomized selection - analysis of the expected number of comparisons. Linear-time selection in the worst case - median of the medians algorithm. [E] - Section 1.8
[CLRS] - Sections 9.2, 9.3
11 Aug 20, 09:00 Median of median algorithms - writing and solving the recurrence. Finding the closest pair of points in 1D - a divide-and-conquer approach. Extending the idea to points on the plane, improving the brute-force algorithm. [E] - Section 1.8
[CLRS] - Section 33.4
12 Aug 21, 08:00 Finding the closest pair of points in 2D - preprocessing the input to improve the efficiency of the divide-and-conquer algorithm. [CLRS] - Section 33.4
SE1 Aug 23, 12:00 Short exam #1.
13 Aug 26, 10:00 Lower bounds for sorting algorithms in the comparison model - decision trees and adversary arguments. Extending to stronger average-case bounds. [CLRS] - Section 8.1
[notes]
14 Aug 27, 09:00 Average-case lower bounds for sorting algorithms in the comparison model - converting full binary trees to balanced binary trees with smaller average depth.
Discussion of solutions of short exam 1.
[notes]
15 Aug 28, 08:00 Compactly representing a large text with a binary encoding. Improving over fixed-length codes using variable length codes. Notion of prefix-free codes. Constructing prefix-free codes via Shannon-Fano coding. Representing prefix-free codes as binary trees. Moving towards Huffman codes. [KT] - Section 4.8
[CLRS] - Section 16.3
T2 Aug 30, 12:00 Tutorial #2.
16 Sep 3, 09:00 Huffman coding - example. When do greedy algorithms work? - Greedy choice property and optimal substructure. Proving the correctness of Huffman's algorithm. [KT] - Section 4.8
[CLRS] - Section 16.3
[notes]
17 Sep 4, 08:00 Huffman coding - correctness of the algorithm (contd.). Implementation details - priority queues and min-heaps. [KT] - Section 4.8
[CLRS] - Section 16.3, 6.5
18 Sep 6, 12:00 Huffman coding - implementation details (contd). Minimum spanning trees - examples, greedy choices and optimal substructure. [CLRS] - Section 23.1
[notes]
Q1 Sep 7, 10:00 Quiz #1.
19 Sep 9, 10:00 Quiz #1 review. Minimum spanning trees - greedy choices and optimal substructure (contd). [CLRS] - Section 23.1
[notes]
20 Sep 10, 09:00 Graphs with distinct edge weights have unique MST. Boruvka's algorithm - example instance, correctness, analysis, implementation details. [E] - Section 7.1, 7.3
21 Sep 13, 12:00 Kruskal's algorithm - idea, implementation details. Union-Find ADT and its implementation. Obtaining a running time of \( O(m \log n) \) for Kruskal's algorithm using a simple Union-Find data structure. [KT] - Section 4.6
[notes]
22 Sep 16, 10:00 Implementing Union-Find using union-by-rank - analyzing the worst case complexity. The path compression heuristic - algorithm and worst case analysis. [KT] - Section 4.6
[notes]
23 Sep 17, 09:00 Amortized analysis of algorithms - aggregate method and accounting method. Examples - incrementing a binary counter, maintaining a dynamic table with insertions. [CLRS] - Section 17.1, 17.2, 17.4
24 Sep 18, 08:00 Analysis of path compression - obtaining the amortized \( O(\log^* n) \) bound using a clever use of the accounting method. Implications for the running time of Kruskal's algorithm for finding MST. [KT] - Section 4.6
[DPV] - Section 5.1.4
T3 Sep 20, 12:00 Tutorial #3.
25 Sep 23, 10:00 Prim's algorithm - implementation using binary heaps. Improving the amortized complexity of decrease-min using better implementations of priority queues. Binomial trees and binomial heaps. [CLRS] - Section 23.2
[class notes]
SE2 Sep 27, 12:00 Short exam #2.
26 Oct 1, 09:00 Algorithms for binomial heaps. Improving the worst-case complexity of insert and merge by postponing the book-keeping. Extract-min in lazy binomial heaps. [class notes]
[Kaplan and Zwick's slides]
27 Oct 9, 08:00 Extract-min in lazy binomial heaps - amortized analysis using the potential method. Naive approach to improving the complexity of decrease key operation. [class notes]
[Erickson's notes]
[Kaplan and Zwick's slides]
[CLRS] - Section 17.3
28 Oct 11, 12:00 Decrease-key operation using cascading cuts. Bounding the degree of the trees. Amortized analysis of Fibonacci heap operations using the potential method. [class notes]
[Erickson's notes]
[Kaplan and Zwick's slides]
[CLRS] - Section 17.3
29 Oct 14, 10:00 Shortest paths in digraphs - optimal substructure property. Introduction to dynamic programming - characterizing the solution, writing the recurrence relation. Bellman-Ford recurrence for single-source shortest path. [class notes]
[E] - Section 8.7
30 Oct 15, 09:00 Bellman-Ford recurrence for single-source shortest path - expressing the algorithm in a bottom-up manner. Extending the idea to all-pairs shortest path problem. The Fischer-Meyer idea - a different recurrence and a faster algorithm. [class notes]
[E] - Section 8.7, 9.5, 9.6
31 Oct 16, 08:00 All pairs shortest path - Floyd-Warshall algorithm. Dynamic programming on strings and sequences - text justification. [class notes]
[E] - Section 9.8
32 Oct 18, 12:00 Dynamic programming on strings and sequences (contd.) - text justification and edit distance. [E] - Section 3.7
[KT] - Section 6.6
Q2 Oct 19, 09:30 Quiz 2.
33 Oct 22, 09:00 Review of Quiz 2. Dynamic programming on strings and sequences (contd.) - saving space for edit distance using divide and conquer. [E] - Section 3.7
[KT] - Section 6.6
34 Oct 23, 08:00 Dynamic programming on strings and sequences (contd.) - linear space algorithm for edit distance using divide and conquer. Dynamic programming on sets - subset sum and knapsack. Notion of peudo-polynomial time. [KT] - Section 6.4, 6.7
35 Oct 25, 12:00 Two variants of knapsack problem - pseudo-polynomial time algorithm. Independent set problem on restricted classes of graphs - paths and trees. [class notes]
[KT] - Section 6.4
[DPV] - Section 6.4
36 Oct 26, 09:30 Maxflow problem - greedy method, correcting wrong greedy choices using a residual graph. Augmenting paths to improve flows, examples. Ford-Fulkerson method - augmenting flows using the residual graph, examples. Proving that the Ford-Fulkerson method terminates with a feasible flow. [class notes]
[KT] - Section 7.1
37 Oct 28, 10:00 Correctness of the Ford-Fulkerson method. Notion of an s-t cut, existence of a cut with value of the maxflow, maxflow-mincut theorem, proof that an integral flow network contains an integral maxflow. [class notes]
[KT] - Section 7.2
38 Oct 29, 09:00 Example instance where Ford-Fulkerson algorithm takes exponential time to converge to the maxflow. Methods to improve the running time: augmenting via the fattest path - algorithm and the idea behind the analysis, augmenting via the path with the least number of edges. [Trevisan's notes]
39 Oct 30, 08:00 Edmonds-Karp/Dinitz algorithm - augmenting via the shortest path, brief overview of the running time bound. Application of the maxflow-mincut theorem and algorithm - maximum matching in bipartite graphs. [class notes]
[Trevisan's notes] - Section 2
40 Nov 1, 12:00 Applications of the maxflow-mincut theorem - König-Egerváry theorem, Hall's theorem characterizing perfect matchings in bipartite graphs. [class notes]
41 Nov 4, 10:00 Algorithmic questions that were considered during the course, and some of their variants. Connections between optimization problems and decision problems. Aiming for a theory of computational complexity. [class notes]
42 Nov 5, 09:00 Definition of the class NP - short witnesses that can be verified efficiently. Comparing the complexity of problems using polynomial-time reductions. Cook-Levin theorem (statement) and the concept of NP-completeness. [class notes]
43 Nov 6, 08:00 Examples of many-one reductions - 3SAT, Independent Set, Vertex Cover and Clique. [class notes]
SE3 Nov 8, 12:00 Short exam #3.
44 Nov 11, 10:00 Going beyond NP-completeness: Exact exponential algorithms and approximation algorithms. Wrap-up of the course.