CS660 Combinatorial Algorithms
Fall Semester, 1996
Intro to Analysis
[To Lecture Notes Index]
San Diego State University -- This page last updated Monday, 02 September, 1996

Contents of Intro to Analysis
- References
- Mathematical Analysis of Algorithms
- Model of Computing
- Asymptotic Notation
- Recurrences
Intro to Analysis Slide # 1
Introduction To Algorithms, Corman, Leiserson,Rivest, Chapters 1-4
Reading
Introduction To Algorithms, Corman, Leiserson,Rivest, Chapters 1-4 by
Tuesday Sept. 10
Intro to Analysis Slide # 2
Intro to Analysis Slide # 3
If analysis of algorithms is the answer, what is the question?
Given two or more algorithms for the same task, which is better?
- Under which condition is bubble sort better than insertion sort?
What computing resources does an algorithm require?
- How long will it take bubble sort to sort a list of N items?
Goal of mathematical analysis is a function of the resources required of an
algorithm
On what computer?
Intro to Analysis Slide # 4
What is a Computer?
Random-access machine (RAM)
- Single processor
- Instructions executed sequentially
- Each operation requires the same amount of time
- "Standard" set of operations
Single cost vs. Lg(N) cost
- Time required for basic operation?
-
- 3 + 6
- 1234!
Intro to Analysis Slide # 5
What is an Algorithm?
A finite set of rules which gives a sequence of operations for solving a
specific type of problem.
The set of rules must have the following features.
1) Finiteness
- Must terminate after a finite number of steps
2) Definiteness
- Each step of an algorithm must be precisely defined
- No ambiguous statements allowed
3) Effectiveness
- A human must be able to perform each step using paper and pencil in a
finite amount of time.
4) Input
- Each algorithm has some input.
5) Output
- Each algorithm has some output.
Intro to Analysis Slide # 6
Insertion Sort
| Cost | times |
A[0] = - infinity |  | 1 |
for K = 2 to N do |  | N |
begin | 0 | N-1 | |
J = K; |  | N-1 |
Key = A[J] |  | N-1 |
while Key < A[J-1] do |  |  |
begin | 0 |  |
A[J] = A[J-1]; |  |  |
J = J - 1; |  |  |
end while; |  |  |
A[J] = Key; |  | N-1 |
end for; |  | N-1 |
Intro to Analysis Slide # 7
T(N) = cost of running Insertion Sort

where
-

What is
?
Intro to Analysis Slide # 8
Summations
Intro to Analysis Slide # 9
Asymptotically tight bound - Theta
-

-

Asymptotic upper boundsBig-O
-

-

Little o
-

-

Intro to Analysis Slide # 10
Asymptotic lower bounds
Omega
-

-

Little Omega
-

-

Intro to Analysis Slide # 11
Common Myths and Errors
- Everyone incorrectly writes:
-
- instead of:
-
-
- or even that there is an n such that

- Let f(n) = 2n + 10, and g(n) = n then
-
- f(n) = O(g(n)) but f(n) > g(n)
- Using O( ) when mean

Intro to Analysis Slide # 12
Terms
Complexity
- Resources required by the algorithm as a function of the input
size
Worst-case Analysis
- Complexity of an algorithm based on worst input of each size
Average-case Analysis
- Complexity of an algorithm averaged over all inputs of each size
-
Insertion Sort
| Comparisons | Element moves |
| | |
worst case | (N+1)N/2 - 1 | (N-1)N/2 |
average case | (N+1)N/4 - 1/2 | (N-1)N/4 |
Intro to Analysis Slide # 13
Quicksort
Input A - array of keys to be sorted
L - index of left most item in array to be sorted
R - index of right most item in array to be sorted
Output A with A[L], A[L+1], ...,A[R] in order
Quicksort(A: array[1..N]; L,R:integer)
if L < R then
begin
Mid := partition(L, R);
Quicksort(A,L,Mid-1);
Quicksort(A,Mid+1,R);
end;
Intro to Analysis Slide # 14
Partition procedure
Input A - array to be split
Left - index of left most element to be split
Right - index of right most element to be split
partition(A,Left,Right)
begin
L := Left + 1;
R := Right;
Key := A[Left];
repeat
repeat L = L + 1 until A[L] >= Key;
repeat R = R - 1 until A[R] <= Key;
swap(A[L],A[R]);
until R <= L
A[Left] := A[L];
A[L] := A[R];
A[R] := Key;
return R;
end
Intro to Analysis Slide # 15
Recurences for Quicksort
Let Q(N) = time complexity of quicksort on input of size N

How to solve for Q(N)?
Methods for solving recurrences
Iteration
Substitution
Master theorem
Intro to Analysis Slide # 16
Master theorem
Theorem 4.1
Let a>= 1 and B > 1 be constants. Let f(n) be a function, and let T(n) be
defined by
- T(n) = aT(n/b) + f(n)
where n/b may be either
or
.
Then:
- if
for some constant
> 0 then
-

- if
then
-

- if
for some constant
and if a*f(n/b)<= c*f(n) for some constant c < 1 and all large n, then
-

Intro to Analysis Slide # 17
Bubble vs. Insertion Sort
| Worst case | Average case |
Bubble sort |  |  |
Insertion Sort |  |  |
Bubble Sort
| Comparisons | Element moves |
worst case | (N-1)N/2 | 3(N-1)N/2 |
average case | (N-1)N/2 | 3(N-1)N/4 |
best case | (N-1)N/2 | 0 |
Insertion Sort
| Comparisons | Element moves |
worst case | (N+1)N/2 - 1 | (N-1)N/2 |
average case | (N+1)N/4 - 1/2 | (N-1)N/4 |
best case | N - 1 | 0 |
Intro to Analysis Slide # 18
Bubble vs. Insertion Sort
Timing Results
Worst Case
N | Bubble | Insertion |
100 | 1 | 1 |
200 | 5 | 3 |
400 | 19 | 11 |
800 | 79 | 42 |
1600 | 317 | 166 |
Average Case
N | Bubble | Insertion |
100 | 1 | 0 |
200 | 3 | 1 |
400 | 14 | 5 |
800 | 56 | 21 |
1600 | 228 | 84 |
What is wrong with this Picture?