CS660: Intro to Analysis

CS660 Combinatorial Algorithms
Fall Semester, 1996
Intro to Analysis

[To Lecture Notes Index]
San Diego State University -- This page last updated Monday, 02 September, 1996

Contents of Intro to Analysis

                                          Intro to Analysis Slide # 1

References

Introduction To Algorithms, Corman, Leiserson,Rivest, Chapters 1-4

Reading

Introduction To Algorithms, Corman, Leiserson,Rivest, Chapters 1-4 by Tuesday Sept. 10

                                          Intro to Analysis Slide # 2

Mathematical Analysis of Algorithms

                                          Intro to Analysis Slide # 3

Model of Computing

If analysis of algorithms is the answer, what is the question?

Given two or more algorithms for the same task, which is better?

Under which condition is bubble sort better than insertion sort?

What computing resources does an algorithm require?

How long will it take bubble sort to sort a list of N items?

Goal of mathematical analysis is a function of the resources required of an algorithm

On what computer?

                                          Intro to Analysis Slide # 4

What is a Computer? Random-access machine (RAM)

Single processor
Instructions executed sequentially
Each operation requires the same amount of time
"Standard" set of operations

Single cost vs. Lg(N) cost

Time required for basic operation?

3 + 6: 1234!

                                          Intro to Analysis Slide # 5

What is an Algorithm?
A finite set of rules which gives a sequence of operations for solving a specific type of problem.

The set of rules must have the following features.

1) Finiteness

Must terminate after a finite number of steps

2) Definiteness

Each step of an algorithm must be precisely defined
No ambiguous statements allowed

3) Effectiveness

A human must be able to perform each step using paper and pencil in a finite amount of time.

4) Input

Each algorithm has some input.

5) Output

Each algorithm has some output.

                                          Intro to Analysis Slide # 6

Insertion Sort

	Cost	times
A[0] = - infinity		1
for K = 2 to N do		N
begin	0	N-1
J = K;		N-1
Key = A[J]		N-1
while Key < A[J-1] do
begin	0
A[J] = A[J-1];
J = J - 1;
end while;
A[J] = Key;		N-1
end for;		N-1

                                          Intro to Analysis Slide # 7

T(N) = cost of running Insertion Sort

where

What is

                                          Intro to Analysis Slide # 8

Summations

                                          Intro to Analysis Slide # 9

Asymptotic Notation

Asymptotically tight bound - Theta

Asymptotic upper boundsBig-O

Little o

                                          Intro to Analysis Slide # 10

Asymptotic lower bounds Omega

Little Omega

                                          Intro to Analysis Slide # 11

Common Myths and Errors

Everyone incorrectly writes:

instead of:

or does not mean

or even that there is an n such that

Let f(n) = 2n + 10, and g(n) = n then

f(n) = O(g(n)) but f(n) > g(n)

Using O( ) when mean

                                          Intro to Analysis Slide # 12

Terms
Complexity

Resources required by the algorithm as a function of the input size

Worst-case Analysis

Complexity of an algorithm based on worst input of each size

Average-case Analysis

Complexity of an algorithm averaged over all inputs of each size

Insertion Sort

	Comparisons	Element moves

worst case	(N+1)N/2 - 1	(N-1)N/2
average case	(N+1)N/4 - 1/2	(N-1)N/4

                                          Intro to Analysis Slide # 13

Recurrences

Quicksort

Input A - array of keys to be sorted
L - index of left most item in array to be sorted
R - index of right most item in array to be sorted

Output A with A[L], A[L+1], ...,A[R] in order

Quicksort(A: array[1..N]; L,R:integer)

   if L < R then 
      begin
      Mid := partition(L, R);
      Quicksort(A,L,Mid-1);
      Quicksort(A,Mid+1,R);
      end;

                                          Intro to Analysis Slide # 14

Partition procedure
Input A - array to be split
Left - index of left most element to be split
Right - index of right most element to be split


partition(A,Left,Right)

begin
   L := Left + 1;
   R := Right;
   Key := A[Left];

   repeat
      repeat L = L + 1 until A[L] >= Key;
      repeat R = R - 1 until A[R] <= Key;
      swap(A[L],A[R]);
   until  R <= L
   
   A[Left] := A[L];
   A[L] := A[R];
   A[R] := Key;
   return R;
end

                                          Intro to Analysis Slide # 15

Recurences for Quicksort
Let Q(N) = time complexity of quicksort on input of size N

How to solve for Q(N)?
Methods for solving recurrences
Iteration
Substitution
Master theorem

                                          Intro to Analysis Slide # 16

Master theorem
Theorem 4.1

Let a>= 1 and B > 1 be constants. Let f(n) be a function, and let T(n) be defined by

T(n) = aT(n/b) + f(n)

where n/b may be either

. Then:

if for some constant > 0 then

if then

if for some constant and if a*f(n/b)<= c*f(n) for some constant c < 1 and all large n, then

Intro to Analysis Slide # 17
Bubble vs. Insertion Sort

	Worst case	Average case
Bubble sort
Insertion Sort

Bubble Sort

	Comparisons	Element moves
worst case	(N-1)N/2	3(N-1)N/2
average case	(N-1)N/2	3(N-1)N/4
best case	(N-1)N/2	0

Insertion Sort

	Comparisons	Element moves
worst case	(N+1)N/2 - 1	(N-1)N/2
average case	(N+1)N/4 - 1/2	(N-1)N/4
best case	N - 1	0

                                          Intro to Analysis Slide # 18

Bubble vs. Insertion Sort Timing Results Worst Case

N	Bubble	Insertion
100	1	1
200	5	3
400	19	11
800	79	42
1600	317	166

Average Case

N	Bubble	Insertion
100	1	0
200	3	1
400	14	5
800	56	21
1600	228	84

What is wrong with this Picture?

CS660 Combinatorial Algorithms Fall Semester, 1996 Intro to Analysis

Contents of Intro to Analysis

References

Mathematical Analysis of Algorithms

Model of Computing

Asymptotic Notation

Recurrences

CS660 Combinatorial Algorithms
Fall Semester, 1996
Intro to Analysis