CS 662 Theory of Parallel Algorithms
Numerical Methods part 2
[To Lecture Notes Index]
San Diego State University -- This page last updated April 30, 1996
![----------](http://www.sdsu.edu/graphics/divider_red.gif)
Contents of Numerical Methods part 2 Lecture
- Linear Equations
Matrix Matrix Multiplication and Scan
Let A and B be a n*n matrices
Compute C = A*B
-
![](numerical26.GIF)
We have
-
![](numerical27.GIF)
Scan can compute
in log(n) time using n/2 processors
Since there are
elements of C
Scan takes log(n) time with
processors to compute C
Slowdown
Let A, B, and C be a n*n matrices such that
- C = A*B
Let
be (n/2)*(n/2) matrices such that
-
and
![](numerical213.GIF)
Then
-
![](numerical214.GIF)
Strassen's Method
Let A, B, and C be a 2*2 matrices such that
- C = A*B
-
-
,
,
and
Define the sums and products:
Then we have:
![](numerical235.GIF)
Using 7 multiplications and 15 additions
On n*n matrices the above method take O(
)
operations
Solving a system of linear equations like:
-
![](numerical237.GIF)
can be reduced to solving Ax = b for x where:
-
-
![](numerical240.GIF)
Gauss-Jordan Method
for j = 1 to n do
for h = 1 to n do
for k = j to n + 1 do
if (h != j) then
end for
end for
end for
for j = 1 to n do
end for
Gauss-Jordan Method - SIMD
for j = 1 to n do
for h = 1 to n do in parallel
for k = j to n + 1 do in parallel
if (h != j) then
end for
end for
end for
for j = 1 to n do in parallel
end for
T(n) = O(n)
P(n) = O(
)
Another Approach
If Ax = b then x =
![](numerical246.GIF)
So just compute
![](numerical247.GIF)
Special Case
Let
![](numerical248.GIF)
Define
![](numerical249.GIF)
We have:
- 1)
-
- 2)
-
- 3) if A = BC then
-
- 4)
![](numerical253.GIF)
So how long does it take to compute
?
Note that A(BC) = (AB)C when A, B, C are n*n matrices
So we can use generalized scan operator with matrix multiplication
Sequential matrix multiplication takes
![](numerical255.GIF)
So with n processors we can compute
in
log(n)
time
But using a scan within a scan to perform all matrix multiplications we can
compute
in
time using
processors
Theorem. L. Csanky 1976
- The A be a nonsingular n*n matrix. Then
can be computed in parallel using
processors in
time.
proof.
- Uses some cool math to do some fancy stuff
-
- Details available on demand
-
- Method is not practical due to number of processors required and numerical
problems
It is an open problem if
time is best possible for computing inverse of an n*n matrix
It is an open problem if
time is best possible for triangular matrices
x =
General Case
Let
where
are n/2*n/2 matrices
We get:
-
![](numerical269.GIF)
So
-
![](numerical270.GIF)
Compute
and
recursively
This requires O(
)
sequential time, where 2 < x < 2.5