I'm running into time and memory problems creating an array to store multiple
KSP objects.
We are using a Fourier method for the temporal portion of our operator;
currently, this is anywhere between 1024-2048 time samples equating to
1024-2048 frequencies to advance our solution. Each frequency has its own
matrix, meaning there are 1024-2048 linear equations of the form Ax = b. So,
while the matrices are different for each frequency, when we propagate the
solution one step, the 1024-2048 matrices don't change since the frequencies
haven't changed between steps.
We would like to store each KSP solution (all 1024-2048 of them) in an array so
that we can reuse them for the next step since this saves us time factoring the
matrix. The problem I am seeing is that storing these KSP objects in any kind
of array increases the runtime and storage space exponentially - I am seeing
times on the order of 30 mins to an hour and memory consumed hover around
20-30GB! Is there something inherent about the KSP object that I am missing
and/or a better way to store these objects? I don't have any problems if I use
one KSP object and set the operators each time to the different matrices,
except that it takes longer than it normally would after the first step since
it has to recalculate the preconditioner matrix.
Note: We are using distributed matrices and vectors for our linear system.
I have included some pseudocode to illustrate our linear system, which works:
KSP k;
for steps = 1 to 100 do:
for time = 1 to 1024 do:
create distributed matrix A and fill with local values
(A[step=1,time=1] = A[step=2,time=1], etc)
create distributed vector b and fill with local values for
given time
create distributed vector x for result
create KSP k and set matrices
KSPSolve(k, b, x) => takes a little extra time each step
However, the following creates time/storage problems:
KSP* kArray;
create kArray => takes inordinate amout of time and storage
for steps = 1 to 100 do:
for time=1 to 1024 do:
create distributed matrix A and fill with local values
(A[step=1,time=1] = A[step=2,time=1], etc)
create distributed vector b and fill with local values for
given time
create distributed vector x for result
if time == 1 do:
set matrices on kArray[time] => also takes an inordinate amount of
time and storage
KSPSolve(kArray[time], b, x)
Any insight would be greatly appreciated!
Brenna
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120517/9516df63/attachment-0001.htm>
