I actually was able to solve my own problem...for some reason, I need to do
PCSetType(pc, PCLU); PCFactorSetMatSolverPackage(pc, MATSOLVERSUPERLU_DIST); KSPSetTolerances(ksp, 1.e-15, PETSC_DEFAULT, PETSC_DEFAULT, PETSC_DEFAULT); instead of the ordering I initially had, though I'm not really clear on what the issue was. However, there seems to be some loss of accuracy as I increase the number of processes. Is this expected, or can I force a lower tolerance somehow? I am able to compare the solutions to a reference solution, and the error increases as I increase the processes. This is the solution in sequential: superlu_1process = [ -6.8035811950925553e-06 1.6324030474375778e-04 5.4145340579614926e-02 1.6640521936281516e-04 -1.7669374392923965e-04 -2.8099208957838207e-04 5.3958133511222223e-02 -5.4077899123806263e-02 -5.3972905090366369e-02 -1.9485020474821160e-04 5.4239813043824400e-02 4.4883984259948430e-04]; superlu_2process = [ -6.8035811950509821e-06 1.6324030474371623e-04 5.4145340579605655e-02 1.6640521936281687e-04 -1.7669374392923807e-04 -2.8099208957839834e-04 5.3958133511212911e-02 -5.4077899123796964e-02 -5.3972905090357078e-02 -1.9485020474824480e-04 5.4239813043815172e-02 4.4883984259953320e-04]; superlu_4process= [ -6.8035811952565206e-06 1.6324030474386164e-04 5.4145340579691455e-02 1.6640521936278326e-04 -1.7669374392921441e-04 -2.8099208957829171e-04 5.3958133511299078e-02 -5.4077899123883062e-02 -5.3972905090443085e-02 -1.9485020474806352e-04 5.4239813043900860e-02 4.4883984259921287e-04]; This is some finite element solution and I can compute the error of the solution against an exact solution in the functional L2 norm: error with 1 process: 1.71340e-02 (accepted value) error with 2 processes: 2.65018e-02 error with 3 processes: 3.00164e-02 error with 4 processes: 3.14544e-02 Is there a way to remedy this? On Wed, Apr 30, 2014 at 2:37 AM, Justin Dong <j...@rice.edu> wrote: > Hi, > > I'm trying to solve a linear system in parallel using SuperLU but for some > reason, it's not giving me the correct solution. I'm testing on a small > example so I can compare the sequential and parallel cases manually. I'm > absolutely sure that my routine for generating the matrix and right-hand > side in parallel is working correctly. > > Running with 1 process and PCLU gives the correct solution. Running with 2 > processes and using SUPERLU_DIST does not give the correct solution (I > tried with 1 process too but according to the superlu documentation, I > would need SUPERLU for sequential?). This is the code for solving the > system: > > /* solve the system */ > KSPCreate(PETSC_COMM_WORLD, &ksp); > KSPSetOperators(ksp, Aglobal, Aglobal, DIFFERENT_NONZERO_PATTERN); > KSPSetType(ksp,KSPPREONLY); > > KSPGetPC(ksp, &pc); > > KSPSetTolerances(ksp, 1.e-13, PETSC_DEFAULT, PETSC_DEFAULT, PETSC_DEFAULT); > PCFactorSetMatSolverPackage(pc, MATSOLVERSUPERLU_DIST); > > KSPSolve(ksp, bglobal, bglobal); > > Sincerely, > Justin > > >
