Hi Karl, Thanks for looking into it. Do let me know if there is anything I can do to help you debug this. Attached is the code.
Cheers, Mani On Sat, Jan 25, 2014 at 3:14 PM, Karl Rupp <[email protected]> wrote: > Hi Mani, > > could you please send me the code including the error checks? > > Thanks and best regards, > Karli > > > > On 01/25/2014 10:07 PM, Mani Chandra wrote: > >> Hi Karl, >> >> I now checked the error flags of all petsc functions in all functions. I >> also recompiled petsc with debug. Here's the report: >> >> Case 1) Still works. All good. >> >> Case 2) This is the case with ComputeResidual using VecGetArrays but >> with DMSetVecType(da, VECVIENNACL) and DMSetMatType(da, MATAIJVIENNACL). >> The flags don't show any errors. *The programs proceeds smoothly but the >> solution is wrong even though SNES is converging.* >> >> >> Case 3) This is the case with ComputeResidualViennaCL using >> VecViennaCLGetArrays with and without DMSetVecType(da, VECVIENNACL) and >> DMSetMatType(da, MATAIJVIENNACL). In both cases I get the following error: >> >> [0]PETSC ERROR: TSComputeIFunction() line 676 in >> /home/mc/Downloads/petsc/src/ts/interface/ts.c >> [0]PETSC ERROR: SNESTSFormFunction_Theta() line 284 in >> /home/mc/Downloads/petsc/src/ts/impls/implicit/theta/theta.c >> [0]PETSC ERROR: SNESTSFormFunction() line 3499 in >> /home/mc/Downloads/petsc/src/ts/interface/ts.c >> [0]PETSC ERROR: SNESComputeFunction() line 2089 in >> /home/mc/Downloads/petsc/src/snes/interface/snes.c >> [0]PETSC ERROR: SNESSolve_NEWTONLS() line 175 in >> /home/mc/Downloads/petsc/src/snes/impls/ls/ls.c >> [0]PETSC ERROR: SNESSolve() line 3812 in >> /home/mc/Downloads/petsc/src/snes/interface/snes.c >> [0]PETSC ERROR: TSStep_Theta() line 183 in >> /home/mc/Downloads/petsc/src/ts/impls/implicit/theta/theta.c >> [0]PETSC ERROR: TSStep() line 2625 in >> /home/mc/Downloads/petsc/src/ts/interface/ts.c >> [0]PETSC ERROR: TSSolve() line 2741 in >> /home/mc/Downloads/petsc/src/ts/interface/ts.c >> [0]PETSC ERROR: main() line 83 in >> /home/mc/PhD/opencl_tests/petsc_opencl/petsc_opencl.cpp >> ------------------------------------------------------------ >> -------------- >> MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD >> with errorcode -473550369. >> >> NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. >> You may or may not see output from other processes, depending on >> exactly when Open MPI kills them. >> ------------------------------------------------------------ >> -------------- >> >> The error occured at VecViennaCLGetArrayRead/Write in my >> ComputeResidualViennaCL function. Note that with the petsc debug mode, >> the code crashes with the above error even if I don't catch the error >> codes. Might be because of some VecViennaCLGetArray calls inside petsc. >> >> Cheers, >> Mani >> >> >> On Sat, Jan 25, 2014 at 2:48 AM, Karl Rupp <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi Mani, >> >> please check the return value of *all* function calls from PETSc, e.g. >> ierr = DMCreateGlobalVector(da, &soln);CHKERRQ(ierr); >> instead of just >> DMCreateGlobalVector(da, &soln); >> Most likely one of the routines threw an error, but your code just >> kept going, producing wrong results. >> >> Best regards, >> Karli >> >> >> >> On 01/25/2014 05:35 AM, Mani Chandra wrote: >> >> Hi Everyone, >> >> I'm trying to use TS with ViennaCL vecs/mats and residual >> evaluation on >> device and have encountered some problems. I have attached a >> small test >> code that illustrates the issue. >> >> The code simply advects a blob diagonally using TS. I have >> written the >> residual evaluation function using 1) the usual Petsc vectors >> (VecGetArray) and 2) using ViennaCL vectors >> (VecViennaCLGetArrayRead/__Write). >> >> >> Run the code using the following: >> ./petsc_opencl -ts_monitor -snes_monitor -ts_max_steps 1000 >> -ts_type >> theta -ts_dt 10 -snes_rtol 1e-4 -ts_final_time 1000 >> -ts_monitor_draw_solution >> >> Case 1) No ViennaCL anywhere. I simply use the usual Petsc >> vectors and >> set the residual evaluation function as ComputeResidual (line >> no. 55). >> This case works and the blob is indeed advected as can be seen. (I >> haven't bothered with the boundaries. The simulation just stops >> before >> the blob hits the boundaries). >> >> Case 2) We again use the ComputeResidual but now enable ViennaCL >> vecs >> and mats (line nos. 48, 49). This case does NOT work. The SNES >> monitor >> shows convergence but the solution makes no sense. >> >> Case 3) We now use ComputeResidualViennaCL (line no. 56). This >> does NOT >> work either with or without enabling the ViennaCL vecs (line >> nos. 48, 49). >> >> Cheers, >> Mani >> >> >> >> >
petsc_opencl.tar.gz
Description: GNU Zip compressed data
