Hi Karl, I now checked the error flags of all petsc functions in all functions. I also recompiled petsc with debug. Here's the report:
Case 1) Still works. All good. Case 2) This is the case with ComputeResidual using VecGetArrays but with DMSetVecType(da, VECVIENNACL) and DMSetMatType(da, MATAIJVIENNACL). The flags don't show any errors. *The programs proceeds smoothly but the solution is wrong even though SNES is converging.* Case 3) This is the case with ComputeResidualViennaCL using VecViennaCLGetArrays with and without DMSetVecType(da, VECVIENNACL) and DMSetMatType(da, MATAIJVIENNACL). In both cases I get the following error: [0]PETSC ERROR: TSComputeIFunction() line 676 in /home/mc/Downloads/petsc/src/ts/interface/ts.c [0]PETSC ERROR: SNESTSFormFunction_Theta() line 284 in /home/mc/Downloads/petsc/src/ts/impls/implicit/theta/theta.c [0]PETSC ERROR: SNESTSFormFunction() line 3499 in /home/mc/Downloads/petsc/src/ts/interface/ts.c [0]PETSC ERROR: SNESComputeFunction() line 2089 in /home/mc/Downloads/petsc/src/snes/interface/snes.c [0]PETSC ERROR: SNESSolve_NEWTONLS() line 175 in /home/mc/Downloads/petsc/src/snes/impls/ls/ls.c [0]PETSC ERROR: SNESSolve() line 3812 in /home/mc/Downloads/petsc/src/snes/interface/snes.c [0]PETSC ERROR: TSStep_Theta() line 183 in /home/mc/Downloads/petsc/src/ts/impls/implicit/theta/theta.c [0]PETSC ERROR: TSStep() line 2625 in /home/mc/Downloads/petsc/src/ts/interface/ts.c [0]PETSC ERROR: TSSolve() line 2741 in /home/mc/Downloads/petsc/src/ts/interface/ts.c [0]PETSC ERROR: main() line 83 in /home/mc/PhD/opencl_tests/petsc_opencl/petsc_opencl.cpp -------------------------------------------------------------------------- MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode -473550369. NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them. -------------------------------------------------------------------------- The error occured at VecViennaCLGetArrayRead/Write in my ComputeResidualViennaCL function. Note that with the petsc debug mode, the code crashes with the above error even if I don't catch the error codes. Might be because of some VecViennaCLGetArray calls inside petsc. Cheers, Mani On Sat, Jan 25, 2014 at 2:48 AM, Karl Rupp <[email protected]> wrote: > Hi Mani, > > please check the return value of *all* function calls from PETSc, e.g. > ierr = DMCreateGlobalVector(da, &soln);CHKERRQ(ierr); > instead of just > DMCreateGlobalVector(da, &soln); > Most likely one of the routines threw an error, but your code just kept > going, producing wrong results. > > Best regards, > Karli > > > > On 01/25/2014 05:35 AM, Mani Chandra wrote: > >> Hi Everyone, >> >> I'm trying to use TS with ViennaCL vecs/mats and residual evaluation on >> device and have encountered some problems. I have attached a small test >> code that illustrates the issue. >> >> The code simply advects a blob diagonally using TS. I have written the >> residual evaluation function using 1) the usual Petsc vectors >> (VecGetArray) and 2) using ViennaCL vectors (VecViennaCLGetArrayRead/ >> Write). >> >> Run the code using the following: >> ./petsc_opencl -ts_monitor -snes_monitor -ts_max_steps 1000 -ts_type >> theta -ts_dt 10 -snes_rtol 1e-4 -ts_final_time 1000 >> -ts_monitor_draw_solution >> >> Case 1) No ViennaCL anywhere. I simply use the usual Petsc vectors and >> set the residual evaluation function as ComputeResidual (line no. 55). >> This case works and the blob is indeed advected as can be seen. (I >> haven't bothered with the boundaries. The simulation just stops before >> the blob hits the boundaries). >> >> Case 2) We again use the ComputeResidual but now enable ViennaCL vecs >> and mats (line nos. 48, 49). This case does NOT work. The SNES monitor >> shows convergence but the solution makes no sense. >> >> Case 3) We now use ComputeResidualViennaCL (line no. 56). This does NOT >> work either with or without enabling the ViennaCL vecs (line nos. 48, 49). >> >> Cheers, >> Mani >> > >
