Hi Pierre, > I try running in parallel the ex19 test case on CPU and GPU:
OPT="_opt" dir=/ccc/cont002/home/den/triou/Version_test_airain-hybrid/Trio_U/lib/src/LIBPETSC/petsc/linux$OPT/src/snes/examples/tutorials option="-pc_type none -ksp_type fgmres -snes_monitor_short -snes_rtol 1.e-5 -log_summary -ksp_view -cuda_show_devices" mpirun -np 2 $dir/ex19 $option 1>cpu$OPT.log 2>&1 mpirun -np 2 $dir/ex19 -dm_vec_type cusp -dm_mat_type aijcusp $option 1>gpu$OPT.log 2>&1 With OPT="", PETSc optimized library is used, parallel calculation runs well on CPU and GPU. With OPT="_opt", PETSc non optimized library is used, parallel calculation crashes on GPU (it is OK on CPU). I join the log files. The only difference seems that PETSc-dev is built with -O3 intead of -g... I could try to rebuild PETSc with -O2 but do you have any idea of the problem ?
I could reproduce the problem and also get some uninitialized variable warnings in Valgrind. The debug version detects these errors, hence you only see the errors in the debug build. For the optimized build, chances are good that the computed values are either wrong or may become wrong in other environments. I'll see what I can do when I'm again at GPU machine tomorrow (parallel GPU debugging via SSH is not great...)
Best regards, Karli
