There are a bunch of nightlybuild regressions with the GPU stuff - perhaps you can take a look.
http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2017/07/27/examples_next_arch-cuda-double_es.log Satish On Thu, 27 Jul 2017, Karl Rupp wrote: > Hi Jose, > > > We have detected another problem. Could you take care of it? > > MatDuplicate() does not work for MATSEQAIJCUSPARSE (probably also for > > MATMPIAIJCUSPARSE). > > The attached example creates a matrix and duplicates it. There are two > > cases: > > > > 1) With a diagonal matrix it fails on GPU because MatDuplicate() did not > > copy the CUDA-specific data. > > > > $ ./ex_duplicate -diag -mat_type aijcusparse -vec_type cuda > > > > 2) With a fully dense matrix (or any matrix where I-node routines are used), > > it does not fail but operations are done on CPU instead of GPU (because it > > changes the pointers to MatMult_SeqAIJ_Inode etc). > > > > $ ./ex_duplicate -mat_type aijcusparse -vec_type cuda > > ah, thanks for finding this and for providing code that reproduces the > problem. I'll take care of it. > > Best regards, > Karli > > > > > > > > > El 26 jul 2017, a las 21:12, Karl Rupp <[email protected]> escribió: > > > > > > Hi Jose, > > > > > > > With pull request #719 we have finished a set of fixes to VECCUDA stuff. > > > > With these changes it is now possible to run many tests in SLEPc's > > > > testsuite on GPU (AIJCUSPARSE+VECCUDA). These tests will be included in > > > > the nightly tests from now on. > > > > > > great! > > > > > > > However, PETSc nightly tests related to VECCUDA are not being run. The > > > > reason is that arch-cuda-double.py and arch-cuda-double.py have > > > > --with-cusp=1 and this option disables VECCUDA code. CUSP tests are > > > > separate from VECCUDA tests. > > > > > > As you may remember, I want to get rid of VECCUSP (and if possible also > > > MATAIJCUSP), because the functionality is now provided natively through > > > the CUDA SDK (VECCUDA, MATAIJCUSPARSE). Only the preconditioners from > > > CUSP, most notably PCSACUSP, will stay. This way we can then easily switch > > > over all the tests to VECCUDA. > > > > > > > > > > Another thing is that not all VECCUDA tests pass, because of a pending > > > > issue related to MatMultTranspose_MPIAIJCUSPARSE. This was reported last > > > > year: > > > > https://bitbucket.org/petsc/petsc/pull-requests/490/gpu-regression-tests > > > > > > Alright, thanks for the reminder. Let me get this fixed. :-) > > > > > > Best regards, > > > Karli >
