Column too large: col 10980 max 124
You need to check the code that is generating the matrix entries. The matrix has 124 columns but you are attempting to put a value at column 10980 Barry > On Aug 15, 2018, at 9:44 PM, Manuel Valera <[email protected]> wrote: > > Thanks Matthew and Barry, > > Now my code looks like: > > call DMSetMatrixPreallocateOnly(daDummy,PETSC_TRUE,ierr) > call DMSetMatType(daDummy,MATMPIAIJVIENNACL,ierr) > call DMSetVecType(daDummy,VECMPIVIENNACL,ierr) > call DMCreateMatrix(daDummy,A,ierr) > call MatSetFromOptions(A,ierr) > call MatSetUp(A,ierr) > [...] > call > MatSetValues(A,1,row,sumpos,pos(0:iter-1),vals(0:iter-1),INSERT_VALUES,ierr) > [...] > call MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY, ierr) > call MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY, ierr) > > And i get a different error, now is: > > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > [0]PETSC ERROR: Argument out of range > [0]PETSC ERROR: Column too large: col 10980 max 124 > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for > trouble shooting. > [0]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53 GIT > Date: 2018-05-31 17:31:13 +0300 > [0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera Wed Aug > 15 19:40:00 2018 > [0]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug > --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 > --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1 > --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64 --download-viennacl > [0]PETSC ERROR: #1 MatSetValues_SeqAIJ() line 442 in > /home/valera/petsc/src/mat/impls/aij/seq/aij.c > [0]PETSC ERROR: #2 MatSetValues() line 1339 in > /home/valera/petsc/src/mat/interface/matrix.c > > > Thanks again, > > > > > > > > > On Wed, Aug 15, 2018 at 7:02 PM, Smith, Barry F. <[email protected]> wrote: > > Should be > > call DMSetMatType(daDummy,MATMPIAIJVIENNACL,ierr) > call DMSetVecType(daDummy,VECMPIVIENNACL,ierr) > call DMCreateMatrix(daDummy,A,ierr) > > and remove the rest. You need to set the type of Mat you want the DM to > return BEFORE you create the matrix. > > Barry > > > > > On Aug 15, 2018, at 4:45 PM, Manuel Valera <[email protected]> wrote: > > > > Ok thanks for clarifying that, i wasn't sure if there were different types, > > > > Here is a stripped down version of my code, it seems like the preallocation > > is working now since the matrix population part is working without problem, > > but here it is for illustration purposes: > > > > call DMSetMatrixPreallocateOnly(daDummy,PETSC_TRUE,ierr) > > call DMCreateMatrix(daDummy,A,ierr) > > call MatSetFromOptions(A,ierr) > > call DMSetMatType(daDummy,MATMPIAIJVIENNACL,ierr) > > call DMSetVecType(daDummy,VECMPIVIENNACL,ierr) > > call > > MatMPIAIJSetPreallocation(A,19,PETSC_NULL_INTEGER,19,PETSC_NULL_INTEGER,ierr) > > call MatSetUp(A,ierr) > > [...] > > call > > MatSetValues(A,1,row,sumpos,pos(0:iter-1),vals(0:iter-1),INSERT_VALUES,ierr) > > [...] > > call MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY, ierr) > > call MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY, ierr) > > > > Adding the first line there did the trick, > > > > Now the problem seems to be the program is not recognizing the matrix as > > ViennaCL type when i try with more than one processor, i get now: > > > > [0]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > [0]PETSC ERROR: No support for this operation for this object type > > [0]PETSC ERROR: Currently only handles ViennaCL matrices > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for > > trouble shooting. > > [0]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53 GIT > > Date: 2018-05-31 17:31:13 +0300 > > [0]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera Wed Aug > > 15 14:44:22 2018 > > [0]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug > > --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 > > --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1 > > --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64 --download-viennacl > > [0]PETSC ERROR: #1 PCSetUp_SAVIENNACL() line 47 in > > /home/valera/petsc/src/ksp/pc/impls/saviennaclcuda/saviennacl.cu > > [0]PETSC ERROR: #2 PCSetUp() line 932 in > > /home/valera/petsc/src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: #3 KSPSetUp() line 381 in > > /home/valera/petsc/src/ksp/ksp/interface/itfunc.c > > > > When running with: > > > > mpirun -n 1 ./gcmLEP.GPU tc=TestCases/LockRelease/LE_6x6x6/ > > jid=tiny_cuda_test_n2 -ksp_type cg -dm_vec_type viennacl -dm_mat_type > > aijviennacl -pc_type saviennacl -log_view > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > On Wed, Aug 15, 2018 at 2:32 PM, Matthew Knepley <[email protected]> wrote: > > On Wed, Aug 15, 2018 at 5:20 PM Manuel Valera <[email protected]> wrote: > > It seems to be resumed on: I do not know how to preallocate a DM Matrix > > correctly. > > > > There is only one matrix type, Mat. There are no separate DM matrices. A DM > > can create a matrix for you > > using DMCreateMatrix(), but that is a Mat and it is preallocated correctly. > > I am not sure what you are doing. > > > > Thanks, > > > > Matt > > > > The interesting part is that it only breaks when i need to populate a GPU > > matrix from MPI, so kudos on that, but it seems i need to do better on my > > code to get this setup working, > > > > Any help would be appreciated, > > > > Thanks, > > > > > > > > On Wed, Aug 15, 2018 at 2:15 PM, Matthew Knepley <[email protected]> wrote: > > On Wed, Aug 15, 2018 at 4:53 PM Manuel Valera <[email protected]> wrote: > > Thanks Matthew, > > > > I try to do that when calling: > > > > call > > MatMPIAIJSetPreallocation(A,19,PETSC_NULL_INTEGER,19,PETSC_NULL_INTEGER,ierr) > > > > But i am not aware on how to do this for the DM if it needs something more > > specific/different, > > > > The error says that your preallocation is wrong for the values you are > > putting in. The DM does not control either, > > so I do not understand your email. > > > > Thanks, > > > > Matt > > > > Thanks, > > > > On Wed, Aug 15, 2018 at 1:51 PM, Matthew Knepley <[email protected]> wrote: > > On Wed, Aug 15, 2018 at 4:39 PM Manuel Valera <[email protected]> wrote: > > Hello PETSc devs, > > > > I am running into an error when trying to use the MATMPIAIJVIENNACL Matrix > > type in MPI calls, the same code runs for MATSEQAIJVIENNACL type in one > > processor. The error happens when calling MatSetValues for this specific > > configuration. It does not occur when using MPI DMMatrix types only. > > > > The DM properly preallocates the matrix. I am assuming you do not here. > > > > Matt > > > > Any help will be appreciated, > > > > Thanks, > > > > > > > > My program call: > > > > mpirun -n 2 ./gcmLEP.GPU tc=TestCases/LockRelease/LE_6x6x6/ > > jid=tiny_cuda_test_n2 -ksp_type cg -dm_vec_type viennacl -dm_mat_type > > aijviennacl -pc_type saviennacl -log_view > > > > > > The error (repeats after each MatSetValues call): > > > > [1]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > [1]PETSC ERROR: Argument out of range > > [1]PETSC ERROR: Inserting a new nonzero at global row/column (75, 50) into > > matrix > > [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for > > trouble shooting. > > [1]PETSC ERROR: Petsc Development GIT revision: v3.9.2-549-g779ab53 GIT > > Date: 2018-05-31 17:31:13 +0300 > > [1]PETSC ERROR: ./gcmLEP.GPU on a cuda-debug named node50 by valera Wed Aug > > 15 13:10:44 2018 > > [1]PETSC ERROR: Configure options PETSC_ARCH=cuda-debug > > --with-mpi-dir=/usr/lib64/openmpi --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 > > --FOPTFLAGS=-O2 --with-shared-libraries=1 --with-debugging=1 --with-cuda=1 > > --CUDAFLAGS=-arch=sm_60 --with-blaslapack-dir=/usr/lib64 --download-viennacl > > [1]PETSC ERROR: #1 MatSetValues_MPIAIJ() line 608 in > > /home/valera/petsc/src/mat/impls/aij/mpi/mpiaij.c > > [1]PETSC ERROR: #2 MatSetValues() line 1339 in > > /home/valera/petsc/src/mat/interface/matrix.c > > > > > > My Code structure: > > > > call DMCreateMatrix(daDummy,A,ierr) > > call MatSetFromOptions(A,ierr) > > call MPI_Comm_size(PETSC_COMM_WORLD, numprocs, ierr) > > if (numprocs > 1) then ! set matrix type parallel > > ! Get local size > > call DMDACreateNaturalVector(daDummy,Tmpnat,ierr) > > call VecGetLocalSize(Tmpnat,locsize,ierr) > > call VecDestroy(Tmpnat,ierr) > > ! Set matrix > > #ifdef GPU > > call MatSetType(A,MATAIJVIENNACL,ierr) > > call DMSetMatType(daDummy,MATMPIAIJVIENNACL,ierr) > > call DMSetVecType(daDummy,VECMPIVIENNACL,ierr) > > print*,'SETTING GPU TYPES' > > #else > > call DMSetMatType(daDummy,MATMPIAIJ,ierr) > > call DMSetMatType(daDummy,VECMPI,ierr) > > call MatSetType(A,MATMPIAIJ,ierr)! > > #endif > > call > > MatMPIAIJSetPreallocation(A,19,PETSC_NULL_INTEGER,19,PETSC_NULL_INTEGER,ierr) > > else ! set matrix type sequential > > #ifdef GPU > > call DMSetMatType(daDummy,MATSEQAIJVIENNACL,ierr) > > call DMSetVecType(daDummy,VECSEQVIENNACL,ierr) > > call MatSetType(A,MATSEQAIJVIENNACL,ierr) > > print*,'SETTING GPU TYPES' > > #else > > call DMSetMatType(daDummy,MATSEQAIJ,ierr) > > call DMSetMatType(daDummy,VECSEQ,ierr) > > call MatSetType(A,MATSEQAIJ,ierr) > > #endif > > call MatSetUp(A,ierr) > > call getCenterInfo(daGrid,xstart,ystart,zstart,xend,yend,zend) > > > > do k=zstart,zend-1 > > do j=ystart,yend-1 > > do i=xstart,xend-1 > > [..] > > call > > MatSetValues(A,1,row,sumpos,pos(0:iter-1),vals(0:iter-1),INSERT_VALUES,ierr) > > [..] > > > > > > > > > > > > > > -- > > What most experimenters take for granted before they begin their > > experiments is infinitely more interesting than any results to which their > > experiments lead. > > -- Norbert Wiener > > > > https://www.cse.buffalo.edu/~knepley/ > > > > > > > > -- > > What most experimenters take for granted before they begin their > > experiments is infinitely more interesting than any results to which their > > experiments lead. > > -- Norbert Wiener > > > > https://www.cse.buffalo.edu/~knepley/ > > > > > > > > -- > > What most experimenters take for granted before they begin their > > experiments is infinitely more interesting than any results to which their > > experiments lead. > > -- Norbert Wiener > > > > https://www.cse.buffalo.edu/~knepley/ > > > >
