Send a data file you generated and your reader program and we'll debug it.
Barry > On Apr 1, 2015, at 2:18 PM, Steena M <[email protected]> wrote: > > Thanks Barry. I removed the Preallocation calls. It is still complaining > about the malloc and incorrect data in the matrix file. I generate binary > matrix files using PETSc's pythonscript to loop through a set of UFL sparse > matrices. For this use case: > > mtx_mat = scipy.io.mmread('trefethen.mtx') > PetscBinaryIO.PetscBinaryIO().writeMatSciPy(open('trefnew.dat','w'), mtx_mat) > > > > > > > On Tuesday, March 31, 2015 9:15 PM, Barry Smith <[email protected]> wrote: > > > > You should not need to call any preallocation routines when using MatLoad() > > > How did you generate the file? Are you sure it has the correct information > for the matrix? > > Barry > > > > > On Mar 31, 2015, at 11:05 PM, Steena M <[email protected]> wrote: > > > > Thanks Matt. I'm still getting the malloc error > > > > [0]PETSC ERROR: Argument out of range! > > [0]PETSC ERROR: New nonzero at (2,18) caused a malloc! > > > > and > > > > a new incorrect matrix file error: > > > > [0]PETSC ERROR: Unexpected data in file! > > [0]PETSC ERROR: not matrix object! > > > > Maybe the order of calls is mixed up. This is the code snippet: > > > > if (rank ==0) > > { > > PetscPrintf (PETSC_COMM_WORLD,"\n On rank %d ", rank); > > > > CHKERRQ(MatSetSizes(A, 15, PETSC_DETERMINE, 20, 20)); > > CHKERRQ(MatSetType(A, MATMPIBAIJ)); > > CHKERRQ( MatMPIBAIJSetPreallocation(A,1,1,NULL,1,NULL)); > > CHKERRQ( MatLoad(A,fd)); > > CHKERRQ(MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE)); > > } > > > > else > > { > > PetscPrintf (PETSC_COMM_WORLD,"\n On rank %d ", rank); > > > > CHKERRQ( MatSetSizes(A, 5, PETSC_DETERMINE, 20, 20) ); > > CHKERRQ(MatSetType(A, MATMPIBAIJ)); > > CHKERRQ( MatMPIBAIJSetPreallocation(A,1,1,NULL,1,NULL)); > > CHKERRQ(MatLoad(A,fd)); > > CHKERRQ(MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_FALSE)); > > } > > > > Is there something I'm missing? > > > > Thanks, > > Steena > > > > > > > > > > > > On Tuesday, March 31, 2015 6:10 PM, Matthew Knepley <[email protected]> > > wrote: > > > > > > On Tue, Mar 31, 2015 at 6:51 PM, Steena M <[email protected]> wrote: > > Thanks Barry. I'm still getting the malloc error with NULL. Is there a way > > to distribute the matrix without explicit preallocation? Different matrices > > will be loaded during runtime and assigning preallocation parameters would > > mean an additional preprocessing step. > > > > 1) MatSetOption(MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) > > > > 2) Note that this is never ever ever more efficient than making another > > pass and preallocating > > > > Thanks, > > > > Matt > > -------------------------------------------- > > On Sun, 3/29/15, Barry Smith <[email protected]> wrote: > > > > Subject: Re: [petsc-users] Unequal sparse matrix row distribution for MPI > > MatMult > > To: "Steena M" <[email protected]> > > Cc: "Matthew Knepley" <[email protected]>, [email protected] > > Date: Sunday, March 29, 2015, 9:26 PM > > > > > > > On > > Mar 29, 2015, at 11:05 PM, Steena M <[email protected]> > > wrote: > > > > > > Thanks > > Matt. I used PETSC_DETERMINE but I'm now getting an > > allocation-based error: > > > > > > [0]PETSC ERROR: --------------------- > > Error Message ------------------------------------ > > > [0]PETSC ERROR: Argument out of range! > > > [0]PETSC ERROR: New nonzero at (2,18) > > caused a malloc! > > > [0]PETSC ERROR: > > ------------------------------------------------------------------------ > > > > > > I tried > > preallocating on each rank for the diagonal and off diagonal > > section of the matrix as the next step My current > > approximations for preallocation > > > > > > CHKERRQ( > > MatMPIBAIJSetPreallocation(A,1,5,PETSC_DEFAULT,5,PETSC_DEFAULT)); > > > > > > These > > arguments where you pass PETSC_DEFAULT are expecting a > > pointer not an integer. You can pass NULL in those > > locations. Though it is better to provide the correct > > preallocation rather than some defaults. > > > > Barry > > > > > > > > are throwing segmentation errors. > > > > > > [0]PETSC ERROR: > > Caught signal number 11 SEGV: Segmentation Violation, > > probably memory access out of range > > > > > > Any insights into what I'm doing > > wrong? > > > > > > Thanks, > > > Steena > > > > > > > > > > > > On Sun, 3/29/15, Matthew Knepley <[email protected]> > > wrote: > > > > > > Subject: > > Re: [petsc-users] Unequal sparse matrix row distribution for > > MPI MatMult > > > To: "Steena M" > > <[email protected]> > > > Cc: "Barry Smith" <[email protected]>, > > [email protected] > > > Date: Sunday, March 29, 2015, 10:02 PM > > > > > > On Sun, Mar 29, 2015 > > at > > > 9:56 PM, Steena M <[email protected]> > > > wrote: > > > Hi > > > Barry, > > > > > > > > > > > > I am trying to partition a 20 row and 20 > > col sparse matrix > > > between two procs > > such that proc 0 has 15 rows and 20 cols > > > and proc 1 has 5 rows and 20 cols. The > > code snippet: > > > > > > > > > > > > > > > > > > > > > > > > > > CHKERRQ(MatCreate(PETSC_COMM_WORLD,&A)); > > // > > > at runtime: -matload_block_size 1 > > > > > > > > > > > > > > > > > > > > if (rank ==0) > > > > > > { > > > > > > > > CHKERRQ( MatSetSizes(A, 15, 20, 20, > > > 20) ); //rank 0 gets 75% of the rows > > > > > > > > CHKERRQ( MatSetType(A, MATMPIBAIJ) > > > ); > > > > > > CHKERRQ( > > MatLoad(A,fd) ); > > > > > > } > > > > > > > > > > > > else > > > > > > > > { > > > > > > > > CHKERRQ( MatSetSizes(A, 5, > > 20, 20, > > > 20) ); //rank 1 gets 25% of the > > rows > > > > > > > > CHKERRQ( MatSetType(A, MATMPIBAIJ) > > > ); > > > > > > CHKERRQ( > > MatLoad(A,fd) ); > > > > > > } > > > > > > > > > > > > > > This throws the following error (probably > > from psplit.c): > > > > > > > > [1]PETSC ERROR: --------------------- Error Message > > > ------------------------------------ > > > > > > [1]PETSC ERROR: > > Nonconforming object sizes! > > > > > > [1]PETSC ERROR: Sum of local lengths 40 > > does not equal > > > global length 20, my > > local length 20 > > > > > > likely a call to > > VecSetSizes() or MatSetSizes() is > > > > > wrong. > > > > > > See > > http://www.mcs.anl.gov/petsc/documentation/faq.html#split! > > > > > > > > > > > > This error printout > > doesn't quite make sense to me. > > > > > I'm trying to specify a total matrix size of 20x20... > > I > > > haven't yet figured out where the > > '40' comes > > > from in the error > > message. > > > > > > > > > > > > Any thoughts on what > > might be going wrong? > > > > > > Its the column specification. Just > > > use PETSC_DETERMINE for the local columns > > since all our > > > sparse matrixformats are > > row divisions > > > anyway. > > > > > > Thanks, > > > > > > Matt > > > Thanks in advance, > > > > > > > > Steena > > > > > > > > > > > > > > > > > > > > > > > > > > -------------------------------------------- > > > > > > On Sun, 3/22/15, > > Barry Smith <[email protected]> > > > wrote: > > > > > > > > > > > > Subject: Re: [petsc-users] Unequal > > sparse matrix row > > > distribution for MPI > > MatMult > > > > > > To: > > "Steena M" <[email protected]> > > > > > > Cc: [email protected] > > > > > > Date: Sunday, > > March 22, 2015, 3:58 PM > > > > > > > > > > > > > > > > > > > > > > > > Steena, > > > > > > > > > > > > I am > > > > > > > > a little unsure of your question. > > > > > > > > > > > > > > 1) > > you can create a MPIBAIJ > > > > > > matrix with any distribution of block > > rows per process > > > you > > > > > > want, just set the > > local row size for each process to > > > > > be > > > > > > what you > > like. Use MatCreateVecs() to get > > > > > correspondingly > > > > > > > > laid out vectors. > > > > > > > > > > > > > > > > or 2) if you have a MPIBAIJ > > > > > > matrix with > > "equal" row layout and you want a > > > new > > > > > > one with uneven row layout you can > > simply use > > > > > > > > MatGetSubMatrix() to create that new matrix. > > > > > > > > > > > > Barry > > > > > > > > > > > > Unless you have > > another reason to have the > > > > > > matrix with an equal number row layout I > > would just > > > generate > > > > > > the matrix with > > the layout you want. > > > > > > > > > > > > > > > > > > > On Mar 22, 2015, at 5:50 PM, Steena > > M > > > > > > <[email protected]> > > > > > > wrote: > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > I need to > > distribute > > > > > > a > > sparse matrix such that each proc owns an unequal > > > number > > > > > > of blocked rows before I proceed with > > MPI MatMult. My > > > > > > > > initial thoughts on doing this: > > > > > > > > > > > > > > 1) Use MatGetSubMatrices() on the > > test > > > > > > MATMPIBAIJ > > matrix to produce a new matrix where each > > > proc > > > > > > has an unequal number of rows. > > > > > > > > > > > > > > 2) Provide > > scatter context for vector X > > > > > > (for MatMult )using IS iscol from > > MatGetSubMatrices() > > > while > > > > > > creating the > > vector X. > > > > > > > > > > > > > > > > 3) > > Call MatMult() > > > > > > > > > > > > > > > > Will > > MatMult_MPIBAIJ continue to scatter > > > > > > this matrix and vector such that each > > proc will own an > > > equal > > > > > > number of matrix > > rows and corresponding diagonal vector > > > > > > > > elements? Should I write my own > > MPIMatMult function to > > > > > > retain my redistribution of the matrix > > and vector? > > > > > > > > > > > > > > > > > > Thanks in > > > > > > > > advance, > > > > > > > > > Steena > > > > > > > > > > > > > > > > > > > > > -- > > > What most > > experimenters > > > take for granted before > > they begin their experiments is > > > > > infinitely more interesting than any results to which > > their > > > experiments lead. > > > -- Norbert > > > Wiener > > > > > > > > > > > > > > > -- > > What most experimenters take for granted before they begin their > > experiments is infinitely more interesting than any results to which their > > experiments lead. > > -- Norbert Wiener > > > > > >
