Thanks Barry. I'm still getting the malloc error with NULL. Is there a way to distribute the matrix without explicit preallocation? Different matrices will be loaded during runtime and assigning preallocation parameters would mean an additional preprocessing step. -------------------------------------------- On Sun, 3/29/15, Barry Smith <[email protected]> wrote:
Subject: Re: [petsc-users] Unequal sparse matrix row distribution for MPI MatMult To: "Steena M" <[email protected]> Cc: "Matthew Knepley" <[email protected]>, [email protected] Date: Sunday, March 29, 2015, 9:26 PM > On Mar 29, 2015, at 11:05 PM, Steena M <[email protected]> wrote: > > Thanks Matt. I used PETSC_DETERMINE but I'm now getting an allocation-based error: > > [0]PETSC ERROR: --------------------- Error Message ------------------------------------ > [0]PETSC ERROR: Argument out of range! > [0]PETSC ERROR: New nonzero at (2,18) caused a malloc! > [0]PETSC ERROR: ------------------------------------------------------------------------ > > I tried preallocating on each rank for the diagonal and off diagonal section of the matrix as the next step My current approximations for preallocation > > CHKERRQ( MatMPIBAIJSetPreallocation(A,1,5,PETSC_DEFAULT,5,PETSC_DEFAULT)); These arguments where you pass PETSC_DEFAULT are expecting a pointer not an integer. You can pass NULL in those locations. Though it is better to provide the correct preallocation rather than some defaults. Barry > > are throwing segmentation errors. > > [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range > > Any insights into what I'm doing wrong? > > Thanks, > Steena > > > > On Sun, 3/29/15, Matthew Knepley <[email protected]> wrote: > > Subject: Re: [petsc-users] Unequal sparse matrix row distribution for MPI MatMult > To: "Steena M" <[email protected]> > Cc: "Barry Smith" <[email protected]>, [email protected] > Date: Sunday, March 29, 2015, 10:02 PM > > On Sun, Mar 29, 2015 at > 9:56 PM, Steena M <[email protected]> > wrote: > Hi > Barry, > > > > I am trying to partition a 20 row and 20 col sparse matrix > between two procs such that proc 0 has 15 rows and 20 cols > and proc 1 has 5 rows and 20 cols. The code snippet: > > > > > > > > CHKERRQ(MatCreate(PETSC_COMM_WORLD,&A)); // > at runtime: -matload_block_size 1 > > > > > > if (rank ==0) > > { > > CHKERRQ( MatSetSizes(A, 15, 20, 20, > 20) ); //rank 0 gets 75% of the rows > > CHKERRQ( MatSetType(A, MATMPIBAIJ) > ); > > CHKERRQ( MatLoad(A,fd) ); > > } > > > > else > > { > > CHKERRQ( MatSetSizes(A, 5, 20, 20, > 20) ); //rank 1 gets 25% of the rows > > CHKERRQ( MatSetType(A, MATMPIBAIJ) > ); > > CHKERRQ( MatLoad(A,fd) ); > > } > > > > This throws the following error (probably from psplit.c): > > [1]PETSC ERROR: --------------------- Error Message > ------------------------------------ > > [1]PETSC ERROR: Nonconforming object sizes! > > [1]PETSC ERROR: Sum of local lengths 40 does not equal > global length 20, my local length 20 > > likely a call to VecSetSizes() or MatSetSizes() is > wrong. > > See http://www.mcs.anl.gov/petsc/documentation/faq.html#split! > > > > This error printout doesn't quite make sense to me. > I'm trying to specify a total matrix size of 20x20... I > haven't yet figured out where the '40' comes > from in the error message. > > > > Any thoughts on what might be going wrong? > > Its the column specification. Just > use PETSC_DETERMINE for the local columns since all our > sparse matrixformats are row divisions > anyway. > > Thanks, > > Matt > Thanks in advance, > > Steena > > > > > > > > -------------------------------------------- > > On Sun, 3/22/15, Barry Smith <[email protected]> > wrote: > > > > Subject: Re: [petsc-users] Unequal sparse matrix row > distribution for MPI MatMult > > To: "Steena M" <[email protected]> > > Cc: [email protected] > > Date: Sunday, March 22, 2015, 3:58 PM > > > > > > > > Steena, > > > > I am > > a little unsure of your question. > > > > 1) you can create a MPIBAIJ > > matrix with any distribution of block rows per process > you > > want, just set the local row size for each process to > be > > what you like. Use MatCreateVecs() to get > correspondingly > > laid out vectors. > > > > or 2) if you have a MPIBAIJ > > matrix with "equal" row layout and you want a > new > > one with uneven row layout you can simply use > > MatGetSubMatrix() to create that new matrix. > > > > Barry > > > > Unless you have another reason to have the > > matrix with an equal number row layout I would just > generate > > the matrix with the layout you want. > > > > > > > On Mar 22, 2015, at 5:50 PM, Steena M > > <[email protected]> > > wrote: > > > > > > Hello, > > > > > > I need to distribute > > a sparse matrix such that each proc owns an unequal > number > > of blocked rows before I proceed with MPI MatMult. My > > initial thoughts on doing this: > > > > > > 1) Use MatGetSubMatrices() on the test > > MATMPIBAIJ matrix to produce a new matrix where each > proc > > has an unequal number of rows. > > > > > > 2) Provide scatter context for vector X > > (for MatMult )using IS iscol from MatGetSubMatrices() > while > > creating the vector X. > > > > > > 3) Call MatMult() > > > > > > Will MatMult_MPIBAIJ continue to scatter > > this matrix and vector such that each proc will own an > equal > > number of matrix rows and corresponding diagonal vector > > elements? Should I write my own MPIMatMult function to > > retain my redistribution of the matrix and vector? > > > > > > Thanks in > > advance, > > > Steena > > > > > > > -- > What most experimenters > take for granted before they begin their experiments is > infinitely more interesting than any results to which their > experiments lead. > -- Norbert > Wiener >
