The matrix is rectangular. What do you need from it? Just its action? Or you need to know the entries to compute Matrix-matrix operations? If you just need the action, have you considered using a MATSHELL?
> On Aug 13, 2020, at 8:16 PM, Matthew Knepley <[email protected]> wrote: > > On Thu, Aug 13, 2020 at 1:54 PM Sajid Ali <[email protected] > <mailto:[email protected]>> wrote: > Hi PETSc-developers, > > When assembling a matrix, what would the relative performance of the > following be : > [a] loop over the rows owned by the mpi-rank twice, first to compute the > values and set preallocation for this mpi-rank and then again to fill in the > values (as recommended in the manual) > [b] loop over the rows once, preallocate and set the values for each row. > > I'm refactoring an application that follows approach [a] but computes the > elements of the matrix twice (once to fill in the nnz arrays and once to set > the values) and I want to know if computing, preallocating and setting the > elements by row instead would be better (so as to not compute the matrix > entries twice which involves calls to boost-geometry). > > I am not sure what you mean by [b]. I do not believe the obvious > interpretation is possible in PETSc. We allocate the > matrix once, not row-by-row, so you could not preallocate just a few rows. > > However, why not have a flag so that on the first pass you do not compute > entries, just the indices? > > Thanks, > > Matt > > I'm attaching a plot that shows (Left) the number of non-zeros per row for a > typical matrix used in this application and (Right) the histogram of the > number of non zeros per row, should this be useful. Note that this matrix has > global dimensions [12800 rows, 65586 columns]. > > PS : This matrix is used for a TAO optimization problem and generating the > matrix takes between ~10 and ~25% of the time (longer on a smaller number of > nodes). > > <image.png> > > Thank You, > Sajid Ali | PhD Candidate > Applied Physics > Northwestern University > s-sajid-ali.github.io <http://s-sajid-ali.github.io/> > > -- > What most experimenters take for granted before they begin their experiments > is infinitely more interesting than any results to which their experiments > lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
