Thank you Barry and Pierre; I will proceed with the first option. I want to use the AMGX preconditioner for the KSP. I will try it out and see how it performs.
Thanks, Sreeram On Thu, Dec 7, 2023 at 2:02 PM Pierre Jolivet <[email protected]> wrote: > To expand on Barry’s answer, we have observed repeatedly that MatMatMult > with MatAIJ performs better than MatMult with MatMAIJ, you can reproduce > this on your own with https://petsc.org/release/src/mat/tests/ex237.c.html > . > Also, I’m guessing you are using some sort of preconditioner within your > KSP. > Not all are “KSPMatSolve-ready”, i.e., they may treat blocks of right-hand > sides column by column, which is very inefficient. > You could run your code with -info dump and send us dump.0 to see what > needs to be done on our end to make things more efficient, should you not > be satisfied with the current performance of the code. > > Thanks, > Pierre > > On 7 Dec 2023, at 8:34 PM, Barry Smith <[email protected]> wrote: > > > > On Dec 7, 2023, at 1:17 PM, Sreeram R Venkat <[email protected]> wrote: > > I have 2 sequential matrices M and R (both MATSEQAIJCUSPARSE of size n x > n) and a vector v of size n*m. v = [v_1 , v_2 ,... , v_m] where v_i has > size n. The data for v can be stored either in column-major or row-major > order. Now, I want to do 2 types of operations: > > 1. Matvecs of the form M*v_i = w_i, for i = 1..m. > 2. KSPSolves of the form R*x_i = v_i, for i = 1..m. > > From what I have read on the documentation, I can think of 2 approaches. > > 1. Get the pointer to the data in v (column-major) and use it to create a > dense matrix V. Then do a MatMatMult with M*V = W, and take the data > pointer of W to create the vector w. For KSPSolves, use KSPMatSolve with R > and V. > > 2. Create a MATMAIJ using M/R and use that for matvecs directly with the > vector v. I don't know if KSPSolve with the MATMAIJ will know that it is a > multiple RHS system and act accordingly. > > Which would be the more efficient option? > > > Use 1. > > > As a side-note, I am also wondering if there is a way to use row-major > storage of the vector v. > > > No > > The reason is that this could allow for more coalesced memory access when > doing matvecs. > > > PETSc matrix-vector products use BLAS GMEV matrix-vector products for > the computation so in theory they should already be well-optimized > > > Thanks, > Sreeram > > >
