N.B., AMGX interface is a bit experimental. Mark On Thu, Dec 7, 2023 at 4:11 PM Sreeram R Venkat <srven...@utexas.edu> wrote:
> Oh, in that case I will try out BoomerAMG. Getting AMGX to build correctly > was also tricky so hopefully the HYPRE build will be easier. > > Thanks, > Sreeram > > On Thu, Dec 7, 2023, 3:03 PM Pierre Jolivet <pie...@joliv.et> wrote: > >> >> >> On 7 Dec 2023, at 9:37 PM, Sreeram R Venkat <srven...@utexas.edu> wrote: >> >> Thank you Barry and Pierre; I will proceed with the first option. >> >> I want to use the AMGX preconditioner for the KSP. I will try it out and >> see how it performs. >> >> >> Just FYI, AMGX does not handle systems with multiple RHS, and thus has no >> PCMatApply() implementation. >> BoomerAMG does, and there is a PCMatApply_HYPRE_BoomerAMG() >> implementation. >> But let us know if you need assistance figuring things out. >> >> Thanks, >> Pierre >> >> Thanks, >> Sreeram >> >> On Thu, Dec 7, 2023 at 2:02 PM Pierre Jolivet <pie...@joliv.et> wrote: >> >>> To expand on Barry’s answer, we have observed repeatedly that MatMatMult >>> with MatAIJ performs better than MatMult with MatMAIJ, you can reproduce >>> this on your own with >>> https://petsc.org/release/src/mat/tests/ex237.c.html. >>> Also, I’m guessing you are using some sort of preconditioner within your >>> KSP. >>> Not all are “KSPMatSolve-ready”, i.e., they may treat blocks of >>> right-hand sides column by column, which is very inefficient. >>> You could run your code with -info dump and send us dump.0 to see what >>> needs to be done on our end to make things more efficient, should you not >>> be satisfied with the current performance of the code. >>> >>> Thanks, >>> Pierre >>> >>> On 7 Dec 2023, at 8:34 PM, Barry Smith <bsm...@petsc.dev> wrote: >>> >>> >>> >>> On Dec 7, 2023, at 1:17 PM, Sreeram R Venkat <srven...@utexas.edu> >>> wrote: >>> >>> I have 2 sequential matrices M and R (both MATSEQAIJCUSPARSE of size n x >>> n) and a vector v of size n*m. v = [v_1 , v_2 ,... , v_m] where v_i has >>> size n. The data for v can be stored either in column-major or row-major >>> order. Now, I want to do 2 types of operations: >>> >>> 1. Matvecs of the form M*v_i = w_i, for i = 1..m. >>> 2. KSPSolves of the form R*x_i = v_i, for i = 1..m. >>> >>> From what I have read on the documentation, I can think of 2 approaches. >>> >>> 1. Get the pointer to the data in v (column-major) and use it to create >>> a dense matrix V. Then do a MatMatMult with M*V = W, and take the data >>> pointer of W to create the vector w. For KSPSolves, use KSPMatSolve with R >>> and V. >>> >>> 2. Create a MATMAIJ using M/R and use that for matvecs directly with the >>> vector v. I don't know if KSPSolve with the MATMAIJ will know that it is a >>> multiple RHS system and act accordingly. >>> >>> Which would be the more efficient option? >>> >>> >>> Use 1. >>> >>> >>> As a side-note, I am also wondering if there is a way to use row-major >>> storage of the vector v. >>> >>> >>> No >>> >>> The reason is that this could allow for more coalesced memory access >>> when doing matvecs. >>> >>> >>> PETSc matrix-vector products use BLAS GMEV matrix-vector products for >>> the computation so in theory they should already be well-optimized >>> >>> >>> Thanks, >>> Sreeram >>> >>> >>> >>