On Fri, Oct 8, 2021 at 10:29 AM Karthikeyan Chockalingam - STFC UKRI < [email protected]> wrote:
> The PCApply timing on > > > > gpu > > > > PCApply 6 1.0 1.0235e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 39 0 0 0 0 39 0 0 0 0 0 0 0 0.00e+00 0 > 0.00e+00 0 > > > > and cpu > > > > PCApply 6 1.0 1.0242e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 41 0 0 0 0 41 0 0 0 0 0 > > > You don't have GPUs. probably. Use -dm_mat_type hypre. > are close. It is hard for me tell if hypre on gpu is on or not. > > > > Best, > > Karthik. > > > > > > *From: *"Chockalingam, Karthikeyan (STFC,DL,HC)" < > [email protected]> > *Date: *Friday, 8 October 2021 at 14:55 > *To: *Mark Adams <[email protected]> > *Cc: *"[email protected]" <[email protected]> > *Subject: *Re: [petsc-users] hypre on gpus > > > > Thanks Mark, I will try your recommendations. > > Should I also change -dm_vec_type to hypre currently I have it as mpicuda? > > > > Karthik. > > > > > > *From: *Mark Adams <[email protected]> > *Date: *Friday, 8 October 2021 at 14:33 > *To: *"Chockalingam, Karthikeyan (STFC,DL,HC)" < > [email protected]> > *Cc: *"[email protected]" <[email protected]> > *Subject: *Re: [petsc-users] hypre on gpus > > > > Hypre does not record its flops with PETSc's timers. > > Configure with and without CUDA and see if the timings change in PCApply. > > Hypre does not dynamically switch between CUDA and CPU solves at > this time, but you want to use -dm_mat_type hypre. > > Mark > > > > On Fri, Oct 8, 2021 at 6:59 AM Karthikeyan Chockalingam - STFC UKRI < > [email protected]> wrote: > > Hello, > > > > I am trying to run ex45 (in KSP tutorial) using hypre on gpus. I have > attached the python configuration file and -log_view output from running > the below command options > > > > mpirun -n 2 ./ex45 -log_view -da_grid_x 169 -da_grid_y 169 -da_grid_z 169 > -dm_mat_type mpiaijcusparse -dm_vec_type mpicuda -ksp_type gmres -pc_type > hypre -pc_hypre_type boomeramg -ksp_gmres_restart 31 > -pc_hypre_boomeramg_strong_threshold 0.7 -ksp_monitor > > > > The problem was solved and converged but from the output file I suspect > hypre is not running on gpus as PCApply and DMCreate does *not* record > any gpu Mflop/s. However, some events such KSPSolve, MatMult etc are > running on gpus. > > > > Can you please let me know if I need to add any extra flag to the attached > arch-ci-linux-cuda11-double-xx.py script file to get hypre working on gpus? > > > > Thanks, > > Karthik. > > > > > > This email and any attachments are intended solely for the use of the > named recipients. If you are not the intended recipient you must not use, > disclose, copy or distribute this email or any of its attachments and > should notify the sender immediately and delete this email from your > system. UK Research and Innovation (UKRI) has taken every reasonable > precaution to minimise risk of this email or any attachments containing > viruses or malware but the recipient should carry out its own virus and > malware checks before opening the attachments. UKRI does not accept any > liability for any losses or damages which the recipient may sustain due to > presence of any viruses. > >
