Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-03 Thread Satish Balay via petsc-dev
you can do [in your branch] git revert 02b1d1997a21007b49a933dc43fe9c1894bb5f07 Satish On Fri, 3 Apr 2020, Junchao Zhang wrote: > Satish, > I have a MR !2688 to fix it. How to revert your revert to get the MR > actually tested? > --Junchao Zhang > > > On Fri, Apr 3, 2020 at 9:11 PM Satish

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-03 Thread Junchao Zhang
Satish, I have a MR !2688 to fix it. How to revert your revert to get the MR actually tested? --Junchao Zhang On Fri, Apr 3, 2020 at 9:11 PM Satish Balay wrote: > https://gitlab.com/petsc/petsc/pipelines/132414153/builds > > this pipeline had 1 rerun for linux-cuda-double and 5 reruns for >

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-03 Thread Satish Balay via petsc-dev
https://gitlab.com/petsc/petsc/pipelines/132414153/builds this pipeline had 1 rerun for linux-cuda-double and 5 reruns for linux-c-exodus-dbg I've reverted 2655 for now. We can revert this once the issue is resolved. Satish On Thu, 2 Apr 2020, Satish Balay via petsc-dev wrote: > Perhaps we

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-02 Thread Satish Balay via petsc-dev
Perhaps we should revert this merge - so that the CI is stabilized for now. Satish On Thu, 2 Apr 2020, Junchao Zhang wrote: > Seems caused by MR 2655 > . I reverted it and > tested in CI several times and the error did not appear. Let's

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-02 Thread Junchao Zhang
Seems caused by MR 2655 . I reverted it and tested in CI several times and the error did not appear. Let's assume the MR has a bug. I am looking into it. --Junchao Zhang On Thu, Apr 2, 2020 at 10:58 AM Satish Balay wrote: > That was a

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-02 Thread Satish Balay via petsc-dev
That was a different error. This does keep coming up occasionally. https://gitlab.com/petsc/petsc/-/issues/360#note_250063306 The current issue is: https://gitlab.com/petsc/petsc/-/issues/360#note_314185490 Satish On Thu, 2 Apr 2020, Karl Rupp wrote: > The fluctuations in this example have

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-01 Thread Karl Rupp
The fluctuations in this example have been fixed a few months ago; the issue was the use of multiple streams instead of a single one. Maybe additional CUDA streams have been reintroduced recently? Best regards, Karli On 4/2/20 5:02 AM, Junchao Zhang wrote: I could not reproduce it locally.

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-01 Thread Junchao Zhang
I could not reproduce it locally. Even in the CI, it is random. --Junchao Zhang On Wed, Apr 1, 2020 at 7:47 PM Matthew Knepley wrote: > I saw Satish talking about this on the CI Tracker MR. > >Matt > > On Wed, Apr 1, 2020 at 8:36 PM Lisandro Dalcin wrote: > >> Well, my request will not

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-01 Thread Matthew Knepley
I saw Satish talking about this on the CI Tracker MR. Matt On Wed, Apr 1, 2020 at 8:36 PM Lisandro Dalcin wrote: > Well, my request will not fix the problem: > https://gitlab.com/petsc/petsc/-/jobs/495147366#L5231 > > On Thu, 2 Apr 2020 at 03:26, Lisandro Dalcin wrote: > >> Can anyone

Re: [petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-01 Thread Lisandro Dalcin
Well, my request will not fix the problem: https://gitlab.com/petsc/petsc/-/jobs/495147366#L5231 On Thu, 2 Apr 2020 at 03:26, Lisandro Dalcin wrote: > Can anyone messing with CPUs please update test snes_tutorials-ex19_cuda_1 > to use -ksp_monitor_short and update its output with REPLACE=1 ? >

[petsc-dev] snes_tutorials-ex19_cuda_1

2020-04-01 Thread Lisandro Dalcin
Can anyone messing with CPUs please update test snes_tutorials-ex19_cuda_1 to use -ksp_monitor_short and update its output with REPLACE=1 ? Please do it in maint, or cherry-pick if already fixed in master. Regards, -- Lisandro Dalcin Research Scientist Extreme Computing Research