Hi,
to clarify, these commits are already in master (either directly, or in
a condensed way in the course of different pull requests, including
additional fixes of memory leaks and output files):
> 5e3edcb81f added check in Inode to skip GPUs for not Factor
> 346e0b6564 changed reduction logic a little and cleaned up format
> 3a282b2d9d added guards for empty process solves in CUDA
> af6bc10070 same bug fix
> 0365c3a97f try a fix
> 9c37fbf3e7 bug fix
> 72f2ad35b3 fixed bug with CUDA transpose mat-vec
> ab62ce3476 remove vecset as per Barry
> c26191aaa4 use non-collective VecSet
> 8bcb2d50b7 fixed MPI lock from call to collective method
> 3c46958f6d fix bug with empty processor
> 54cfeb1831 added missing settypes
this commit is not in master because I couldn't get the example to run:
> fd2e5db618 added cude test to ex56
these commits are ViennaCL-specific and don't need further consideration
(the wrapper will take care of it):
> 12042c4bfa removing ViennaCL fix to GAMG
> 9508265e8e adding support for MatTranspose
> e5a6000419 adding fix for ViennaCL in MG
The *only* commit with functionality that is not in master is:
> 57224a7035 fixed up pinning CUDA to CPUs
for which PR #1954 holds the discussion of why this commit has
problematic parts.
I've opened a pull request for Barry's fixes to the builds on Summit in
PR #1963. If there are no surprises, this PR will be in master by the
end of the day.
**Overall**: Please start freshly off master (and merge PR 1963 if
needed). Then, have a look at the pinning-to-CPUs-commit and decide
whether it needs to be reworked as for what is discussed in PR #1954. Do
not try to rebase
mark/gamg-fix-viennacl-rebased
or any of its offsprings any further, as this will only cause headaches
and create conflicts with code that has already been fixed.
Best regards,
Karli
On 8/12/19 5:23 PM, Balay, Satish wrote:
I don't really understand the workflow here [with merged branches (to
master) and cherry-picking other stuff]
I've attempted to rebase this branch against latest master. The result is at:
mark/gamg-fix-viennacl-rebased-v3
There were too many merge conflicts that I had to resolve. Its
possible I made mistakes here.
So if this branch is what you need - its best to check each commit
here and verify the changes - before using them.
As Karl suggests - its probably best to pick commits that you need and
fix them [if they have bad code - from merge conflict resolution]
[and some of them can be collapsed]
Satish
------
balay@sb /home/balay/petsc (mark/gamg-fix-viennacl-rebased-v3=)
$ git log --oneline master..
dee2b8b21b (HEAD -> mark/gamg-fix-viennacl-rebased-v3,
origin/mark/gamg-fix-viennacl-rebased-v3, mark/gamg-fix-viennacl-rebased-v2)
protected pinnedtocpu
03d489bde4 1) When detecting version info handle blanks introducted by
preprocessor, error if needed version cannot be detected
056432fa93 Use outputPreprocess instead of preprocess since it prints source to
log
94885b4f80 add back code missing from rebaseing over latest master?
2a748c2fa1 fixed compile errors
5e3edcb81f added check in Inode to skip GPUs for not Factor
fc14e5b821 removed comment
57224a7035 fixed up pinning CUDA to CPUs
346e0b6564 changed reduction logic a little and cleaned up format
3a282b2d9d added guards for empty process solves in CUDA
af6bc10070 same bug fix
0365c3a97f try a fix
9c37fbf3e7 bug fix
fd2e5db618 added cude test to ex56
72f2ad35b3 fixed bug with CUDA transpose mat-vec
ab62ce3476 remove vecset as per Barry
c26191aaa4 use non-collective VecSet
12042c4bfa removing ViennaCL fix to GAMG
3c46958f6d fix bug with empty processor
8bcb2d50b7 fixed MPI lock from call to collective method
54cfeb1831 added missing settypes
9508265e8e adding support for MatTranspose
e5a6000419 adding fix for ViennaCL in MG
On Mon, 12 Aug 2019, Karl Rupp via petsc-dev wrote:
Hi Mark,
most of the CUDA-related fixes from your PR are now in master. Thank you!
The pinning of GPU-matrices to CPUs is not in master because it had several
issues:
https://bitbucket.org/petsc/petsc/pull-requests/1954/cuda-fixes-to-pinning-onto-cpu/diff
The ViennaCL-related changes in mark/gamg-fix-viennacl-rebased can be safely
discarded as the new GPU wrapper will come in place over the next days. ex56
has not been pulled over as it's not running properly on GPUs yet (the pinning
in your branch effectively turned GPU matrices into normal PETSc matrices,
effectively running (almost) everything on the CPU again)
So at this point I recommend to start a new branch off master and manually
transfer over any bits from the pinning that you want to keep.
Best regards,
Karli
On 8/3/19 8:47 PM, Mark Adams wrote:
Karl,
Did you want me to do anything at this point? (on vacation this week) I will
verify that master is all fixed if you get all my stuff integrated when I
get back to work in a week.
Thanks,
Mark
On Sat, Aug 3, 2019 at 10:50 AM Karl Rupp <[email protected]
<mailto:[email protected]>> wrote:
If you ignore the initial ViennaCL-related commits and check against
current master (that just received cherry-picked updates from your PR),
then there are really only a few commits left that are not yet
integrated.
(I'll extract two more PRs on Monday, so master will soon have your
fixes in.)
Best regards,
Karli
On 8/3/19 5:21 AM, Balay, Satish wrote:
> I've attempted to rebase this branch over latest master - and pushed
> my changes to branch mark/gamg-fix-viennacl-rebased-v2
>
> You might want to check each of your commits in this branch to see if
> they are ok. I had to add one extra commit - to make it match 'merge
> of mark/gamg-fix-viennacl-rebased and master'.
>
> This branch has 21 commits. I think its best if you can collapse them
> into reasonable chunks of changes. [presumably a single commit
for all
> the changes is not the correct thing here. But the current set of 21
> commits are all over the place]
>
> If you are able to migrate to this branch - its best to delete
the old
> one [i.e origin/mark/gamg-fix-viennacl-rebased]
>
> Satish
>
> On Fri, 2 Aug 2019, Mark Adams via petsc-dev wrote:
>
>> I have been cherry-picking, etc, branch
mark/gamg-fix-viennacl-rebased and
>> it is very messed up. Can someone please update this branch when
all the
>> fixes are settled down? eg, I am seeing dozens of modified files
that I
>> don't know anything about and I certainly don't want to put in a
PR for
>> them.
>>
>> I also seem to lose my pinToCPU method for cuda matrices. I don't
>> understand how that conflicted with anyone else but it did.
>>
>> Thanks,
>> Mark
>>
>
