Thanks for the quick reply Barry!
I have not tried the version with PCApply_Redistribute that you suggest, but I
have a code that does roughly what you describe. It works when running on one
rank, but fails on multiple ranks. I suspect the issue is with the use of
ISEmbed as, quoting the PETSc-manual, "the resulting IS is sequential, since
the index substitution it encodes is purely local" (admittedly I don't fully
understand what that means). If you think using ISEmbed is not a good idea,
I'll try PCApply_Redistribute()
________________________________
From: Barry Smith <[email protected]>
Sent: 16 April 2023 21:11:18
To: Carl-Johan Thore <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: [petsc-users] Fieldsplit with redistribute
There is no code to do this currently.
I would start by building your IS for each split before the PCRedistribute
and then adding to the PCApply_Redistribute() code that "fixes" these IS by
"removing" the entries of the IS associated with removed degrees of freedom and
then shifting the entries indices of the IS by taking into account the removed
indices. But you have probably already been trying this? It does require
digging directly into the PCApply_Redistribute() to get the needed information
(which degrees of freedom are removed by the redistribute code), plus it
requires shifting the MPI rank ownership of the entries of the IS in the same
way the MPI rank ownership of the degrees of freedom of the vector are moved.
If you have some code that you think should be doing this but doesn't work
feel free to send it to us and we may be able to fix it.
Barry
> On Apr 16, 2023, at 2:50 PM, Carl-Johan Thore via petsc-users
> <[email protected]> wrote:
>
> Hello,
> I'm solving a blocksystem
> [A C;
> C' D],
> where D is not zero, using the PCFIELDSPLIT preconditioner and set the split
> using PetscFieldSplitSetIS. This works very well until I try PCREDISTRIBUTE
> (which is attractive as I have many locked DOFs). I suspect something goes
> wrong when constructing the IS for the split (I've tried various things using
> the IS-routines). Can PETSc do this automatically? Or else, any hints?
> Kind regards,
> Carl-Johan
