Re: [petsc-users] VecScatter betweeen Vectors with different parallel layouts

[Jed Brown]

"Barletta, Ivano"  writes:

> Hello PETSc users
>
> I was wondering if you've got any suggestion for the issue
> mentioned in the topic.
>
> I'm solving a linear system on my unstructured grid that has its
> own partitioning. When I set up the linear system in PETSc I
> use the model partitioning whose grid points might not be
> evenly balanced (generally is not) over the processors.
>
> So far I use VecScatter and IS to re-order the solution vector
> with the application numbering.

Mapping between two orderings with different distributions is usually
not a good strategy.  Much better to balance the partitioning for your
unstructured grid.

> What I aim at, now, is to let PETSc to decide the linear system
> partitioning with PETSC_DECIDE local dimension and
> see if I get some gain in performances, but in
> this case I have to scatter between vectors that, at any rate, have
> different parallel layouts (one of the requirements of
> VecScatter is to have 2 Vecs with same layouts)

PETSC_DECIDE just creates equal vertex partitions, but it has no
information with which to make the partitions have locality.  Unless you
have a particular ordering, like a space-filling curve, this kind of
simplistic partitioning by vertex index will tend to produce parts that
have very poor locality (i.e., very high edge cut).

> Does anyone of you have ever faced a similar problem?
> any suggestion about how to circumvent this?
>
> Feel free to ask for further information if my explanation is
> not clear enough
>
> Thanks in advance
> Ivano


signature.asc
Description: PGP signature

Re: [petsc-users] VecScatter betweeen Vectors with different parallel layouts

[Matthew Knepley]

On Mon, Feb 13, 2017 at 5:51 AM, Barletta, Ivano  wrote:

> Hello PETSc users
>
> I was wondering if you've got any suggestion for the issue
> mentioned in the topic.
>
> I'm solving a linear system on my unstructured grid that has its
> own partitioning. When I set up the linear system in PETSc I
> use the model partitioning whose grid points might not be
> evenly balanced (generally is not) over the processors.
>
> So far I use VecScatter and IS to re-order the solution vector
> with the application numbering.
>
> What I aim at, now, is to let PETSc to decide the linear system
> partitioning with PETSC_DECIDE local dimension and
> see if I get some gain in performances, but in
> this case I have to scatter between vectors that, at any rate, have
> different parallel layouts (one of the requirements of
> VecScatter is to have 2 Vecs with same layouts)
>

That is exactly what VecScatter does. It does not require the same layout.

Example:
  proc 0 proc1
Vec A 012   3
Vec B 01  2  3

You create a VecScatter A --> B with indices
   proc 0 proc 1
from   0 1   23
to   3  2  1 0

and it will reverse the vector into the new layout.

   Matt


> Does anyone of you have ever faced a similar problem?
> any suggestion about how to circumvent this?
>
> Feel free to ask for further information if my explanation is
> not clear enough
>
> Thanks in advance
> Ivano
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

[petsc-users] VecScatter betweeen Vectors with different parallel layouts

[Barletta, Ivano]

Hello PETSc users

I was wondering if you've got any suggestion for the issue
mentioned in the topic.

I'm solving a linear system on my unstructured grid that has its
own partitioning. When I set up the linear system in PETSc I
use the model partitioning whose grid points might not be
evenly balanced (generally is not) over the processors.

So far I use VecScatter and IS to re-order the solution vector
with the application numbering.

What I aim at, now, is to let PETSc to decide the linear system
partitioning with PETSC_DECIDE local dimension and
see if I get some gain in performances, but in
this case I have to scatter between vectors that, at any rate, have
different parallel layouts (one of the requirements of
VecScatter is to have 2 Vecs with same layouts)

Does anyone of you have ever faced a similar problem?
any suggestion about how to circumvent this?

Feel free to ask for further information if my explanation is
not clear enough

Thanks in advance
Ivano