In the application the storage is contiguous but the global indexing is
not. I would like to use AO as a translation layer but I don't
understand it.
My case is actually simple even if it is in a large application, I have
Mat A, Vec b and Vec x
After calling KSPSolve, I use VecGetArrayReadF90 to get a pointer to the
data and they are in the wrong ordering, so for example the first
element of the solution array on process 0 belongs to process 1 in the
application.
Is it at this point that I should use the AO translation layer? This
would be quite bad, it means to build Mat A and Vec b there is MPI
communication and also to get the data of Vec x back in the application.
Anyway, I've tried to use AOPetscToApplicationPermuteReal on the
solution array but it doesn't work as I would like. Is this function
suppose to do MPI communication between processes and fetch the values
of the application ordering?
Cheers,
Enrico
On 19/10/2023 15:25, Matthew Knepley wrote:
On Thu, Oct 19, 2023 at 8:57 AM Enrico <[email protected]
<mailto:[email protected]>> wrote:
Maybe I wasn't clear enough. I would like to completely get rid of
Petsc
ordering because I don't want extra communication between processes to
construct the vector and the matrix (since I have to fill them every
time step because I'm just using the linear solver with a Mat and a Vec
data structure). I don't understand how I can do that.
Any program you write to do linear algebra will have contiguous storage
because it
is so much faster. Contiguous indexing makes sense for contiguous
storage. If you
want to use non-contiguous indexing for contiguous storage, you would
need some
translation layer. The AO is such a translation, but you could do this
any way you want.
Thanks,
Matt
My initial idea was to create another global index ordering within my
application to use only for the Petsc interface but then I think that
the ghost cells are wrong.
On 19/10/2023 14:50, Matthew Knepley wrote:
> On Thu, Oct 19, 2023 at 6:51 AM Enrico <[email protected]
<mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>> wrote:
>
> Hello,
>
> if I create an application ordering using AOCreateBasic, should I
> provide the same array for const PetscInt myapp[] and const
PetscInt
> mypetsc[] in order to get the same ordering of the application
> within PETSC?
>
>
> Are you asking if the identity permutation can be constructed
using the
> same array twice? Yes.
>
> And once I define the ordering so that the local vector and
matrix are
> defined in PETSC as in my application, how can I use it to
create the
> actual vector and matrix?
>
>
> The vectors and matrices do not change. The AO is a permutation.
You can
> use it to permute
> a vector into another order, or to convert on index to another.
>
> Thanks,
>
> Matt
>
> Thanks in advance for the help.
>
> Cheers,
> Enrico
>
> On 18/10/2023 13:39, Matthew Knepley wrote:
> > On Wed, Oct 18, 2023 at 5:55 AM Enrico <[email protected]
<mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>
> > <mailto:[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>>> wrote:
> >
> > Hello,
> >
> > I'm trying to use Petsc to solve a linear system in an
> application. I'm
> > using the coordinate format to define the matrix and the
> vector (it
> > should work better on GPU but at the moment every test
is on
> CPU).
> > After
> > the call to VecSetValuesCOO, I've noticed that the
vector is
> storing
> > the
> > data in a different way from my application. For
example with two
> > processes in the application
> >
> > process 0 owns cells 2, 3, 4
> >
> > process 1 owns cells 0, 1, 5
> >
> > But in the vector data structure of Petsc
> >
> > process 0 owns cells 0, 1, 2
> >
> > process 1 owns cells 3, 4, 5
> >
> > This is in principle not a big issue, but after
solving the
> linear
> > system I get the solution vector x and I want to get the
> values in the
> > correct processes. Is there a way to get vector values
from other
> > processes or to get a mapping so that I can do it myself?
> >
> >
> > By definition, PETSc vectors and matrices own contiguous row
> blocks. If
> > you want to have another,
> > global ordering, we support that with
> > https://petsc.org/main/manualpages/AO/
<https://petsc.org/main/manualpages/AO/>
> <https://petsc.org/main/manualpages/AO/
<https://petsc.org/main/manualpages/AO/>>
> > <https://petsc.org/main/manualpages/AO/
<https://petsc.org/main/manualpages/AO/>
> <https://petsc.org/main/manualpages/AO/
<https://petsc.org/main/manualpages/AO/>>>
> >
> > Thanks,
> >
> > Matt
> >
> > Cheers,
> > Enrico Degregori
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin
their
> > experiments is infinitely more interesting than any
results to which
> > their experiments lead.
> > -- Norbert Wiener
> >
> > https://www.cse.buffalo.edu/~knepley/
<https://www.cse.buffalo.edu/~knepley/>
> <https://www.cse.buffalo.edu/~knepley/
<https://www.cse.buffalo.edu/~knepley/>>
> <http://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/~knepley/>
> <http://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/~knepley/>>>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
<https://www.cse.buffalo.edu/~knepley/>
<http://www.cse.buffalo.edu/~knepley/
<http://www.cse.buffalo.edu/~knepley/>>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
