Re: [petsc-users] DMSWARM with DMDA and KSP

2023-04-30 Thread Matthew Knepley 
On Sun, Apr 30, 2023 at 1:12 PM Matthew Young <
myoung.space.scie...@gmail.com> wrote:

> Hi all,
>
> I am developing a particle-in-cell code that models ions as particles and
> electrons as an inertialess fluid. I use a PIC DMSWARM for the ions, which
> I gather into density and flux before solving a linear system for the
> electrostatic potential (phi). I currently have one DMDA with 5 degrees of
> freedom -- one each for density, 3 flux components, and phi.
>
> When setting up the linear system to solve for phi, I've been following
> examples like KSP ex34.c and ex42.c when writing the KSP operator and RHS
> functions but I'm not sure I have the right approach, since 4 of the DOFs
> are known and 1 is unknown.
>
> I saw this thread
> 
> that recommended using DMDAGetReducedDMDA, which I gather has been
> deprecated in favor of DMDACreateCompatibleDMDA. Is that a good approach
> for managing a regular grid with known and unknown quantities on each node?
> Could a composite DM be useful? Has anyone else worked on a problem like
> this?
>

I recommend making a different DM for each kind of solve you want.
DMDACreateCompatibleDMDA() should be the implementation of DMClone(), but
we have yet to harmonize all things for all DMs. I would create one DM for
your Vlasov components and one for the Poisson.
We follow this strategy in our Vlasov-Poisson test for Landau damping:
https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/swarm/tests/ex9.c

  Thanks,

 Matt


> --Matt
> ==
> Matthew Young, PhD (he/him)
> Research Scientist II
> Space Science Center
> University of New Hampshire
> matthew.yo...@unh.edu
> ==
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

[petsc-users] DMSWARM with DMDA and KSP

2023-04-30 Thread Matthew Young 
Hi all,

I am developing a particle-in-cell code that models ions as particles and
electrons as an inertialess fluid. I use a PIC DMSWARM for the ions, which
I gather into density and flux before solving a linear system for the
electrostatic potential (phi). I currently have one DMDA with 5 degrees of
freedom -- one each for density, 3 flux components, and phi.

When setting up the linear system to solve for phi, I've been following
examples like KSP ex34.c and ex42.c when writing the KSP operator and RHS
functions but I'm not sure I have the right approach, since 4 of the DOFs
are known and 1 is unknown.

I saw this thread

that recommended using DMDAGetReducedDMDA, which I gather has been
deprecated in favor of DMDACreateCompatibleDMDA. Is that a good approach
for managing a regular grid with known and unknown quantities on each node?
Could a composite DM be useful? Has anyone else worked on a problem like
this?

--Matt
==
Matthew Young, PhD (he/him)
Research Scientist II
Space Science Center
University of New Hampshire
matthew.yo...@unh.edu
==

Re: [petsc-users] PETSC ERROR in DMGetLocalBoundingBox?

2023-04-30 Thread Matthew Knepley 
On Sun, Apr 30, 2023 at 2:57 AM Danyang Su  wrote:

> Hi Matt,
>
>
>
> Just let you know that the error problem in DMGetLocalBoundingBox seems
> fixed in the latest dev version. I didn’t catch the error information any
> more.
>

Sorry, I forgot to mail back. Thanks for reporting that.

   Matt


>
>
> Regards,
>
>
>
> Danyang
>
>
>
> *From: *
> *Date: *Friday, March 17, 2023 at 11:02 AM
> *To: *'Matthew Knepley' 
> *Cc: *
> *Subject: *RE: [petsc-users] PETSC ERROR in DMGetLocalBoundingBox?
>
>
>
> Hi Matt,
>
>
>
> I am following up to check if you can reproduce the problem on your side.
>
>
>
> Thanks and have a great weekend,
>
>
>
> Danyang
>
>
>
> *From:* Danyang Su 
> *Sent:* March 4, 2023 4:38 PM
> *To:* Matthew Knepley 
> *Cc:* petsc-users@mcs.anl.gov
> *Subject:* Re: [petsc-users] PETSC ERROR in DMGetLocalBoundingBox?
>
>
>
> Hi Matt,
>
>
>
> Attached is the source code and example. I have deleted most of the unused
> source code but it is still a bit length. Sorry about that. The errors come 
> after
> DMGetLocalBoundingBox and DMGetBoundingBox.
>
>
>
> -> To compile the code
>
> Please type 'make exe' and the executable file petsc_bounding will be
> created under the same folder.
>
>
>
>
>
> -> To test the code
>
> Please go to fold 'test' and type 'mpiexec -n 1 ../petsc_bounding'.
>
>
>
>
>
> -> The output from PETSc 3.18, error information
>
> input file: stedvs.dat
>
>
>
> 
>
> global control parameters
>
> 
>
>
>
> [0]PETSC ERROR: - Error Message
> --
>
> [0]PETSC ERROR: Corrupt argument: https://petsc.org/release/faq/#valgrind
>
> [0]PETSC ERROR: Object already free: Parameter # 1
>
> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
>
> [0]PETSC ERROR: Petsc Release Version 3.18.3, Dec 28, 2022
>
> [0]PETSC ERROR: ../petsc_bounding on a linux-gnu-dbg named starblazer by
> dsu Sat Mar  4 16:20:51 2023
>
> [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++
> --with-fc=gfortran --download-mpich --download-scalapack
> --download-parmetis --download-metis --download-mumps --download-ptscotch
> --download-chaco --download-fblaslapack --download-hypre
> --download-superlu_dist --download-hdf5=yes --download-ctetgen
> --download-zlib --download-pnetcdf --download-cmake
> --with-hdf5-fortran-bindings --with-debugging=1
>
> [0]PETSC ERROR: #1 VecGetArrayRead() at
> /home/dsu/Soft/petsc/petsc-3.18.3/src/vec/vec/interface/rvector.c:1928
>
> [0]PETSC ERROR: #2 DMGetLocalBoundingBox() at
> /home/dsu/Soft/petsc/petsc-3.18.3/src/dm/interface/dmcoordinates.c:897
>
> [0]PETSC ERROR: #3
> /home/dsu/Work/bug-check/petsc_bounding/src/solver_ddmethod.F90:1920
>
> Total volume of simulation domain   0.2000E+01
>
> Total volume of simulation domain   0.2000E+01
>
>
>
>
>
> -> The output from PETSc 3.17 and earlier, no error
>
> input file: stedvs.dat
>
>
>
> 
>
> global control parameters
>
> 
>
>
>
> Total volume of simulation domain   0.2000E+01
>
> Total volume of simulation domain   0.2000E+01
>
>
>
>
>
> Thanks,
>
>
>
> Danyang
>
> *From: *Matthew Knepley 
> *Date: *Friday, March 3, 2023 at 8:58 PM
> *To: *
> *Cc: *
> *Subject: *Re: [petsc-users] PETSC ERROR in DMGetLocalBoundingBox?
>
>
>
> On Sat, Mar 4, 2023 at 1:35 AM  wrote:
>
> Hi All,
>
>
>
> I get a very strange error after upgrading PETSc version to 3.18.3,
> indicating some object is already free. The error is begin and does not
> crash the code. There is no error before PETSc 3.17.5 versions.
>
>
>
> We have changed the way coordinates are handled in order to support higher
> order coordinate fields. Is it possible
>
> to send something that we can run that has this error? It could be on our
> end, but it could also be that you are
>
> destroying a coordinate vector accidentally.
>
>
>
>   Thanks,
>
>
>
>  Matt
>
>
>
>
>
> !Check coordinates
>
> call DMGetCoordinateDM(dmda_flow%da,cda,ierr)
>
> CHKERRQ(ierr)
>
> call DMGetCoordinates(dmda_flow%da,gc,ierr)
>
> CHKERRQ(ierr)
>
> call DMGetLocalBoundingBox(dmda_flow%da,lmin,lmax,ierr)
>
> CHKERRQ(ierr)
>
> call DMGetBoundingBox(dmda_flow%da,gmin,gmax,ierr)
>
> CHKERRQ(ierr)
>
>
>
>
>
> [0]PETSC ERROR: - Error Message
> --
>
> [0]PETSC ERROR: Corrupt argument: https://petsc.org/release/faq/#valgrind
>
> [0]PETSC ERROR: Object already free: Parameter # 1
>
> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
>
> [0]PETSC ERROR: Petsc Release Version 3.18.3, Dec 28, 2022
>
> [0]PETSC ERROR:

Re: [petsc-users] PETSC ERROR in DMGetLocalBoundingBox?

2023-04-30 Thread Danyang Su 
Hi Matt,

 

Just let you know that the error problem in DMGetLocalBoundingBox seems fixed 
in the latest dev version. I didn’t catch the error information any more.

 

Regards,

 

Danyang

 

From: 
Date: Friday, March 17, 2023 at 11:02 AM
To: 'Matthew Knepley' 
Cc: 
Subject: RE: [petsc-users] PETSC ERROR in DMGetLocalBoundingBox?

 

Hi Matt,

 

I am following up to check if you can reproduce the problem on your side. 

 

Thanks and have a great weekend,

 

Danyang

 

From: Danyang Su  
Sent: March 4, 2023 4:38 PM
To: Matthew Knepley 
Cc: petsc-users@mcs.anl.gov
Subject: Re: [petsc-users] PETSC ERROR in DMGetLocalBoundingBox?

 

Hi Matt,

 

Attached is the source code and example. I have deleted most of the unused 
source code but it is still a bit length. Sorry about that. The errors come 
after DMGetLocalBoundingBox and DMGetBoundingBox.

 

-> To compile the code

Please type 'make exe' and the executable file petsc_bounding will be created 
under the same folder.

 

 

-> To test the code

Please go to fold 'test' and type 'mpiexec -n 1 ../petsc_bounding'.

 

 

-> The output from PETSc 3.18, error information

input file: stedvs.dat

 



global control parameters



 

[0]PETSC ERROR: - Error Message 
--

[0]PETSC ERROR: Corrupt argument: https://petsc.org/release/faq/#valgrind

[0]PETSC ERROR: Object already free: Parameter # 1

[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.

[0]PETSC ERROR: Petsc Release Version 3.18.3, Dec 28, 2022 

[0]PETSC ERROR: ../petsc_bounding on a linux-gnu-dbg named starblazer by dsu 
Sat Mar  4 16:20:51 2023

[0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ 
--with-fc=gfortran --download-mpich --download-scalapack --download-parmetis 
--download-metis --download-mumps --download-ptscotch --download-chaco 
--download-fblaslapack --download-hypre --download-superlu_dist 
--download-hdf5=yes --download-ctetgen --download-zlib --download-pnetcdf 
--download-cmake --with-hdf5-fortran-bindings --with-debugging=1

[0]PETSC ERROR: #1 VecGetArrayRead() at 
/home/dsu/Soft/petsc/petsc-3.18.3/src/vec/vec/interface/rvector.c:1928

[0]PETSC ERROR: #2 DMGetLocalBoundingBox() at 
/home/dsu/Soft/petsc/petsc-3.18.3/src/dm/interface/dmcoordinates.c:897

[0]PETSC ERROR: #3 
/home/dsu/Work/bug-check/petsc_bounding/src/solver_ddmethod.F90:1920

Total volume of simulation domain   0.2000E+01

Total volume of simulation domain   0.2000E+01

 

 

-> The output from PETSc 3.17 and earlier, no error

input file: stedvs.dat

 



global control parameters



 

Total volume of simulation domain   0.2000E+01

Total volume of simulation domain   0.2000E+01

 

 

Thanks,

 

Danyang

From: Matthew Knepley 
Date: Friday, March 3, 2023 at 8:58 PM
To: 
Cc: 
Subject: Re: [petsc-users] PETSC ERROR in DMGetLocalBoundingBox?

 

On Sat, Mar 4, 2023 at 1:35 AM  wrote:

Hi All,

 

I get a very strange error after upgrading PETSc version to 3.18.3, indicating 
some object is already free. The error is begin and does not crash the code. 
There is no error before PETSc 3.17.5 versions.

 

We have changed the way coordinates are handled in order to support higher 
order coordinate fields. Is it possible

to send something that we can run that has this error? It could be on our end, 
but it could also be that you are

destroying a coordinate vector accidentally.

 

  Thanks,

 

 Matt

 

 

!Check coordinates

call DMGetCoordinateDM(dmda_flow%da,cda,ierr)

CHKERRQ(ierr)

call DMGetCoordinates(dmda_flow%da,gc,ierr)

CHKERRQ(ierr)

call DMGetLocalBoundingBox(dmda_flow%da,lmin,lmax,ierr)

CHKERRQ(ierr)

call DMGetBoundingBox(dmda_flow%da,gmin,gmax,ierr)

CHKERRQ(ierr)

 

 

[0]PETSC ERROR: - Error Message 
--

[0]PETSC ERROR: Corrupt argument: https://petsc.org/release/faq/#valgrind

[0]PETSC ERROR: Object already free: Parameter # 1

[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.

[0]PETSC ERROR: Petsc Release Version 3.18.3, Dec 28, 2022

[0]PETSC ERROR: ../min3p-hpc-mpi-petsc-3.18.3 on a linux-gnu-dbg named 
starblazer by dsu Fri Mar  3 16:26:03 2023

[0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ 
--with-fc=gfortran --download-mpich --download-scalapack --download-parmetis 
--download-metis --download-mumps --download-ptscotch --download-chaco 
--download-fblaslapack --download-hypre --download-superlu_dist 
--download-hdf5=yes --download-ctetgen --download-zlib --download-pnetcdf