Rongliang,
It appears the NASM solver creates its own subdomains for which it calls
the local function on each one:
PetscErrorCode SNESSetUp_NASM(SNES snes)
....
if (!nasm->subsnes) {
ierr = SNESGetDM(snes,&dm);CHKERRQ(ierr);
if (dm) {
nasm->usesdm = PETSC_TRUE;
ierr =
DMCreateDomainDecomposition(dm,&nasm->n,NULL,NULL,NULL,&subdms);CHKERRQ(ierr);
if (!subdms)
SETERRQ(PetscObjectComm((PetscObject)dm),PETSC_ERR_ARG_WRONGSTATE,"DM has no
default decomposition defined. Set subsolves manually with
SNESNASMSetSubdomains().");
ierr =
DMCreateDomainDecompositionScatters(dm,nasm->n,subdms,&nasm->iscatter,&nasm->oscatter,&nasm->gscatter);CHKERRQ(ierr);
ierr = SNESGetOptionsPrefix(snes, &optionsprefix);CHKERRQ(ierr);
ierr = PetscMalloc1(nasm->n,&nasm->subsnes);CHKERRQ(ierr);
for (i=0; i<nasm->n; i++) {
ierr = SNESCreate(PETSC_COMM_SELF,&nasm->subsnes[i]);CHKERRQ(ierr);
these subdomains do not correspond to the "normal" one subdomain per process
decomposition of the domain that you expect when you call
ierr = DMDAVecGetArray(info->da,user->temp, &z);CHKERRQ(ierr); // added by
rlchen
inside PetscErrorCode FormFunctionLocal(DMDALocalInfo *info,Field **x,Field
**f,void *ptr)
In other words you are expecting that the info->da passed into
FormFunctionLocal() is the same as the DA in your original main() program. But
with NASM solver it simply is not, it is some other DA that is created by the
NASM solver.
If you want to pass some vector like your .temp vector into your local function
you will need to understand exactly how the NASM solver works so you can pass
the correct vectors in. Simple passing a vector associated with the original DA
of the main program won't work.
Barry
> On May 30, 2015, at 8:02 PM, Rongliang Chen <[email protected]> wrote:
>
> Hi Barry and Matt,
>
> Many thanks for your reply. I only have one DA and I do not think I changed
> others when I changed the overlap.
>
> This error can be reproduced using the attached code (added some lines in the
> src/snes/examples/tutorials/ex19.c) run with:
>
> runex19_1:
> -@${MPIEXEC} -n 2 ./ex19 -da_refine 3 -snes_type nasm -da_overlap 2
> -snes_monitor
>
> The full error messages are followed:
> ------------------
> lid velocity = 0.0016, prandtl # = 1, grashof # = 1
> 0 SNES Function norm 4.066115181565e-02
> [1]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> [1]PETSC ERROR: Arguments are incompatible
> [1]PETSC ERROR: Vector local size 1300 is not compatible with DMDA local
> sizes 1400 1728
>
> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for
> trouble shooting.
> [1]PETSC ERROR: Petsc Release Version 3.5.2, unknown
> [1]PETSC ERROR: ./ex19 on a 64bit-debug named rlchen by rlchen Sun May 31
> 08:55:16 2015
> [1]PETSC ERROR: Configure options --download-fblaslapack --download-blacs
> --download-scalapack --download-metis --download-parmetis --download-exodusii
> --download-netcdf --download-hdf5 --with-64-bit-indices --with-c2html=0
> --with-mpi=1 --with-debugging=1 --with-shared-libraries=0
> [1]PETSC ERROR: #1 DMDAVecGetArray() line 71 in
> /home/rlchen/soft/petsc-3.5.2/src/dm/impls/da/dagetarray.c
> [1]PETSC ERROR: #2 FormFunctionLocal() line 275 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/examples/tutorials/ex19.c
> [1]PETSC ERROR: #3 SNESComputeFunction_DMDA() line 90 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/utils/dmdasnes.c
> [1]PETSC ERROR: #4 SNESComputeFunction() line 2033 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/interface/snes.c
> [1]PETSC ERROR: #5 SNESSolve_NEWTONLS() line 174 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/impls/ls/ls.c
> [1]PETSC ERROR: #6 SNESSolve() line 3743 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/interface/snes.c
> [1]PETSC ERROR: #7 SNESNASMSolveLocal_Private() line 716 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/impls/nasm/nasm.c
> [1]PETSC ERROR: #8 SNESSolve_NASM() line 859 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/impls/nasm/nasm.c
> [1]PETSC ERROR: #9 SNESSolve() line 3743 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/interface/snes.c
> [1]PETSC ERROR: #10 main() line 162 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/examples/tutorials/ex19.c
> [1]PETSC ERROR: ----------------End of Error Message -------send entire error
> message to [email protected]
> application called MPI_Abort(MPI_COMM_WORLD, 75) - process 1
> [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> [0]PETSC ERROR: Arguments are incompatible
> [0]PETSC ERROR: Vector local size 1400 is not compatible with DMDA local
> sizes 1500 1836
>
> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for
> trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown
> [0]PETSC ERROR: ./ex19 on a 64bit-debug named rlchen by rlchen Sun May 31
> 08:55:16 2015
> [0]PETSC ERROR: Configure options --download-fblaslapack --download-blacs
> --download-scalapack --download-metis --download-parmetis --download-exodusii
> --download-netcdf --download-hdf5 --with-64-bit-indices --with-c2html=0
> --with-mpi=1 --with-debugging=1 --with-shared-libraries=0
> [0]PETSC ERROR: #1 DMDAVecGetArray() line 71 in
> /home/rlchen/soft/petsc-3.5.2/src/dm/impls/da/dagetarray.c
> [0]PETSC ERROR: #2 FormFunctionLocal() line 275 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/examples/tutorials/ex19.c
> [0]PETSC ERROR: #3 SNESComputeFunction_DMDA() line 90 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/utils/dmdasnes.c
> [0]PETSC ERROR: #4 SNESComputeFunction() line 2033 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/interface/snes.c
> [0]PETSC ERROR: #5 SNESSolve_NEWTONLS() line 174 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/impls/ls/ls.c
> [0]PETSC ERROR: #6 SNESSolve() line 3743 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/interface/snes.c
> [0]PETSC ERROR: #7 SNESNASMSolveLocal_Private() line 716 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/impls/nasm/nasm.c
> [0]PETSC ERROR: #8 SNESSolve_NASM() line 859 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/impls/nasm/nasm.c
> [0]PETSC ERROR: #9 SNESSolve() line 3743 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/interface/snes.c
> [0]PETSC ERROR: #10 main() line 162 in
> /home/rlchen/soft/petsc-3.5.2/src/snes/examples/tutorials/ex19.c
> [0]PETSC ERROR: ----------------End of Error Message -------send entire error
> message to [email protected]
> application called MPI_Abort(MPI_COMM_WORLD, 75) - process 0
>
> =====================================================================================
> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
> = EXIT CODE: 19200
> = CLEANING UP REMAINING PROCESSES
> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
> =====================================================================================
> make: [runex19_1] Error 75 (ignored)
> -----------------------
>
> Best regards,
> Rongliang
>
> On 05/31/2015 01:09 AM, Barry Smith wrote:
>>> On May 30, 2015, at 5:05 AM, Rongliang Chen <[email protected]>
>>> wrote:
>>>
>>> Hi there,
>>>
>>> I am trying to use NASM to solve a nonlinear problem. In my
>>> FormFunctionLocal, I need to call DMDAVecGetArray(da, myvec, z). For the
>>> "-da_overlap 0" case, the code works fine.
>>>
>>> For the "-da_overlap 1" case, when I use the vector "myvec" created by
>>> DMCreateLocalVector, it works fine. But if I use the vector "myvec" created
>>> by DMCreateGlobalVector, I got the error messages:
>>> -----------------------
>>> [6]Arguments are incompatible
>>> [6]PETSC ERROR: Vector local size 597870 is not compatible with DMDA local
>>> sizes 619528 664392
>> I think you are changing something else at the same time you changed the
>> overlap. The number ^^^^ 619528 is the local size of the global vector and
>> it should remain the same when you change the overlap but below you have TWO
>> other numbers 633144 and 650256 Do you have more than one DMDA?
>> Run changing absolutely nothing except the overlap.
>>> ----------------------
>>>
>>> And when I switch to "-da_overlap 2", both of the global and local version
>>> of "mrvec" got error messages:
>>> ----------------------------------
>>> [10]PETSC ERROR: Arguments are incompatible
>>> [10]PETSC ERROR: Vector local size 589680 is not compatible with DMDA local
>>> sizes 633144 678680
>>>
>>> [3]PETSC ERROR: Arguments are incompatible
>>> [3]PETSC ERROR: Vector local size 619528 is not compatible with DMDA local
>>> sizes 650256 696540
>>> -----------------------------------
>>>
>>> Can any one tell me how to use DMDAVecGetArray in FormFunctionLocal for
>>> NASM? Thanks.
>>>
>>> Best regards,
>>> Rongliang
>>>
>
> <ex19.c>
