date:20220617

Re: [petsc-users] Pestc-matlab issue

2022-06-17 Thread Barry Smith


  Please send configure.log and make.log to petsc-ma...@mcs.anl.gov 
 



> On Jun 17, 2022, at 1:52 PM, Yongxiang Lei  wrote:
> 
> Dear concerns,
> 
> I met such issues when I  confirm that my Matlab installation is finished. I 
> am also sure that the g++ version is given. Could you please check this 
> problem for me?  The related information is given as follows. 
> 
> Looking forward to hearing from you
> Best regards
> Xiang
> 
> >> mex -setup cpp
> MEX configured to use 'g++' for C++ language compilation.
> .
> [ADS+u2192020@cos8-25136ecf ~]$ g++ --version
> g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-13)
> Copyright (C) 2018 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> .
> export PETSC_DIR=$PWD
> export PETSC_ARCH=linux-debug
> ./configure \
>   --CC=$HOME/sfw/linux/openmpi-4.0.2/bin/mpicc \
>   --CXX=$HOME/sfw/linux/openmpi-4.0.2/bin/mpicxx \
>   --FC=$HOME/sfw/linux/openmpi-4.0.2/bin/mpif90 \
>   --with-debugging=1 \
>   --download-hypre=1 \
>   --download-fblaslapack=1 \
>   --with-x=0\
>   --with-matlab-dir=/home/ADS/u2192020/Matlab2022/ \
>   --with-matlab-engine=1 \
>   --with-matlab-engine-dir=/home/ADS/u2192020/Matlab2022/extern/engines/
> =
> =
> Now to check if the libraries are working do:
> make PETSC_DIR=/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB 
> PETSC_ARCH=linux-debug check
> =
> [ADS+u2192020@cos8-25136ecf PETSc-MATLAB]$ make -j4 check
> Running check examples to verify correct installation
> Using PETSC_DIR=/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB and 
> PETSC_ARCH=linux-debug
> gmake[3]: 
> [/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/lib/petsc/conf/rules:320: 
> ex19.PETSc] Error 2 (ignored)
> ***Error detected during compile or link!***
> See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> 
> /home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/src/snes/tutorials ex19
> *
> /home/ADS/u2192020/sfw/linux/openmpi-4.0.2/bin/mpicc -fPIC -Wall 
> -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector 
> -fvisibility=hidden -g3 -O0  -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas 
> -Wno-lto-type-mismatch -fstack-protector -fvisibility=hidden -g3 -O0
> -I/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/include 
> -I/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/include 
> -I/home/ADS/u2192020/Matlab2022/extern/include ex19.c  
> -Wl,-rpath,/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/lib 
> -L/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/lib 
> -Wl,-rpath,/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/lib 
> -L/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/lib 
> -Wl,-rpath,/home/ADS/u2192020/Matlab2022/bin/glnxa64 
> -L/home/ADS/u2192020/Matlab2022/bin/glnxa64 
> -Wl,-rpath,/home/ADS/u2192020/Matlab2022/extern/lib/glnxa64 
> -L/home/ADS/u2192020/Matlab2022/extern/lib/glnxa64 
> -Wl,-rpath,/home/ADS/u2192020/sfw/linux/openmpi-4.0.2/lib 
> -L/home/ADS/u2192020/sfw/linux/openmpi-4.0.2/lib 
> -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/8 
> -L/usr/lib/gcc/x86_64-redhat-linux/8 -lpetsc -lHYPRE -lflapack -lfblas -lm 
> -leng -lmex -lmx -lmat -lmwm_dispatcher -lmwopcmodel -lmwservices 
> -lmwopcmodel -lmwm_dispatcher -lmwmpath -lmwopcmodel -lmwservices 
> -lmwopcmodel -lmwservices -lxerces-c -lstdc++ -ldl -lmpi_usempif08 
> -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm 
> -lgcc_s -lquadmath -lpthread -lquadmath -lstdc++ -ldl -o ex19
> //home/ADS/u2192020/Matlab2022/bin/glnxa64/libmwopccore.so: undefined 
> reference to `std::__cxx11::basic_ostringstream std::char_traits, std::allocator 
> >::basic_ostringstream()@GLIBCXX_3.4.26'
> //home/ADS/u2192020/Matlab2022/bin/glnxa64/libmwfoundation_matlabdata_matlab.so:
>  undefined reference to 
> `std::logic_error::logic_error(std::logic_error&&)@GLIBCXX_3.4.26'
> /home/ADS/u2192020/Matlab2022/bin/glnxa64/libmex.so: undefined reference to 
> `std::__cxx11::basic_ostringstream, 
> std::allocator >::basic_ostringstream()@GLIBCXX_3.4.26'
> /home/ADS/u2192020/Matlab2022/bin/glnxa64/libmx.so: undefined reference to 
> `std::__cxx11::basic_stringstream, 
> std::allocator >::basic_stringstream()@GLIBCXX_3.4.26'
> collect2: error: ld returned 1 exit status
> gmake[4]: *** [: ex19] Error 1
> 1,5c1,10
> < lid velocity = 0.0016, prandtl # = 1., grashof # = 1.
> <   0 SNES Function norm 0.0406612 
> <   1 SNES Function

[petsc-users] Pestc-matlab issue

2022-06-17 Thread Yongxiang Lei

Dear concerns,

I met such issues when I  confirm that my Matlab installation is finished.
I am also sure that the g++ version is given. Could you please check this
problem for me?  The related information is given as follows.

Looking forward to hearing from you
Best regards
Xiang

>> mex -setup cpp
MEX configured to use 'g++' for C++ language compilation.
.
[ADS+u2192020@cos8-25136ecf ~]$ g++ --version
g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-13)
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
.
export PETSC_DIR=$PWD
export PETSC_ARCH=linux-debug
./configure \
  --CC=$HOME/sfw/linux/openmpi-4.0.2/bin/mpicc \
  --CXX=$HOME/sfw/linux/openmpi-4.0.2/bin/mpicxx \
  --FC=$HOME/sfw/linux/openmpi-4.0.2/bin/mpif90 \
  --with-debugging=1 \
  --download-hypre=1 \
  --download-fblaslapack=1 \
  --with-x=0\
  --with-matlab-dir=/home/ADS/u2192020/Matlab2022/ \
  --with-matlab-engine=1 \
  --with-matlab-engine-dir=/home/ADS/u2192020/Matlab2022/extern/engines/
=
=
Now to check if the libraries are working do:
make PETSC_DIR=/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB
PETSC_ARCH=linux-debug check
=
[ADS+u2192020@cos8-25136ecf PETSc-MATLAB]$ make -j4 check
Running check examples to verify correct installation
Using PETSC_DIR=/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB and
PETSC_ARCH=linux-debug
gmake[3]:
[/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/lib/petsc/conf/rules:320:
ex19.PETSc] Error 2 (ignored)
***Error detected during compile or link!***
See http://www.mcs.anl.gov/petsc/documentation/faq.html
/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/src/snes/tutorials ex19
*
/home/ADS/u2192020/sfw/linux/openmpi-4.0.2/bin/mpicc -fPIC -Wall
-Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch
-fstack-protector -fvisibility=hidden -g3 -O0  -fPIC -Wall -Wwrite-strings
-Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector
-fvisibility=hidden -g3 -O0
 -I/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/include
-I/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/include
-I/home/ADS/u2192020/Matlab2022/extern/include ex19.c
 -Wl,-rpath,/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/lib
-L/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/lib
-Wl,-rpath,/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/lib
-L/home/ADS/u2192020/sfw/petsc/PETSc-MATLAB/linux-debug/lib
-Wl,-rpath,/home/ADS/u2192020/Matlab2022/bin/glnxa64
-L/home/ADS/u2192020/Matlab2022/bin/glnxa64
-Wl,-rpath,/home/ADS/u2192020/Matlab2022/extern/lib/glnxa64
-L/home/ADS/u2192020/Matlab2022/extern/lib/glnxa64
-Wl,-rpath,/home/ADS/u2192020/sfw/linux/openmpi-4.0.2/lib
-L/home/ADS/u2192020/sfw/linux/openmpi-4.0.2/lib
-Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/8
-L/usr/lib/gcc/x86_64-redhat-linux/8 -lpetsc -lHYPRE -lflapack -lfblas -lm
-leng -lmex -lmx -lmat -lmwm_dispatcher -lmwopcmodel -lmwservices
-lmwopcmodel -lmwm_dispatcher -lmwmpath -lmwopcmodel -lmwservices
-lmwopcmodel -lmwservices -lxerces-c -lstdc++ -ldl -lmpi_usempif08
-lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm
-lgcc_s -lquadmath -lpthread -lquadmath -lstdc++ -ldl -o ex19
//home/ADS/u2192020/Matlab2022/bin/glnxa64/libmwopccore.so: undefined
reference to `std::__cxx11::basic_ostringstream, std::allocator
>::basic_ostringstream()@GLIBCXX_3.4.26'
//home/ADS/u2192020/Matlab2022/bin/glnxa64/libmwfoundation_matlabdata_matlab.so:
undefined reference to
`std::logic_error::logic_error(std::logic_error&&)@GLIBCXX_3.4.26'
/home/ADS/u2192020/Matlab2022/bin/glnxa64/libmex.so: undefined reference to
`std::__cxx11::basic_ostringstream,
std::allocator >::basic_ostringstream()@GLIBCXX_3.4.26'
/home/ADS/u2192020/Matlab2022/bin/glnxa64/libmx.so: undefined reference to
`std::__cxx11::basic_stringstream,
std::allocator >::basic_stringstream()@GLIBCXX_3.4.26'
collect2: error: ld returned 1 exit status
gmake[4]: *** [: ex19] Error 1
1,5c1,10
< lid velocity = 0.0016, prandtl # = 1., grashof # = 1.
<   0 SNES Function norm 0.0406612
<   1 SNES Function norm 4.12227e-06
<   2 SNES Function norm 6.098e-11
< Number of SNES iterations = 2
---
> --
> mpiexec was unable to launch the specified application as it could not
access
> or execute an executable:
>
> Executable: ./ex19
> Node: cos8-25136ecf
>
> while attempting to start process rank 0.
> --
> 2 total processes

Re: [petsc-users] Parallel matrix-vector product with Matrix Shell

2022-06-17 Thread Mario Rossi

Thanks a lot, Jose!
I looked at the eps folder (where I found the test8.c that has been my
starting point) but I did not look at the nep folder (my fault!)
Thanks again,
Mario

Il giorno ven 17 giu 2022 alle ore 17:34 Jose E. Roman 
ha scritto:

> You can use VecGetOwnershipRange() to determine the range of global
> indices corresponding to the local portion of a vector, and VecGetArray()
> to access the values. In SLEPc, you can assume that X and Y will have the
> same parallel distribution.
>
> For an example of a shell matrix that implements the matrix-vector product
> in parallel, have a look at this:
> https://slepc.upv.es/documentation/current/src/nep/tutorials/ex21.c.html
> It is a simple tridiagonal example, where neighborwise communication is
> done with two calls to MPI_Sendrecv().
>
> Jose
>
>
> > El 17 jun 2022, a las 17:21, Mario Rossi  escribió:
> >
> > I need to find the largest eigenvalues (say the first three) of a very
> large matrix and I am using
> > a combination of PetSc and SLEPc. In particular, I am using a shell
> matrix. I wrote a "custom"
> > matrix-vector product and everything works fine in serial (one task)
> mode for a "small" case.
> > For the real case, I need multiple (at least 128) tasks for memory
> reasons so I need a parallel variant of the custom matrix-vector product. I
> know exactly how to write the parallel variant
> > (in plain MPI) but I am, somehow, blocked because it is not clear to me
> what each task receives
> > and what is expected to provide in the parallel matrix-vector product.
> > More in detail, with a single task, the function receives the full X
> vector and is expected to provide the full Y vector resulting from Y=A*X.
> > What does it happen with multiple tasks? If I understand correctly
> > in the matrix shell definition, I can choose to split the matrix into
> blocks of rows so that the matrix-vector function should compute a block of
> elements of the vector Y but does it receive only the corresponding subset
> of the X (input vector)? (this is what I guess happens) and in output, does
> > each task return its subset of elements of Y as if it were the whole
> array and then PetSc manages all the subsets? Is there anyone who has a
> working example of a parallel matrix-vector product for matrix shell?
> > Thanks in advance for any help you can provide!
> > Mario
> > i
> >
>
>

Re: [petsc-users] Customizing NASM subsnes

2022-06-17 Thread Matthew Knepley

On Fri, Jun 17, 2022 at 11:02 AM Barry Smith  wrote:

>
>   You do not need the loop over size. Each rank sets options and options
> prefix for its local objects and never anyone elses.
>
>char prefix[10];
>
>sprintf(prefix,"sub_%d_",rank);
>
>SNESNASMGetSNES(snes,0,);
>
>  if (rank  SNESSetType(subsnes,SNESNEWTONLS); CHKERRQ(ierr); // newton
> for regular domains
>  } else {
> SNESSetType(subsnes,SNESFAS); CHKERRQ(ierr); // fas for last
> domain
>  }
>  SNESSetOptionsPrefix(subsnes,prefix);
>   }
>}
>
>
>To get the prefix to work try calling   SNESSetFromOptions(subsnes);
> immediately after your SNESSetOptionsPrefix(subsnes,prefix); call
>
>Matt, it looks like there may be a bug in NASM, except in one
> particular case, it never calls SNESSetFromOptions() on the subsenses.
>

I bet Barry is correct. I can fix it, but unfortunately, I leave for a week
long conference tomorrow, of which I am an organizer, so I don't
think I can do it for a week.

  Thanks,

 Matt


>   Barry
>
>
> On Jun 17, 2022, at 10:47 AM, Takahashi, Tadanaga  wrote:
>
> Thank you. I am now able to pull each subsnes, change its snes type
> through the API, and set a prefix. This is my updated code:
>
>SNES   snes, subsnes;
>PetscMPIIntrank, size;
>...
>ierr = SNESCreate(PETSC_COMM_WORLD,); CHKERRQ(ierr);
>ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);
>ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);
>...
>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>ierr = SNESSetUp(snes); CHKERRQ(ierr);
>PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);
>PetscBarrier(NULL);
>for (i=0; i   char prefix[10];
>   sprintf(prefix,"sub_%d_",i);
>   if(i==rank) {
>  ierr = SNESNASMGetNumber(snes,);
>  printf("rank = %d has %d block(s)\n",i,Nd);
>  if (i  SNESNASMGetSNES(snes,0,);
> SNESSetType(subsnes,SNESNEWTONLS); CHKERRQ(ierr); // newton
> for regular domains
>  } else {
> SNESNASMGetSNES(snes,0,);
> SNESSetType(subsnes,SNESFAS); CHKERRQ(ierr); // fas for last
> domain
>  }
>  SNESSetOptionsPrefix(subsnes,prefix);
>   }
>}
>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>...
>ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr);
>
> However, I still cannot change SNES, KSP, and PC types for
> individual domains through the command arguments. I checked the subdomains
> with -snes_view ::ascii_info_detail and it does show that the prefixes
> are properly changed. It also shows that the SNES type for the last domain
> was successfully changed. But for some reason, I only have access to the
> SNES viewer options during runtime. For example, if I run mpiexec -n 4
> ./test1 -sub_0_ksp_type gmres -help | grep sub_0 I get the output:
>
> Viewer (-sub_0_snes_convergence_estimate) options:
>   -sub_0_snes_convergence_estimate ascii[:[filename][:[format][:append]]]:
> Prints object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate
> binary[:[filename][:[format][:append]]]: Saves object to a binary file
> (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate draw[:[drawtype][:filename|format]]
> Draws object (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate socket[:port]: Pushes object to a Unix
> socket (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate saws[:communicatorname]: Publishes
> object to SAWs (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_view_pre) options:
>   -sub_0_snes_view_pre ascii[:[filename][:[format][:append]]]: Prints
> object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre binary[:[filename][:[format][:append]]]: Saves
> object to a binary file (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre draw[:[drawtype][:filename|format]] Draws object
> (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre socket[:port]: Pushes object to a Unix socket
> (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre saws[:communicatorname]: Publishes object to SAWs
> (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_test_jacobian_view) options:
>   -sub_0_snes_test_jacobian_view ascii[:[filename][:[format][:append]]]:
> Prints object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view binary[:[filename][:[format][:append]]]:
> Saves object to a binary file (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view draw[:[drawtype][:filename|format]] Draws
> object (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view socket[:port]: Pushes object to a Unix
> socket (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view saws[:communicatorname]: Publishes object
> to SAWs (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_test_jacobian_display) options:
>   -sub_0_snes_test_jacobian_display
> ascii[:[filename][:[format][:append]]]: Prints object to stdout

Re: [petsc-users] Parallel matrix-vector product with Matrix Shell

2022-06-17 Thread Jose E. Roman

You can use VecGetOwnershipRange() to determine the range of global indices 
corresponding to the local portion of a vector, and VecGetArray() to access the 
values. In SLEPc, you can assume that X and Y will have the same parallel 
distribution.

For an example of a shell matrix that implements the matrix-vector product in 
parallel, have a look at this: 
https://slepc.upv.es/documentation/current/src/nep/tutorials/ex21.c.html
It is a simple tridiagonal example, where neighborwise communication is done 
with two calls to MPI_Sendrecv().

Jose


> El 17 jun 2022, a las 17:21, Mario Rossi  escribió:
> 
> I need to find the largest eigenvalues (say the first three) of a very large 
> matrix and I am using
> a combination of PetSc and SLEPc. In particular, I am using a shell matrix. I 
> wrote a "custom"
> matrix-vector product and everything works fine in serial (one task) mode for 
> a "small" case.
> For the real case, I need multiple (at least 128) tasks for memory reasons so 
> I need a parallel variant of the custom matrix-vector product. I know exactly 
> how to write the parallel variant 
> (in plain MPI) but I am, somehow, blocked because it is not clear to me what 
> each task receives
> and what is expected to provide in the parallel matrix-vector product.
> More in detail, with a single task, the function receives the full X vector 
> and is expected to provide the full Y vector resulting from Y=A*X. 
> What does it happen with multiple tasks? If I understand correctly
> in the matrix shell definition, I can choose to split the matrix into blocks 
> of rows so that the matrix-vector function should compute a block of elements 
> of the vector Y but does it receive only the corresponding subset of the X 
> (input vector)? (this is what I guess happens) and in output, does
> each task return its subset of elements of Y as if it were the whole array 
> and then PetSc manages all the subsets? Is there anyone who has a working 
> example of a parallel matrix-vector product for matrix shell?
> Thanks in advance for any help you can provide!
> Mario
> i
>

[petsc-users] Parallel matrix-vector product with Matrix Shell

2022-06-17 Thread Mario Rossi

I need to find the largest eigenvalues (say the first three) of a very
large matrix and I am using
a combination of PetSc and SLEPc. In particular, I am using a shell matrix.
I wrote a "custom"
matrix-vector product and everything works fine in serial (one task) mode
for a "small" case.
For the real case, I need multiple (at least 128) tasks for memory reasons
so I need a parallel variant of the custom matrix-vector product. I know
exactly how to write the parallel variant
(in plain MPI) but I am, somehow, blocked because it is not clear to me
what each task receives
and what is expected to provide in the parallel matrix-vector product.
More in detail, with a single task, the function receives the full X vector
and is expected to provide the full Y vector resulting from Y=A*X.
What does it happen with multiple tasks? If I understand correctly
in the matrix shell definition, I can choose to split the matrix into
blocks of rows so that the matrix-vector function should compute a block of
elements of the vector Y but does it receive only the corresponding subset
of the X (input vector)? (this is what I guess happens) and in output, does
each task return its subset of elements of Y as if it were the whole array
and then PetSc manages all the subsets? Is there anyone who has a working
example of a parallel matrix-vector product for matrix shell?
Thanks in advance for any help you can provide!
Mario
i

Re: [petsc-users] Customizing NASM subsnes

2022-06-17 Thread Takahashi, Tadanaga

Ahh, I understand now. I got rid of the loop. Adding
SNESSetFromOptions(subsnes) right after
SNESSetOptionsPrefix(subsnes,prefix) did not fix the issue.

On Fri, Jun 17, 2022 at 11:02 AM Barry Smith  wrote:

>
>   You do not need the loop over size. Each rank sets options and options
> prefix for its local objects and never anyone elses.
>
>char prefix[10];
>
>sprintf(prefix,"sub_%d_",rank);
>
>SNESNASMGetSNES(snes,0,);
>
>  if (rank  SNESSetType(subsnes,SNESNEWTONLS); CHKERRQ(ierr); // newton
> for regular domains
>  } else {
> SNESSetType(subsnes,SNESFAS); CHKERRQ(ierr); // fas for last
> domain
>  }
>  SNESSetOptionsPrefix(subsnes,prefix);
>   }
>}
>
>
>To get the prefix to work try calling   SNESSetFromOptions(subsnes);
> immediately after your SNESSetOptionsPrefix(subsnes,prefix); call
>
>Matt, it looks like there may be a bug in NASM, except in one
> particular case, it never calls SNESSetFromOptions() on the subsenses.
>
>   Barry
>
>
> On Jun 17, 2022, at 10:47 AM, Takahashi, Tadanaga  wrote:
>
> Thank you. I am now able to pull each subsnes, change its snes type
> through the API, and set a prefix. This is my updated code:
>
>SNES   snes, subsnes;
>PetscMPIIntrank, size;
>...
>ierr = SNESCreate(PETSC_COMM_WORLD,); CHKERRQ(ierr);
>ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);
>ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);
>...
>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>ierr = SNESSetUp(snes); CHKERRQ(ierr);
>PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);
>PetscBarrier(NULL);
>for (i=0; i   char prefix[10];
>   sprintf(prefix,"sub_%d_",i);
>   if(i==rank) {
>  ierr = SNESNASMGetNumber(snes,);
>  printf("rank = %d has %d block(s)\n",i,Nd);
>  if (i  SNESNASMGetSNES(snes,0,);
> SNESSetType(subsnes,SNESNEWTONLS); CHKERRQ(ierr); // newton
> for regular domains
>  } else {
> SNESNASMGetSNES(snes,0,);
> SNESSetType(subsnes,SNESFAS); CHKERRQ(ierr); // fas for last
> domain
>  }
>  SNESSetOptionsPrefix(subsnes,prefix);
>   }
>}
>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>...
>ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr);
>
> However, I still cannot change SNES, KSP, and PC types for
> individual domains through the command arguments. I checked the subdomains
> with -snes_view ::ascii_info_detail and it does show that the prefixes
> are properly changed. It also shows that the SNES type for the last domain
> was successfully changed. But for some reason, I only have access to the
> SNES viewer options during runtime. For example, if I run mpiexec -n 4
> ./test1 -sub_0_ksp_type gmres -help | grep sub_0 I get the output:
>
> Viewer (-sub_0_snes_convergence_estimate) options:
>   -sub_0_snes_convergence_estimate ascii[:[filename][:[format][:append]]]:
> Prints object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate
> binary[:[filename][:[format][:append]]]: Saves object to a binary file
> (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate draw[:[drawtype][:filename|format]]
> Draws object (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate socket[:port]: Pushes object to a Unix
> socket (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate saws[:communicatorname]: Publishes
> object to SAWs (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_view_pre) options:
>   -sub_0_snes_view_pre ascii[:[filename][:[format][:append]]]: Prints
> object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre binary[:[filename][:[format][:append]]]: Saves
> object to a binary file (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre draw[:[drawtype][:filename|format]] Draws object
> (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre socket[:port]: Pushes object to a Unix socket
> (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre saws[:communicatorname]: Publishes object to SAWs
> (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_test_jacobian_view) options:
>   -sub_0_snes_test_jacobian_view ascii[:[filename][:[format][:append]]]:
> Prints object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view binary[:[filename][:[format][:append]]]:
> Saves object to a binary file (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view draw[:[drawtype][:filename|format]] Draws
> object (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view socket[:port]: Pushes object to a Unix
> socket (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view saws[:communicatorname]: Publishes object
> to SAWs (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_test_jacobian_display) options:
>   -sub_0_snes_test_jacobian_display
> ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII
> file (PetscOptionsGetViewer)
>

Re: [petsc-users] Customizing NASM subsnes

2022-06-17 Thread Barry Smith


  You do not need the loop over size. Each rank sets options and options prefix 
for its local objects and never anyone elses. 

>char prefix[10];
>sprintf(prefix,"sub_%d_",rank);
>SNESNASMGetSNES(snes,0,);
>  if (rank  SNESSetType(subsnes,SNESNEWTONLS); CHKERRQ(ierr); // newton for 
> regular domains
>  } else {
> SNESSetType(subsnes,SNESFAS); CHKERRQ(ierr); // fas for last 
> domain 
>  }
>  SNESSetOptionsPrefix(subsnes,prefix);
>   }
>}

   To get the prefix to work try calling   SNESSetFromOptions(subsnes); 
immediately after your SNESSetOptionsPrefix(subsnes,prefix); call

   Matt, it looks like there may be a bug in NASM, except in one particular 
case, it never calls SNESSetFromOptions() on the subsenses.

  Barry


> On Jun 17, 2022, at 10:47 AM, Takahashi, Tadanaga  wrote:
> 
> Thank you. I am now able to pull each subsnes, change its snes type through 
> the API, and set a prefix. This is my updated code: 
> 
>SNES   snes, subsnes;
>PetscMPIIntrank, size;
>... 
>ierr = SNESCreate(PETSC_COMM_WORLD,); CHKERRQ(ierr);
>ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);
>ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);
>... 
>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>ierr = SNESSetUp(snes); CHKERRQ(ierr);
>PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);
>PetscBarrier(NULL);
>for (i=0; i   char prefix[10];
>   sprintf(prefix,"sub_%d_",i);
>   if(i==rank) {
>  ierr = SNESNASMGetNumber(snes,);
>  printf("rank = %d has %d block(s)\n",i,Nd);
>  if (i  SNESNASMGetSNES(snes,0,);
> SNESSetType(subsnes,SNESNEWTONLS); CHKERRQ(ierr); // newton for 
> regular domains
>  } else {
> SNESNASMGetSNES(snes,0,);
> SNESSetType(subsnes,SNESFAS); CHKERRQ(ierr); // fas for last 
> domain 
>  }
>  SNESSetOptionsPrefix(subsnes,prefix);
>   }
>}
>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>... 
>ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr);
> 
> However, I still cannot change SNES, KSP, and PC types for individual domains 
> through the command arguments. I checked the subdomains with -snes_view 
> ::ascii_info_detail and it does show that the prefixes are properly changed. 
> It also shows that the SNES type for the last domain was successfully 
> changed. But for some reason, I only have access to the SNES viewer options 
> during runtime. For example, if I run mpiexec -n 4 ./test1 -sub_0_ksp_type 
> gmres -help | grep sub_0 I get the output: 
> 
> Viewer (-sub_0_snes_convergence_estimate) options:
>   -sub_0_snes_convergence_estimate ascii[:[filename][:[format][:append]]]: 
> Prints object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate binary[:[filename][:[format][:append]]]: 
> Saves object to a binary file (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate draw[:[drawtype][:filename|format]] Draws 
> object (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate socket[:port]: Pushes object to a Unix 
> socket (PetscOptionsGetViewer)
>   -sub_0_snes_convergence_estimate saws[:communicatorname]: Publishes object 
> to SAWs (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_view_pre) options:
>   -sub_0_snes_view_pre ascii[:[filename][:[format][:append]]]: Prints object 
> to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre binary[:[filename][:[format][:append]]]: Saves object 
> to a binary file (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre draw[:[drawtype][:filename|format]] Draws object 
> (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre socket[:port]: Pushes object to a Unix socket 
> (PetscOptionsGetViewer)
>   -sub_0_snes_view_pre saws[:communicatorname]: Publishes object to SAWs 
> (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_test_jacobian_view) options:
>   -sub_0_snes_test_jacobian_view ascii[:[filename][:[format][:append]]]: 
> Prints object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view binary[:[filename][:[format][:append]]]: 
> Saves object to a binary file (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view draw[:[drawtype][:filename|format]] Draws 
> object (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view socket[:port]: Pushes object to a Unix 
> socket (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_view saws[:communicatorname]: Publishes object to 
> SAWs (PetscOptionsGetViewer)
> Viewer (-sub_0_snes_test_jacobian_display) options:
>   -sub_0_snes_test_jacobian_display ascii[:[filename][:[format][:append]]]: 
> Prints object to stdout or ASCII file (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_display binary[:[filename][:[format][:append]]]: 
> Saves object to a binary file (PetscOptionsGetViewer)
>   -sub_0_snes_test_jacobian_display

Re: [petsc-users] Customizing NASM subsnes

2022-06-17 Thread Takahashi, Tadanaga

Thank you. I am now able to pull each subsnes, change its snes type through
the API, and set a prefix. This is my updated code:

   SNES   snes, subsnes;
   PetscMPIIntrank, size;
   ...
   ierr = SNESCreate(PETSC_COMM_WORLD,); CHKERRQ(ierr);
   ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);
   ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);
   ...
   ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
   ierr = SNESSetUp(snes); CHKERRQ(ierr);
   PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);
   PetscBarrier(NULL);
   for (i=0; i wrote:

>
>  MPI_Comm_size(PETSC_COMM_WORLD,);
>  MPI_Comm_rank(PETSC_COMM_WORLD,);
>
> SNESNASMGetSNES(snes,0,);
>>  char prefix[10];
>>  sprintf(prefix,"sub_%d_",rank);
>>  SNESSetOptionsPrefix(subsnes,prefix);
>>
>
>
>
> On Jun 17, 2022, at 9:35 AM, Matthew Knepley  wrote:
>
> On Fri, Jun 17, 2022 at 9:22 AM Takahashi, Tadanaga  wrote:
>
>> I'm having some trouble pulling out the subsolver. I tried to use
>> SNESNASMGetSNES in a loop over each subdomain. However I get an error when
>> I run the code with more than one MPI processors. Here is a snippet from my
>> code:
>>
>>SNES   snes, subsnes;
>>PetscMPIIntrank, size;
>>...
>>ierr = SNESCreate(PETSC_COMM_WORLD,); CHKERRQ(ierr);
>>ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);
>>ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);
>>...
>>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>>ierr = SNESSetUp(snes); CHKERRQ(ierr);
>>PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);
>>for (i=0; i>   PetscPrintf(PETSC_COMM_WORLD, "rank = %d\n",i);
>>   SNESNASMGetSNES(snes,i,);
>>   // char prefix[10];
>>   // sprintf(prefix,"sub_%d_",i);
>>   // SNESSetOptionsPrefix(subsnes,prefix);
>>}
>>...
>>ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr);
>>
>>
>> And, here is the output of the code when I run with 2 MPI procs:
>>
>
> SNESNASMGetSNES() gets the local subsolvers. It seems you only have one
> per process.
> You can check
> https://petsc.org/main/docs/manualpages/SNES/SNESNASMGetNumber/
>
> Notice that your current code will not work because, according to your
> explanation, you only want to change
> the prefix on a single rank, so you need to check the rank when you do it.
>
>   Thanks,
>
>  Matt
>
>
>> takahashi@ubuntu:~/Desktop/MA-DDM/C/Rectangle$ mpiexec -n 2 ./test1
>> Size = 2
>> rank = 0
>> rank = 1
>> [0]PETSC ERROR: - Error Message
>> --
>> [0]PETSC ERROR: Argument out of range
>> [0]PETSC ERROR: No such subsolver
>> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
>> [0]PETSC ERROR: Petsc Release Version 3.17.1, unknown
>> [0]PETSC ERROR: ./test1 on a linux-gnu-c-debug named ubuntu by takahashi
>> Fri Jun 17 06:06:38 2022
>> [0]PETSC ERROR: Configure options --with-mpi-dir=/usr --with-fc=0
>> [0]PETSC ERROR: #1 SNESNASMGetSNES() at
>> /home/takahashi/Desktop/petsc/src/snes/impls/nasm/nasm.c:923
>>
>>
>> ===
>> =   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
>> =   RANK 0 PID 976566 RUNNING AT ubuntu
>> =   KILLED BY SIGNAL: 9 (Killed)
>>
>> ===
>>
>> This error doesn't occur when I run this without MPI. However, I tried to
>> change the prefix of the subdomain to `sub_0_` but I am not able to change
>> the snes_type using this prefix. Running ./test1 -snes_view -help | grep
>> sub_0_snes_type prints nothing.
>>
>> On Thu, Jun 16, 2022 at 6:23 PM Matthew Knepley 
>> wrote:
>>
>>> On Thu, Jun 16, 2022 at 5:57 PM tt73  wrote:
>>>

 Hi,

 I am using  NASM as the outer solver for a nonlinear problem. For one
 of the subdomains, I want to run the local solve with a different set of
 options form the others. Is there any way to set options for each
 subdomain?

>>>
>>> I can see two ways:
>>>
>>>   1) Pull out the subsolver and set it using the API
>>>
>>>   2) Pull out the subsolver and give it a different prefix
>>>
>>>   Thanks,
>>>
>>>  Matt
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>> https://www.cse.buffalo.edu/~knepley/
>>> 
>>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> 
>
>
>

Re: [petsc-users] Customizing NASM subsnes

2022-06-17 Thread Barry Smith


 MPI_Comm_size(PETSC_COMM_WORLD,);
 MPI_Comm_rank(PETSC_COMM_WORLD,);
> SNESNASMGetSNES(snes,0,);
>  char prefix[10];
>  sprintf(prefix,"sub_%d_",rank);
>  SNESSetOptionsPrefix(subsnes,prefix);



> On Jun 17, 2022, at 9:35 AM, Matthew Knepley  wrote:
> 
> On Fri, Jun 17, 2022 at 9:22 AM Takahashi, Tadanaga  > wrote:
> I'm having some trouble pulling out the subsolver. I tried to use 
> SNESNASMGetSNES in a loop over each subdomain. However I get an error when I 
> run the code with more than one MPI processors. Here is a snippet from my 
> code: 
> 
>SNES   snes, subsnes;
>PetscMPIIntrank, size;
>... 
>ierr = SNESCreate(PETSC_COMM_WORLD,); CHKERRQ(ierr);
>ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);
>ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);
>... 
>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>ierr = SNESSetUp(snes); CHKERRQ(ierr);
>PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);
>for (i=0; i   PetscPrintf(PETSC_COMM_WORLD, "rank = %d\n",i);
>   SNESNASMGetSNES(snes,i,);
>   // char prefix[10];
>   // sprintf(prefix,"sub_%d_",i);
>   // SNESSetOptionsPrefix(subsnes,prefix);
>}
>... 
>ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr);
> 
> 
> And, here is the output of the code when I run with 2 MPI procs: 
> 
> SNESNASMGetSNES() gets the local subsolvers. It seems you only have one per 
> process.
> You can check https://petsc.org/main/docs/manualpages/SNES/SNESNASMGetNumber/ 
> 
> 
> Notice that your current code will not work because, according to your 
> explanation, you only want to change
> the prefix on a single rank, so you need to check the rank when you do it.
> 
>   Thanks,
> 
>  Matt
>  
> takahashi@ubuntu:~/Desktop/MA-DDM/C/Rectangle$ mpiexec -n 2 ./test1
> Size = 2
> rank = 0
> rank = 1
> [0]PETSC ERROR: - Error Message 
> --
> [0]PETSC ERROR: Argument out of range
> [0]PETSC ERROR: No such subsolver
> [0]PETSC ERROR: See https://petsc.org/release/faq/ 
>  for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.17.1, unknown 
> [0]PETSC ERROR: ./test1 on a linux-gnu-c-debug named ubuntu by takahashi Fri 
> Jun 17 06:06:38 2022
> [0]PETSC ERROR: Configure options --with-mpi-dir=/usr --with-fc=0
> [0]PETSC ERROR: #1 SNESNASMGetSNES() at 
> /home/takahashi/Desktop/petsc/src/snes/impls/nasm/nasm.c:923
> 
> ===
> =   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
> =   RANK 0 PID 976566 RUNNING AT ubuntu
> =   KILLED BY SIGNAL: 9 (Killed)
> ===
> 
> This error doesn't occur when I run this without MPI. However, I tried to 
> change the prefix of the subdomain to `sub_0_` but I am not able to change 
> the snes_type using this prefix. Running ./test1 -snes_view -help | grep 
> sub_0_snes_type prints nothing. 
> 
> On Thu, Jun 16, 2022 at 6:23 PM Matthew Knepley  > wrote:
> On Thu, Jun 16, 2022 at 5:57 PM tt73 mailto:t...@njit.edu>> 
> wrote:
> 
> Hi, 
> 
> I am using  NASM as the outer solver for a nonlinear problem. For one of the 
> subdomains, I want to run the local solve with a different set of options 
> form the others. Is there any way to set options for each subdomain? 
> 
> I can see two ways:
> 
>   1) Pull out the subsolver and set it using the API
> 
>   2) Pull out the subsolver and give it a different prefix
> 
>   Thanks,
> 
>  Matt
> 
> -- 
> What most experimenters take for granted before they begin their experiments 
> is infinitely more interesting than any results to which their experiments 
> lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/ 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments 
> is infinitely more interesting than any results to which their experiments 
> lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/

Re: [petsc-users] Customizing NASM subsnes

2022-06-17 Thread Matthew Knepley

On Fri, Jun 17, 2022 at 9:22 AM Takahashi, Tadanaga  wrote:

> I'm having some trouble pulling out the subsolver. I tried to use
> SNESNASMGetSNES in a loop over each subdomain. However I get an error when
> I run the code with more than one MPI processors. Here is a snippet from my
> code:
>
>SNES   snes, subsnes;
>PetscMPIIntrank, size;
>...
>ierr = SNESCreate(PETSC_COMM_WORLD,); CHKERRQ(ierr);
>ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);
>ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);
>...
>ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
>ierr = SNESSetUp(snes); CHKERRQ(ierr);
>PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);
>for (i=0; i   PetscPrintf(PETSC_COMM_WORLD, "rank = %d\n",i);
>   SNESNASMGetSNES(snes,i,);
>   // char prefix[10];
>   // sprintf(prefix,"sub_%d_",i);
>   // SNESSetOptionsPrefix(subsnes,prefix);
>}
>...
>ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr);
>
>
> And, here is the output of the code when I run with 2 MPI procs:
>

SNESNASMGetSNES() gets the local subsolvers. It seems you only have one per
process.
You can check
https://petsc.org/main/docs/manualpages/SNES/SNESNASMGetNumber/

Notice that your current code will not work because, according to your
explanation, you only want to change
the prefix on a single rank, so you need to check the rank when you do it.

  Thanks,

 Matt


> takahashi@ubuntu:~/Desktop/MA-DDM/C/Rectangle$ mpiexec -n 2 ./test1
> Size = 2
> rank = 0
> rank = 1
> [0]PETSC ERROR: - Error Message
> --
> [0]PETSC ERROR: Argument out of range
> [0]PETSC ERROR: No such subsolver
> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.17.1, unknown
> [0]PETSC ERROR: ./test1 on a linux-gnu-c-debug named ubuntu by takahashi
> Fri Jun 17 06:06:38 2022
> [0]PETSC ERROR: Configure options --with-mpi-dir=/usr --with-fc=0
> [0]PETSC ERROR: #1 SNESNASMGetSNES() at
> /home/takahashi/Desktop/petsc/src/snes/impls/nasm/nasm.c:923
>
>
> ===
> =   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
> =   RANK 0 PID 976566 RUNNING AT ubuntu
> =   KILLED BY SIGNAL: 9 (Killed)
>
> ===
>
> This error doesn't occur when I run this without MPI. However, I tried to
> change the prefix of the subdomain to `sub_0_` but I am not able to change
> the snes_type using this prefix. Running ./test1 -snes_view -help | grep
> sub_0_snes_type prints nothing.
>
> On Thu, Jun 16, 2022 at 6:23 PM Matthew Knepley  wrote:
>
>> On Thu, Jun 16, 2022 at 5:57 PM tt73  wrote:
>>
>>>
>>> Hi,
>>>
>>> I am using  NASM as the outer solver for a nonlinear problem. For one of
>>> the subdomains, I want to run the local solve with a different set of
>>> options form the others. Is there any way to set options for each
>>> subdomain?
>>>
>>
>> I can see two ways:
>>
>>   1) Pull out the subsolver and set it using the API
>>
>>   2) Pull out the subsolver and give it a different prefix
>>
>>   Thanks,
>>
>>  Matt
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> 
>>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/

Re: [petsc-users] Customizing NASM subsnes

2022-06-17 Thread Takahashi, Tadanaga

I'm having some trouble pulling out the subsolver. I tried to use
SNESNASMGetSNES in a loop over each subdomain. However I get an error when
I run the code with more than one MPI processors. Here is a snippet from my
code:

   SNES   snes, subsnes;
   PetscMPIIntrank, size;
   ...
   ierr = SNESCreate(PETSC_COMM_WORLD,); CHKERRQ(ierr);
   ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr);
   ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr);
   ...
   ierr = SNESSetFromOptions(snes); CHKERRQ(ierr);
   ierr = SNESSetUp(snes); CHKERRQ(ierr);
   PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size);
   for (i=0; ihttps://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.17.1, unknown
[0]PETSC ERROR: ./test1 on a linux-gnu-c-debug named ubuntu by takahashi
Fri Jun 17 06:06:38 2022
[0]PETSC ERROR: Configure options --with-mpi-dir=/usr --with-fc=0
[0]PETSC ERROR: #1 SNESNASMGetSNES() at
/home/takahashi/Desktop/petsc/src/snes/impls/nasm/nasm.c:923

===
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   RANK 0 PID 976566 RUNNING AT ubuntu
=   KILLED BY SIGNAL: 9 (Killed)
===

This error doesn't occur when I run this without MPI. However, I tried to
change the prefix of the subdomain to `sub_0_` but I am not able to change
the snes_type using this prefix. Running ./test1 -snes_view -help | grep
sub_0_snes_type prints nothing.

On Thu, Jun 16, 2022 at 6:23 PM Matthew Knepley  wrote:

> On Thu, Jun 16, 2022 at 5:57 PM tt73  wrote:
>
>>
>> Hi,
>>
>> I am using  NASM as the outer solver for a nonlinear problem. For one of
>> the subdomains, I want to run the local solve with a different set of
>> options form the others. Is there any way to set options for each
>> subdomain?
>>
>
> I can see two ways:
>
>   1) Pull out the subsolver and set it using the API
>
>   2) Pull out the subsolver and give it a different prefix
>
>   Thanks,
>
>  Matt
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> 
>

Re: [petsc-users] How to find the map between the high order coordinates of DMPlex and vertex numbering?

2022-06-17 Thread Zongze Yang

I tried the projection operation. However, it seems that the projection
gives the wrong solution. After projection, the bounding box is changed!
See logs below.

First, I patch the petsc4py by adding `DMProjectCoordinates`:
```
diff --git a/src/binding/petsc4py/src/PETSc/DM.pyx
b/src/binding/petsc4py/src/PETSc/DM.pyx
index d8a58d183a..dbcdb280f1 100644
--- a/src/binding/petsc4py/src/PETSc/DM.pyx
+++ b/src/binding/petsc4py/src/PETSc/DM.pyx
@@ -307,6 +307,12 @@ cdef class DM(Object):
 PetscINCREF(c.obj)
 return c

+def projectCoordinates(self, FE fe=None):
+if fe is None:
+CHKERR( DMProjectCoordinates(self.dm, NULL) )
+else:
+CHKERR( DMProjectCoordinates(self.dm, fe.fe) )
+
 def getBoundingBox(self):
 cdef PetscInt i,dim=0
 CHKERR( DMGetCoordinateDim(self.dm, ) )
diff --git a/src/binding/petsc4py/src/PETSc/petscdm.pxi
b/src/binding/petsc4py/src/PETSc/petscdm.pxi
index 514b6fa472..c778e39884 100644
--- a/src/binding/petsc4py/src/PETSc/petscdm.pxi
+++ b/src/binding/petsc4py/src/PETSc/petscdm.pxi
@@ -90,6 +90,7 @@ cdef extern from * nogil:
 int DMGetCoordinateDim(PetscDM,PetscInt*)
 int DMSetCoordinateDim(PetscDM,PetscInt)
 int DMLocalizeCoordinates(PetscDM)
+int DMProjectCoordinates(PetscDM, PetscFE)

 int DMCreateInterpolation(PetscDM,PetscDM,PetscMat*,PetscVec*)
 int DMCreateInjection(PetscDM,PetscDM,PetscMat*)
```

Then in python, I load a mesh and project the coordinates to P2:
```
import firedrake as fd
from firedrake.petsc import PETSc

# plex = fd.mesh._from_gmsh('test-fd-load-p2.msh')
plex = fd.mesh._from_gmsh('test-fd-load-p2-rect.msh')
print('old bbox:', plex.getBoundingBox())

dim = plex.getDimension()
# (dim,  nc, isSimplex, k,  qorder,
comm=None)
fe_new = PETSc.FE().createLagrange(dim, dim,  True, 2, PETSc.DETERMINE)
plex.projectCoordinates(fe_new)
fe_new.view()

print('new bbox:', plex.getBoundingBox())
```

The output is (The bounding box is changed!)
```

old bbox: ((0.0, 1.0), (0.0, 1.0), (0.0, 1.0))
PetscFE Object: P2 1 MPI processes
  type: basic
  Basic Finite Element in 3 dimensions with 3 components
  PetscSpace Object: P2 1 MPI processes
type: sum
Space in 3 variables with 3 components, size 30
Sum space of 3 concatenated subspaces (all identical)
  PetscSpace Object: sum component (sumcomp_) 1 MPI processes
type: poly
Space in 3 variables with 1 components, size 10
Polynomial space of degree 2
  PetscDualSpace Object: P2 1 MPI processes
type: lagrange
Dual space with 3 components, size 30
Continuous Lagrange dual space
Quadrature of order 5 on 27 points (dim 3)
new bbox: ((-6.530133708576188e-17, 36.30670832662781),
(-3.899962995254311e-17, 36.2406171632539), (-8.8036464152166e-17,
36.111577025012224))

```


By the way, for the original DG coordinates, where can I find the
relation of the closure and the order of the dofs for the cell?


Thanks!


  Zongze



Matthew Knepley  于2022年6月17日周五 01:11写道：

> On Thu, Jun 16, 2022 at 12:06 PM Zongze Yang  wrote:
>
>>
>>
>> 在 2022年6月16日，23:22，Matthew Knepley  写道：
>>
>> 
>> On Thu, Jun 16, 2022 at 11:11 AM Zongze Yang 
>> wrote:
>>
>>> Hi, if I load a `gmsh` file with second-order elements, the coordinates
>>> will be stored in a DG-P2 space. After obtaining the coordinates of a cell,
>>> how can I map the coordinates to vertex and edge?
>>>
>>
>> By default, they are stored as P2, not DG.
>>
>>
>> I checked the coordinates vector, and found the dogs only defined on cell
>> other than vertex and edge, so I said they are stored as DG.
>> Then the function DMPlexVecGetClosure
>>  seems 
>> return
>> the coordinates in lex order.
>>
>> Some code in reading gmsh file reads that
>>
>>
>> 1756: if (isSimplex) continuity = PETSC_FALSE
>> ; /* XXX FIXME
>> Requires DMPlexSetClosurePermutationLexicographic() */
>>
>>
>> 1758: GmshCreateFE(comm, NULL, isSimplex, continuity, nodeType, dim,
>> coordDim, order, )
>>
>>
>> The continuity is set to false for simplex.
>>
>
> Oh, yes. That needs to be fixed. For now, you can just project it to P2 if
> you want using
>
>   https://petsc.org/main/docs/manualpages/DM/DMProjectCoordinates/
>
>   Thanks,
>
>  Matt
>
>
>> Thanks,
>> Zongze
>>
>> You can ask for the coordinates of a vertex or an edge directly using
>>
>>   https://petsc.org/main/docs/manualpages/DMPLEX/DMPlexPointLocalRead/
>>
>> by giving the vertex or edge point. You can get all the coordinates on a
>> cell, in the closure order, using
>>
>>   https://petsc.org/main/docs/manualpages/DMPLEX/DMPlexVecGetClosure/
>>
>>   Thanks,
>>
>>  Matt
>>
>>
>>> Below is some code load the gmsh file, I want to know the relation
>>> between `cl` and `cell_coords`.
>>>
>>> ```
>>> import firedrake as fd
>>> import numpy as np

Re: [petsc-users] Pestc-matlab issue

[petsc-users] Pestc-matlab issue

Re: [petsc-users] Parallel matrix-vector product with Matrix Shell

Re: [petsc-users] Customizing NASM subsnes

Re: [petsc-users] Parallel matrix-vector product with Matrix Shell

[petsc-users] Parallel matrix-vector product with Matrix Shell

Re: [petsc-users] Customizing NASM subsnes

Re: [petsc-users] Customizing NASM subsnes

Re: [petsc-users] Customizing NASM subsnes

Re: [petsc-users] Customizing NASM subsnes

Re: [petsc-users] Customizing NASM subsnes

Re: [petsc-users] Customizing NASM subsnes

Re: [petsc-users] How to find the map between the high order coordinates of DMPlex and vertex numbering?

13 matches

Site Navigation

Mail list logo

Footer information