Thank you. I am now able to pull each subsnes, change its snes type through the API, and set a prefix. This is my updated code:
SNES snes, subsnes; PetscMPIInt rank, size; ... ierr = SNESCreate(PETSC_COMM_WORLD,&snes); CHKERRQ(ierr); ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr); ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr); ... ierr = SNESSetFromOptions(snes); CHKERRQ(ierr); ierr = SNESSetUp(snes); CHKERRQ(ierr); PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size); PetscBarrier(NULL); for (i=0; i<size; i++) { char prefix[10]; sprintf(prefix,"sub_%d_",i); if(i==rank) { ierr = SNESNASMGetNumber(snes,&Nd); printf("rank = %d has %d block(s)\n",i,Nd); if (i <size-1) { SNESNASMGetSNES(snes,0,&subsnes); SNESSetType(subsnes,SNESNEWTONLS); CHKERRQ(ierr); // newton for regular domains } else { SNESNASMGetSNES(snes,0,&subsnes); SNESSetType(subsnes,SNESFAS); CHKERRQ(ierr); // fas for last domain } SNESSetOptionsPrefix(subsnes,prefix); } } ierr = SNESSetFromOptions(snes); CHKERRQ(ierr); ... ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr); However, I still cannot change SNES, KSP, and PC types for individual domains through the command arguments. I checked the subdomains with -snes_view ::ascii_info_detail and it does show that the prefixes are properly changed. It also shows that the SNES type for the last domain was successfully changed. But for some reason, I only have access to the SNES viewer options during runtime. For example, if I run mpiexec -n 4 ./test1 -sub_0_ksp_type gmres -help | grep sub_0 I get the output: Viewer (-sub_0_snes_convergence_estimate) options: -sub_0_snes_convergence_estimate ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII file (PetscOptionsGetViewer) -sub_0_snes_convergence_estimate binary[:[filename][:[format][:append]]]: Saves object to a binary file (PetscOptionsGetViewer) -sub_0_snes_convergence_estimate draw[:[drawtype][:filename|format]] Draws object (PetscOptionsGetViewer) -sub_0_snes_convergence_estimate socket[:port]: Pushes object to a Unix socket (PetscOptionsGetViewer) -sub_0_snes_convergence_estimate saws[:communicatorname]: Publishes object to SAWs (PetscOptionsGetViewer) Viewer (-sub_0_snes_view_pre) options: -sub_0_snes_view_pre ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII file (PetscOptionsGetViewer) -sub_0_snes_view_pre binary[:[filename][:[format][:append]]]: Saves object to a binary file (PetscOptionsGetViewer) -sub_0_snes_view_pre draw[:[drawtype][:filename|format]] Draws object (PetscOptionsGetViewer) -sub_0_snes_view_pre socket[:port]: Pushes object to a Unix socket (PetscOptionsGetViewer) -sub_0_snes_view_pre saws[:communicatorname]: Publishes object to SAWs (PetscOptionsGetViewer) Viewer (-sub_0_snes_test_jacobian_view) options: -sub_0_snes_test_jacobian_view ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII file (PetscOptionsGetViewer) -sub_0_snes_test_jacobian_view binary[:[filename][:[format][:append]]]: Saves object to a binary file (PetscOptionsGetViewer) -sub_0_snes_test_jacobian_view draw[:[drawtype][:filename|format]] Draws object (PetscOptionsGetViewer) -sub_0_snes_test_jacobian_view socket[:port]: Pushes object to a Unix socket (PetscOptionsGetViewer) -sub_0_snes_test_jacobian_view saws[:communicatorname]: Publishes object to SAWs (PetscOptionsGetViewer) Viewer (-sub_0_snes_test_jacobian_display) options: -sub_0_snes_test_jacobian_display ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII file (PetscOptionsGetViewer) -sub_0_snes_test_jacobian_display binary[:[filename][:[format][:append]]]: Saves object to a binary file (PetscOptionsGetViewer) -sub_0_snes_test_jacobian_display draw[:[drawtype][:filename|format]] Draws object (PetscOptionsGetViewer) -sub_0_snes_test_jacobian_display socket[:port]: Pushes object to a Unix socket (PetscOptionsGetViewer) -sub_0_snes_test_jacobian_display saws[:communicatorname]: Publishes object to SAWs (PetscOptionsGetViewer) Viewer (-sub_0_ksp_converged_reason) options: -sub_0_ksp_converged_reason ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII file (PetscOptionsGetViewer) -sub_0_ksp_converged_reason binary[:[filename][:[format][:append]]]: Saves object to a binary file (PetscOptionsGetViewer) -sub_0_ksp_converged_reason draw[:[drawtype][:filename|format]] Draws object (PetscOptionsGetViewer) -sub_0_ksp_converged_reason socket[:port]: Pushes object to a Unix socket (PetscOptionsGetViewer) -sub_0_ksp_converged_reason saws[:communicatorname]: Publishes object to SAWs (PetscOptionsGetViewer) Viewer (-sub_0_snes_converged_reason) options: -sub_0_snes_converged_reason ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII file (PetscOptionsGetViewer) -sub_0_snes_converged_reason binary[:[filename][:[format][:append]]]: Saves object to a binary file (PetscOptionsGetViewer) -sub_0_snes_converged_reason draw[:[drawtype][:filename|format]] Draws object (PetscOptionsGetViewer) -sub_0_snes_converged_reason socket[:port]: Pushes object to a Unix socket (PetscOptionsGetViewer) -sub_0_snes_converged_reason saws[:communicatorname]: Publishes object to SAWs (PetscOptionsGetViewer) Viewer (-sub_0_snes_view) options: -sub_0_snes_view ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII file (PetscOptionsGetViewer) -sub_0_snes_view binary[:[filename][:[format][:append]]]: Saves object to a binary file (PetscOptionsGetViewer) -sub_0_snes_view draw[:[drawtype][:filename|format]] Draws object (PetscOptionsGetViewer) -sub_0_snes_view socket[:port]: Pushes object to a Unix socket (PetscOptionsGetViewer) -sub_0_snes_view saws[:communicatorname]: Publishes object to SAWs (PetscOptionsGetViewer) Viewer (-sub_0_snes_view_solution) options: -sub_0_snes_view_solution ascii[:[filename][:[format][:append]]]: Prints object to stdout or ASCII file (PetscOptionsGetViewer) -sub_0_snes_view_solution binary[:[filename][:[format][:append]]]: Saves object to a binary file (PetscOptionsGetViewer) -sub_0_snes_view_solution draw[:[drawtype][:filename|format]] Draws object (PetscOptionsGetViewer) -sub_0_snes_view_solution socket[:port]: Pushes object to a Unix socket (PetscOptionsGetViewer) -sub_0_snes_view_solution saws[:communicatorname]: Publishes object to SAWs (PetscOptionsGetViewer) Option left: name:-sub_0_ksp_type value: gmres Do you know what could be causing this? On Fri, Jun 17, 2022 at 10:00 AM Barry Smith <bsm...@petsc.dev> wrote: > > MPI_Comm_size(PETSC_COMM_WORLD,&size); > MPI_Comm_rank(PETSC_COMM_WORLD,&rank); > > SNESNASMGetSNES(snes,0,&subsnes); >> char prefix[10]; >> sprintf(prefix,"sub_%d_",rank); >> SNESSetOptionsPrefix(subsnes,prefix); >> > > > > On Jun 17, 2022, at 9:35 AM, Matthew Knepley <knep...@gmail.com> wrote: > > On Fri, Jun 17, 2022 at 9:22 AM Takahashi, Tadanaga <t...@njit.edu> wrote: > >> I'm having some trouble pulling out the subsolver. I tried to use >> SNESNASMGetSNES in a loop over each subdomain. However I get an error when >> I run the code with more than one MPI processors. Here is a snippet from my >> code: >> >> SNES snes, subsnes; >> PetscMPIInt rank, size; >> ... >> ierr = SNESCreate(PETSC_COMM_WORLD,&snes); CHKERRQ(ierr); >> ierr = SNESSetType(snes,SNESNASM); CHKERRQ(ierr); >> ierr = SNESNASMSetType(snes,PC_ASM_RESTRICT); CHKERRQ(ierr); >> ... >> ierr = SNESSetFromOptions(snes); CHKERRQ(ierr); >> ierr = SNESSetUp(snes); CHKERRQ(ierr); >> PetscPrintf(PETSC_COMM_WORLD, "Size = %d\n",size); >> for (i=0; i<size; i++) { >> PetscPrintf(PETSC_COMM_WORLD, "rank = %d\n",i); >> SNESNASMGetSNES(snes,i,&subsnes); >> // char prefix[10]; >> // sprintf(prefix,"sub_%d_",i); >> // SNESSetOptionsPrefix(subsnes,prefix); >> } >> ... >> ierr = SNESSolve(snes,NULL,u_initial); CHKERRQ(ierr); >> >> >> And, here is the output of the code when I run with 2 MPI procs: >> > > SNESNASMGetSNES() gets the local subsolvers. It seems you only have one > per process. > You can check > https://petsc.org/main/docs/manualpages/SNES/SNESNASMGetNumber/ > > Notice that your current code will not work because, according to your > explanation, you only want to change > the prefix on a single rank, so you need to check the rank when you do it. > > Thanks, > > Matt > > >> takahashi@ubuntu:~/Desktop/MA-DDM/C/Rectangle$ mpiexec -n 2 ./test1 >> Size = 2 >> rank = 0 >> rank = 1 >> [0]PETSC ERROR: --------------------- Error Message >> -------------------------------------------------------------- >> [0]PETSC ERROR: Argument out of range >> [0]PETSC ERROR: No such subsolver >> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. >> [0]PETSC ERROR: Petsc Release Version 3.17.1, unknown >> [0]PETSC ERROR: ./test1 on a linux-gnu-c-debug named ubuntu by takahashi >> Fri Jun 17 06:06:38 2022 >> [0]PETSC ERROR: Configure options --with-mpi-dir=/usr --with-fc=0 >> [0]PETSC ERROR: #1 SNESNASMGetSNES() at >> /home/takahashi/Desktop/petsc/src/snes/impls/nasm/nasm.c:923 >> >> >> =================================================================================== >> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >> = RANK 0 PID 976566 RUNNING AT ubuntu >> = KILLED BY SIGNAL: 9 (Killed) >> >> =================================================================================== >> >> This error doesn't occur when I run this without MPI. However, I tried to >> change the prefix of the subdomain to `sub_0_` but I am not able to change >> the snes_type using this prefix. Running ./test1 -snes_view -help | grep >> sub_0_snes_type prints nothing. >> >> On Thu, Jun 16, 2022 at 6:23 PM Matthew Knepley <knep...@gmail.com> >> wrote: >> >>> On Thu, Jun 16, 2022 at 5:57 PM tt73 <t...@njit.edu> wrote: >>> >>>> >>>> Hi, >>>> >>>> I am using NASM as the outer solver for a nonlinear problem. For one >>>> of the subdomains, I want to run the local solve with a different set of >>>> options form the others. Is there any way to set options for each >>>> subdomain? >>>> >>> >>> I can see two ways: >>> >>> 1) Pull out the subsolver and set it using the API >>> >>> 2) Pull out the subsolver and give it a different prefix >>> >>> Thanks, >>> >>> Matt >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> https://www.cse.buffalo.edu/~knepley/ >>> <http://www.cse.buffalo.edu/~knepley/> >>> >> > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> > > >