Hi guys, Since I found out that with TSSetEquationType things work better I have made some tests with a system of 2 PDEs (diff equa) and 1 ODE, with different boundary conditions (Dirichlet and Neumann). I played around with extreme values of the Fields and diffusion coefficients and the conclusion is that now it works well; and I think the code is ready for the implementation of dof >> 1.
So far I made the tests with 1 proc but with doff >> 1 I guess it will become very time consuming and that it would be better to use several procs. Therefore, I have few questions: - Basically, what should I change in my code to prepare it for several procs ? - Should I change the matrix type ? In the current version of the code I declared the matrix with: ierr = DMCreateMatrix(da,MATAIJ,&J); Is it ok ? - I think I should define the pattern for diagonal and offdiag blocks ierr = DMDASetBlockFills(da,dfill,ofill); It's already done but I understand it only makes sense when using MPIAIJ matrix format. - Regarding the solver of the system, what should I change ? For the moment I use the defaults: KSPGMRES and PCILU. - To understand what I'm doing, when using several procs, how is actually solved the system ? Is the whole system solved in parallel or only IFunction and IJacobian are evaluated in parallel ? Thanks in advance for your help. Christophe CIEMAT Laboratorio Nacional de Fusión por Confinamiento Magnético Unidad de Materiales Edificio 2 - Planta 0 - Despacho 28m Avenida Complutense 40, 28040 Madrid, Spain Tel: +34 91496 2582 Fax: +34 91346 6442 -- Q Por favor, piense en el medio ambiente antes de imprimir este mensaje. Please consider the environment before printing this email. On Thu, Oct 24, 2013 at 12:09 PM, Christophe Ortiz < [email protected]> wrote: > > On Wed, Oct 23, 2013 at 7:14 PM, Jed Brown <[email protected]> wrote: > >> Christophe Ortiz <[email protected]> writes: >> >> >> That does not sound right. Can you send a reproducible test case (best >> >> as a patch to PETSc)? >> >> >> > >> > What do you mean by patch to PETSc ? How do I do that ? >> >> See "contributing a small patch" (or any of the other workflows). >> >> https://bitbucket.org/petsc/petsc/wiki/Home > > > Ok, I will read the instructions and try. > > >> >> >> > Sorry about that. Here is the entire error message: >> > >> > [0]PETSC ERROR: No support for this operation for this object type! >> > [0]PETSC ERROR: Mat type mffd! >> > [0]PETSC ERROR: >> > ------------------------------------------------------------------------ >> > [0]PETSC ERROR: Petsc Release Version 3.4.1, Jun, 10, 2013 >> > [0]PETSC ERROR: See docs/changes/index.html for recent updates. >> > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. >> > [0]PETSC ERROR: See docs/index.html for manual pages. >> > [0]PETSC ERROR: >> > ------------------------------------------------------------------------ >> > [0]PETSC ERROR: ./diffusion_c on a ifort-icc-nompi-double-debug named >> > mazinger.ciemat.es by u5751 Wed Oct 23 12:46:04 2013 >> > [0]PETSC ERROR: Libraries linked from >> > /home/u5751/petsc-3.4.1/ifort-icc-nompi-double-debug/lib >> > [0]PETSC ERROR: Configure run at Mon Oct 14 05:43:50 2013 >> > [0]PETSC ERROR: Configure options --with-mpi=0 --with-fc=ifort >> > --with-cc=icc --with-debugging=1 --with-scalar-type=real >> > --with-precision=double --with-blas-lapack-dir=/opt/intel/mkl >> > [0]PETSC ERROR: >> > ------------------------------------------------------------------------ >> > [0]PETSC ERROR: MatZeroEntries() line 5189 in src/mat/interface/matrix.c >> > [0]PETSC ERROR: FormIJacobian() line 689 in >> > src/ts/examples/tutorials/diffusion.c >> >> This is your code calling MatZeroEntries on the Amat (first Mat >> argument). The convention is to assemble into Pmat (the preconditioning >> matrix; second Mat argument). >> > > I assemble first Pmat but in my code MatZeroEntries was used with Amat, > which PETSc did not appreciate... > I changed that and here is the output. While without these options it > converges--though giving solution with oscillations--now with > -ts_arkimex_fully_implicit -snes_mf_operator it does not converge at all. > No step is accepted, not even the first one: > > TSAdapt 'basic': step 0 stage rejected t=0 + 1.000e-12 > retrying with dt=2.500e-13 > TSAdapt 'basic': step 0 stage rejected t=0 + 2.500e-13 > retrying with dt=6.250e-14 > TSAdapt 'basic': step 0 stage rejected t=0 + 6.250e-14 > retrying with dt=1.562e-14 > TSAdapt 'basic': step 0 stage rejected t=0 + 1.562e-14 > retrying with dt=3.906e-15 > > ... .... > TSAdapt 'basic': step 0 stage rejected t=0 +2.888e-287 > retrying with dt=7.221e-288 > TSAdapt 'basic': step 0 stage rejected t=0 +7.221e-288 > retrying with dt=1.805e-288 > TSAdapt 'basic': step 0 stage rejected t=0 +1.805e-288 > retrying with dt=4.513e-289 > [0]PETSC ERROR: --------------------- Error Message > ------------------------------------ > [0]PETSC ERROR: Floating point exception! > [0]PETSC ERROR: Vec entry at local location 1103 is not-a-number or > infinite at end of function: Parameter number 3! > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: Petsc Release Version 3.4.1, Jun, 10, 2013 > [0]PETSC ERROR: See docs/changes/index.html for recent updates. > [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. > [0]PETSC ERROR: See docs/index.html for manual pages. > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: ./diffusion_c on a ifort-icc-nompi-double-debug named > mazinger.ciemat.es by u5751 Thu Oct 24 03:37:56 2013 > [0]PETSC ERROR: Libraries linked from > /home/u5751/petsc-3.4.1/ifort-icc-nompi-double-debug/lib > [0]PETSC ERROR: Configure run at Mon Oct 14 05:43:50 2013 > [0]PETSC ERROR: Configure options --with-mpi=0 --with-fc=ifort > --with-cc=icc --with-debugging=1 --with-scalar-type=real > --with-precision=double --with-blas-lapack-dir=/opt/intel/mkl > [0]PETSC ERROR: > ------------------------------------------------------------------------ > [0]PETSC ERROR: VecValidValues() line 30 in src/vec/vec/interface/rvector.c > [0]PETSC ERROR: SNESComputeFunction() line 1998 in > src/snes/interface/snes.c > [0]PETSC ERROR: SNESSolve_NEWTONLS() line 162 in src/snes/impls/ls/ls.c > [0]PETSC ERROR: SNESSolve() line 3636 in src/snes/interface/snes.c > [0]PETSC ERROR: TSStep_ARKIMEX() line 765 in src/ts/impls/arkimex/arkimex.c > [0]PETSC ERROR: TSStep() line 2458 in src/ts/interface/ts.c > [0]PETSC ERROR: TSSolve() line 2583 in src/ts/interface/ts.c > [0]PETSC ERROR: main() line 457 in src/ts/examples/tutorials/diffusion.c > ./compile_diffusion: line 17: 23584 Aborted ./diffusion_c > -ts_adapt_monitor -ts_adapt_basic_clip 0.1,1.1 -draw_pause -2 > -ts_arkimex_type 1bee -ts_max_snes_failures -1 -snes_type newtonls > -snes_linesearch_type bt -ts_arkimex_fully_implicit -snes_mf_operator > > Using -snes_converged_reason and -ksp_converged_reason gives: > > Linear solve did not converge due to DIVERGED_DTOL iterations 30 > Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations > 0 > TSAdapt 'basic': step 0 stage rejected t=0 + 1.000e-12 > retrying with dt=2.500e-13 > > Thus, I increased the maximum number of iterations of the KSP with > ierr = SNESSetMaxLinearSolveFailures(snes,100); > and > ierr = > KSPSetTolerances(ksp,PETSC_DEFAULT,PETSC_DEFAULT,PETSC_DEFAULT,1000); > > Then it converges, steps are accepted. However, the solution obtained is > weird, as if there was no diffusion. Moreover, values are incorrect. I > start with a gaussian peak at 1e19 and end up with a gaussian with a peak > at 1e25. > > BTW, why using -ts_arkimex_fully_implicit if all the terms are in the F > (LHS) ? You said that by definition there is no explicit stage in that case. > > However, I could obtain a good solution by adding mesh points (without > fully implicit and mf_operator). Maybe it was due to large gradients and > not enough points. > > Christophe > > > > >
