On Mon, Oct 2, 2017 at 11:49 PM, Mark Adams <mfad...@lbl.gov> wrote: > Whenbo, do you build your PETSc? > > Yes. My configure option is listed below ./configure --with-mpi=1 --with-shared-libraries=1 \ --with-64-bit-indices=1 --with-debugging=1
And I set PETSC_DIR, PETSC_ARCH and SLEPC_DIR in my ~/.bashrc. The Makefile for my problem is listed below, PETSC_ARCH = arch-linux2-c-debug PETSC_DIR = /home/zhaowenbo/research/petsc/petsc_git SLEPC_DIR = /home/zhaowenbo/research/slepc/slepc_git #PETSC_DIR = /home/zhaowenbo/research/petsc/petsc-3.7.4 #SLEPC_DIR = /home/zhaowenbo/research/slepc/slepc-3.7.3 HYPRE_DIR = /usr/local/hypre # DEBUG_OPT = -g COMP_FLAGS = -fPIC -Wall \ -I${SLEPC_DIR}/include -I${SLEPC_DIR}/${PETSC_ARCH}/include \ -I${PETSC_DIR}/include -I${PETSC_DIR}/${PETSC_ARCH}/include \ -Isrc LINK_FLAGS = -fPIC -Wall \ -Wl,-rpath,${SLEPC_DIR}/${PETSC_ARCH}/lib -L${SLEPC_DIR}/${PETSC_ARCH}/lib -lslepc \ -Wl,-rpath,${PETSC_DIR}/${PETSC_ARCH}/lib -L${PETSC_DIR}/${PETSC_ARCH}/lib -lpetsc \ -llapack -lblas -lhwloc -lm -lgfortran -lquadmath step-41: src/main.o src/readinp.o src/base.o src/sp3.o src/diffu.o mpicxx -o step-41 $^ ${LINK_FLAGS} ${DEBUG_OPT} src/main.o: src/main.c mpicxx -o src/main.o -c $^ ${COMP_FLAGS} ${DEBUG_OPT} src/readinp.o: src/readinp.c mpicxx -o src/readinp.o -c $^ ${COMP_FLAGS} ${DEBUG_OPT} src/sp3.o: src/sp3.c mpicxx -o src/sp3.o -c $^ ${COMP_FLAGS} ${DEBUG_OPT} src/diffu.o: src/diffu.c mpicxx -o src/diffu.o -c $^ ${COMP_FLAGS} ${DEBUG_OPT} src/base.o: src/base.c mpicxx -o src/base.o -c $^ ${COMP_FLAGS} ${DEBUG_OPT} clean: rm step-41 src/main.o src/readinp.o src/sp3.o src/diffu.o src/base.o runkr_smooth: mpirun -n ${NCORE} ./step-41 \ -st_ksp_type gmres -st_ksp_view -st_ksp_monitor \ -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 1 \ -mata AMAT.dat -matb BMAT.dat \ -st_gamg_est_ksp_view -st_gamg_est_ksp_monitor \ -st_gamg_est_ksp_converged_reason \ -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > log_smooth 2>&1 runkr_nonsmooth: mpirun -n ${NCORE} ./step-41 \ -st_ksp_type gmres -st_ksp_view -st_ksp_monitor \ -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 0 \ -mata AMAT.dat -matb BMAT.dat \ -st_gamg_est_ksp_view -st_gamg_est_ksp_monitor \ -st_gamg_est_ksp_converged_reason \ -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > log_nonsmooth 2>&1 Thanks, Wenbo > On Mon, Oct 2, 2017 at 11:45 AM, Mark Adams <mfad...@lbl.gov> wrote: > >> This is normal: >> >> Linear st_gamg_est_ solve did not converge due to DIVERGED_ITS iterations >> 10 >> >> It looks like ksp->errorifnotconverged got set somehow. If the default >> changed in KSP then (SAGG) GAMG would not ever work. >> >> I assume you don't have a .petscrc file with more (crazy) options in it >> ... >> >> >> On Mon, Oct 2, 2017 at 11:39 AM, Wenbo Zhao <zhaowenbo.n...@gmail.com> >> wrote: >> >>> >>> >>> On Mon, Oct 2, 2017 at 11:30 PM, Matthew Knepley <knep...@gmail.com> >>> wrote: >>> >>>> On Mon, Oct 2, 2017 at 11:15 AM, Mark Adams <mfad...@lbl.gov> wrote: >>>> >>>>> non-smoothed aggregation is converging very fast. smoothed fails in >>>>> the eigen estimator. >>>>> >>>>> Run this again with -st_gamg_est_ksp_view and >>>>> -st_gamg_est_ksp_monitor, and see if you get more output (I'm not 100% >>>>> sure >>>>> about these args). >>>>> >>>> >>>> I also want -st_gamg_est_ksp_converged_reason >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>> $make NCORE=1 runkr_smooth >>> mpirun -n 1 ./step-41 \ >>> -st_ksp_type gmres -st_ksp_view -st_ksp_monitor \ >>> -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 1 \ >>> -mata AMAT.dat -matb BMAT.dat \ >>> -st_gamg_est_ksp_view -st_gamg_est_ksp_monitor \ >>> -st_gamg_est_ksp_converged_reason \ >>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > log_smooth 2>&1 >>> makefile:43: recipe for target 'runkr_smooth' failed >>> make: *** [runkr_smooth] Error 91 >>> >>> Thanks >>> Wenbo >>> >>> >>>> >>>> >>>>> >>>>> On Mon, Oct 2, 2017 at 11:06 AM, Wenbo Zhao <zhaowenbo.n...@gmail.com> >>>>> wrote: >>>>> >>>>>> Matt, >>>>>> >>>>>> Test 1 nonsmooth >>>>>> zhaowenbo@ubuntu:~/test_slepc/SPARK/spark$ make NCORE=1 >>>>>> runkr_nonsmooth >>>>>> mpirun -n 1 ./step-41 \ >>>>>> -st_ksp_type gmres -st_ksp_view -st_ksp_monitor \ >>>>>> -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 0 \ >>>>>> -mata AMAT.dat -matb BMAT.dat \ >>>>>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > log_nonsmooth >>>>>> 2>&1 >>>>>> >>>>>> Test 2 smooth >>>>>> zhaowenbo@ubuntu:~/test_slepc/SPARK/spark$ make NCORE=1 runkr_smooth >>>>>> mpirun -n 1 ./step-41 \ >>>>>> -st_ksp_type gmres -st_ksp_view -st_ksp_monitor \ >>>>>> -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 1 \ >>>>>> -mata AMAT.dat -matb BMAT.dat \ >>>>>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > log_smooth 2>&1 >>>>>> makefile:43: recipe for target 'runkr_smooth' failed >>>>>> make: *** [runkr_smooth] Error 91 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Wenbo >>>>>> >>>>>> On Mon, Oct 2, 2017 at 10:48 PM, Matthew Knepley <knep...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> On Mon, Oct 2, 2017 at 10:43 AM, Wenbo Zhao < >>>>>>> zhaowenbo.n...@gmail.com> wrote: >>>>>>> >>>>>>>> Mark, >>>>>>>> >>>>>>>> Thanks for your reply. >>>>>>>> >>>>>>>> On Mon, Oct 2, 2017 at 9:51 PM, Mark Adams <mfad...@lbl.gov> wrote: >>>>>>>> >>>>>>>>> Please send the output with -st_ksp_view and -st_ksp_monitor and >>>>>>>>> we can start to debug it. >>>>>>>>> >>>>>>>>> Test 1 with nonsmooth and preonly is OK >>>>>>>> zhaowenbo@ubuntu:~/test_slepc/SPARK/spark$ make NCORE=1 >>>>>>>> runkr_nonsmooth >>>>>>>> mpirun -n 1 ./step-41 \ >>>>>>>> -st_ksp_type gmres -st_ksp_view -st_ksp_monitor \ >>>>>>>> -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 0 >>>>>>>> \ >>>>>>>> -st_ksp_view -mata AMAT.dat -matb BMAT.dat \ >>>>>>>> -st_mg_coarse_ksp_type preonly -st_mg_coarse_ksp_monitor \ >>>>>>>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > log_nonsmooth >>>>>>>> 2>&1 >>>>>>>> >>>>>>>> Test 2 smooth and preonly is not OK >>>>>>>> zhaowenbo@ubuntu:~/test_slepc/SPARK/spark$ make NCORE=1 >>>>>>>> runkr_smooth >>>>>>>> mpirun -n 1 ./step-41 \ >>>>>>>> -st_ksp_type gmres -st_ksp_view -st_ksp_monitor \ >>>>>>>> -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 1 >>>>>>>> \ >>>>>>>> -st_ksp_view -mata AMAT.dat -matb BMAT.dat \ >>>>>>>> -st_mg_coarse_ksp_type preonly -st_mg_coarse_ksp_monitor \ >>>>>>>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > log_smooth 2>&1 >>>>>>>> makefile:43: recipe for target 'runkr_smooth' failed >>>>>>>> make: *** [runkr_smooth] Error 91 >>>>>>>> >>>>>>>> Test 3 nonsmooth and gmres is not OK >>>>>>>> zhaowenbo@ubuntu:~/test_slepc/SPARK/spark$ make NCORE=1 >>>>>>>> runkr_gmres >>>>>>>> mpirun -n 1 ./step-41 \ >>>>>>>> -st_ksp_type gmres -st_ksp_view -st_ksp_monitor \ >>>>>>>> -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 0 >>>>>>>> \ >>>>>>>> -st_ksp_view -mata AMAT.dat -matb BMAT.dat \ >>>>>>>> -st_mg_coarse_ksp_type gmres -st_mg_coarse_ksp_monitor >>>>>>>> -st_mg_coarse_ksp_rtol 1.0e-6 \ >>>>>>>> >>>>>>> >>>>>>> DO NOT DO THIS. Please send the output where you do NOTHING to the >>>>>>> coarse solver. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Matt >>>>>>> >>>>>>> >>>>>>>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > log_gmres 2>&1 >>>>>>>> makefile:59: recipe for target 'runkr_gmres' failed >>>>>>>> make: *** [runkr_gmres] Error 91 >>>>>>>> >>>>>>>> log-files is attached. >>>>>>>> >>>>>>>> >>>>>>>> You mentioned that B is not symmetric. I assume it is elliptic >>>>>>>>> (diffusion). Where does the asymmetry come from? >>>>>>>>> >>>>>>>>> >>>>>>>> It is a two-group diffusion equations, where group denotes neutron >>>>>>>> enegry discretisation. >>>>>>>> Matrix B consists of neutron diffusion/leakage term, removal term >>>>>>>> and minus neutron scatter source term between different energies, when >>>>>>>> matrix A denotes neutron fission source. >>>>>>>> >>>>>>>> Diffusion term(Laplace operator) is elliptic and symmetric. Removal >>>>>>>> term is diagonal only. However scatter term is asymmetry since scatter >>>>>>>> term >>>>>>>> from high energy to low energy is far greater than the term from low to >>>>>>>> high. >>>>>>>> >>>>>>>> >>>>>>>> Wenbo >>>>>>>> >>>>>>>> >>>>>>>>> On Mon, Oct 2, 2017 at 9:39 AM, Wenbo Zhao < >>>>>>>>> zhaowenbo.n...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Matt, >>>>>>>>>> Thanks for your reply. >>>>>>>>>> For the defalt option doesnt work firstly( -st_ksp_type gmres >>>>>>>>>> -st_pc_type gamg -st_pc_gamg_type agg -st_pc_gamg_agg_nsmooths 1), I >>>>>>>>>> tried >>>>>>>>>> to test those options. >>>>>>>>>> >>>>>>>>>> Wenbo >>>>>>>>>> >>>>>>>>>> On Mon, Oct 2, 2017 at 9:08 PM, Matthew Knepley < >>>>>>>>>> knep...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> On Mon, Oct 2, 2017 at 8:30 AM, Wenbo Zhao < >>>>>>>>>>> zhaowenbo.n...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Matt >>>>>>>>>>>> >>>>>>>>>>>> Because I am not clear about what will happen using 'preonly' >>>>>>>>>>>> for large scale problem. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The size of the problem has nothing to do with 'preonly'. All it >>>>>>>>>>> means is to apply a preconditioner without a Krylov solver. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> It seems to use a direct solver from below, >>>>>>>>>>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/ >>>>>>>>>>>> KSP/KSPPREONLY.html >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> However, I still cannot understand why you would change the >>>>>>>>>>> default? >>>>>>>>>>> >>>>>>>>>>> Matt >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> Wenbo >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Oct 2, 2017 at 5:09 PM, Matthew Knepley < >>>>>>>>>>>> knep...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Oct 1, 2017 at 9:53 PM, Wenbo Zhao < >>>>>>>>>>>>> zhaowenbo.n...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Matt, >>>>>>>>>>>>>> Thanks for your reply. >>>>>>>>>>>>>> It DOES make no sense for this problem. >>>>>>>>>>>>>> But I am not clear about the 'preonly' option. Which solver >>>>>>>>>>>>>> is used in preonly? I wonder if 'preonly' is suitable for large >>>>>>>>>>>>>> scale >>>>>>>>>>>>>> problem such as 400,000,000 unknowns. >>>>>>>>>>>>>> So I tried 'gmres' option and found these error messages. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I mean, why are you setting this at all. Just do not set the >>>>>>>>>>>>> coarse solver. The default should work fine. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Matt >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> Could you give me some suggestions? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Wenbo >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 12:34 AM, Matthew Knepley < >>>>>>>>>>>>>> knep...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, Oct 1, 2017 at 6:49 AM, Wenbo Zhao < >>>>>>>>>>>>>>> zhaowenbo.n...@gmail.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I met some questions when I use PETSC/SLEPC to solve >>>>>>>>>>>>>>>> two-group neutron diffusion equations with finite difference >>>>>>>>>>>>>>>> method. The >>>>>>>>>>>>>>>> grid is 3*3*3, when DOF on each points is 2. So the matrix >>>>>>>>>>>>>>>> size is 54*54. >>>>>>>>>>>>>>>> It is generalized eigenvalue problem Ax=\lamda Bx, where B >>>>>>>>>>>>>>>> is diagonally dominant matrix but not symmetry. >>>>>>>>>>>>>>>> EPS is set as below, >>>>>>>>>>>>>>>> ierr = EPSSetProblemType(eps,EPS_GNHEP);CHKERRQ(ierr);¬ >>>>>>>>>>>>>>>> ierr = EPSSetWhichEigenpairs(eps,EPS_ >>>>>>>>>>>>>>>> LARGEST_REAL);CHKERRQ(ierr);¬ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Krylovschur is used as eps sovler. GAMG is used as PC. >>>>>>>>>>>>>>>> I tried agg_nsmooths and mg_coarse_ksp_type. Only >>>>>>>>>>>>>>>> non-smooths and preonly is OK. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Why are you setting the coarse solver. This makes no sense. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Matt >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Test 1 >>>>>>>>>>>>>>>> $ make NCORE=1 runkr_nonsmooth >>>>>>>>>>>>>>>> mpirun -n 1 ./step-41 \ >>>>>>>>>>>>>>>> -st_ksp_type gmres \ >>>>>>>>>>>>>>>> -st_pc_type gamg -st_pc_gamg_type agg >>>>>>>>>>>>>>>> -st_pc_gamg_agg_nsmooths 0 \ >>>>>>>>>>>>>>>> -st_ksp_view -mata AMAT.dat -matb BMAT.dat \ >>>>>>>>>>>>>>>> -st_mg_coarse_ksp_type preonly >>>>>>>>>>>>>>>> -st_mg_coarse_ksp_monitor \ >>>>>>>>>>>>>>>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > >>>>>>>>>>>>>>>> log_nonsmooth 2>&1 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Test 2 >>>>>>>>>>>>>>>> $ make NCORE=1 runkr_smooth >>>>>>>>>>>>>>>> mpirun -n 1 ./step-41 \ >>>>>>>>>>>>>>>> -st_ksp_type gmres \ >>>>>>>>>>>>>>>> -st_pc_type gamg -st_pc_gamg_type agg >>>>>>>>>>>>>>>> -st_pc_gamg_agg_nsmooths 1 \ >>>>>>>>>>>>>>>> -st_ksp_view -mata AMAT.dat -matb BMAT.dat \ >>>>>>>>>>>>>>>> -st_mg_coarse_ksp_type preonly >>>>>>>>>>>>>>>> -st_mg_coarse_ksp_monitor \ >>>>>>>>>>>>>>>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > >>>>>>>>>>>>>>>> log_smooth 2>&1 >>>>>>>>>>>>>>>> makefile:43: recipe for target 'runkr_smooth' failed >>>>>>>>>>>>>>>> make: *** [runkr_smooth] Error 91 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Test 3 >>>>>>>>>>>>>>>> $ make NCORE=1 runkr_gmres >>>>>>>>>>>>>>>> mpirun -n 1 ./step-41 \ >>>>>>>>>>>>>>>> -st_ksp_type gmres \ >>>>>>>>>>>>>>>> -st_pc_type gamg -st_pc_gamg_type agg >>>>>>>>>>>>>>>> -st_pc_gamg_agg_nsmooths 0 \ >>>>>>>>>>>>>>>> -st_ksp_view -mata AMAT.dat -matb BMAT.dat \ >>>>>>>>>>>>>>>> -st_mg_coarse_ksp_type gmres -st_mg_coarse_ksp_monitor >>>>>>>>>>>>>>>> -st_mg_coarse_ksp_rtol 1.0e-6 \ >>>>>>>>>>>>>>>> -eps_nev 1 -eps_ncv 10 -eps_monitor -log_view > >>>>>>>>>>>>>>>> log_gmres 2>&1 >>>>>>>>>>>>>>>> makefile:59: recipe for target 'runkr_gmres' failed >>>>>>>>>>>>>>>> make: *** [runkr_gmres] Error 91 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Log files were attched. >>>>>>>>>>>>>>>> The matrix file were also attched as AMAT.dat and BMAT.dat. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Is it correct? Or something wrong with my code or >>>>>>>>>>>>>>>> commad-line? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Wenbo >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> What most experimenters take for granted before they begin >>>>>>>>>>>>>>> their experiments is infinitely more interesting than any >>>>>>>>>>>>>>> results to which >>>>>>>>>>>>>>> their experiments lead. >>>>>>>>>>>>>>> -- Norbert Wiener >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>>>>>>>>>> <http://www.caam.rice.edu/%7Emk51/> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> What most experimenters take for granted before they begin >>>>>>>>>>>>> their experiments is infinitely more interesting than any results >>>>>>>>>>>>> to which >>>>>>>>>>>>> their experiments lead. >>>>>>>>>>>>> -- Norbert Wiener >>>>>>>>>>>>> >>>>>>>>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>>>>>>>> <http://www.caam.rice.edu/%7Emk51/> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> What most experimenters take for granted before they begin their >>>>>>>>>>> experiments is infinitely more interesting than any results to >>>>>>>>>>> which their >>>>>>>>>>> experiments lead. >>>>>>>>>>> -- Norbert Wiener >>>>>>>>>>> >>>>>>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>>>>>> <http://www.caam.rice.edu/%7Emk51/> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> What most experimenters take for granted before they begin their >>>>>>> experiments is infinitely more interesting than any results to which >>>>>>> their >>>>>>> experiments lead. >>>>>>> -- Norbert Wiener >>>>>>> >>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>> <http://www.caam.rice.edu/%7Emk51/> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> <http://www.caam.rice.edu/%7Emk51/> >>>> >>> >>> >> >