Hi Jed (and others),

Here is an update of my previous problem about SNES convergence 
difference on different machines:

The convergence difference seems to be due to the different compilation 
modes of Petsc. When running the libmesh (and Petsc) in the debug mode 
(flag= -g),  both the desktop and the cluster returned the same 
convergence results. However, when running libmesh in the 
RelWithDebugInfo mode (flag= -O2), the desktop yielded good convergence 
whereas the cluster yielded bad convergence.

The cluster uses a specific Intel compiler which is different from the 
desktop.

In response to Jed's previous questions:
1. No worry for the seg fault problem, which has been resolved.
2. The initial state for the nonlinear system was an all-zero solution 
vector.

This problem seems to be relevant to Petsc rather than Libmesh. Any 
comments would be appreciated. I can provide further details about my 
simulation if needed.

Below are the convergence results obtained with the flags "-pc_type lu 
-ksp_monitor -ksp_converged_reason -snes_monitor -snes_converged_reason
".
------------------------------------------------------------------------
Debug mode (same results on both machines):

Desktop machine:
   NL step  0, |residual|_2 = 1.440000e+00
   0 SNES Function norm 1.440000000000e+00
     0 KSP Residual norm 4.021760102139e+00
     1 KSP Residual norm 1.040484538968e-14
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  1, |residual|_2 = 5.960576e-01
   1 SNES Function norm 5.960575795099e-01
     0 KSP Residual norm 4.727634711341e-01
     1 KSP Residual norm 1.169178369623e-14
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  2, |residual|_2 = 3.745537e-02
   2 SNES Function norm 3.745536863104e-02
     0 KSP Residual norm 5.571738493521e-02
     1 KSP Residual norm 4.222606354657e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  3, |residual|_2 = 7.370172e-04
   3 SNES Function norm 7.370171737019e-04
     0 KSP Residual norm 6.418014965935e-04
     1 KSP Residual norm 2.132391208383e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  4, |residual|_2 = 1.433233e-07
   4 SNES Function norm 1.433233447799e-07
     0 KSP Residual norm 5.038509414466e-07
     1 KSP Residual norm 1.948639405146e-21
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  5, |residual|_2 = 3.440465e-13
   5 SNES Function norm 3.440465235277e-13
Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 5
Mechanics system solved at nonlinear iteration 5 , final nonlinear 
residual norm: 3.440465e-13, converge reason= 3

Cluster:
   NL step  0, |residual|_2 = 1.440000e+00
   0 SNES Function norm 1.440000000000e+00
     0 KSP Residual norm 4.021760102139e+00
     1 KSP Residual norm 2.379314305795e-14
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  1, |residual|_2 = 5.960576e-01
   1 SNES Function norm 5.960575795099e-01
     0 KSP Residual norm 4.727634711341e-01
     1 KSP Residual norm 7.976234556294e-15
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  2, |residual|_2 = 3.745537e-02
   2 SNES Function norm 3.745536863104e-02
     0 KSP Residual norm 5.571738493521e-02
     1 KSP Residual norm 2.844838895371e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  3, |residual|_2 = 7.370172e-04
   3 SNES Function norm 7.370171737018e-04
     0 KSP Residual norm 6.418014965934e-04
     1 KSP Residual norm 4.308781214538e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  4, |residual|_2 = 1.433233e-07
   4 SNES Function norm 1.433233452409e-07
     0 KSP Residual norm 5.038509410375e-07
     1 KSP Residual norm 2.281704035369e-21
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  5, |residual|_2 = 3.436775e-13
   5 SNES Function norm 3.436774565070e-13
Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 5
Mechanics system solved at nonlinear iteration 5 , final nonlinear 
residual norm: 3.436775e-13, converge reason= 3

------------------------------------------------------------------------
RelWIthDebugInfo mode (-O2), different convergence behavior

Desktop (good convergence, also same as the debug mode):
   NL step  0, |residual|_2 = 1.440000e+00
   0 SNES Function norm 1.440000000000e+00
     0 KSP Residual norm 4.021760102139e+00
     1 KSP Residual norm 1.040484538968e-14
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  1, |residual|_2 = 5.960576e-01
   1 SNES Function norm 5.960575795099e-01
     0 KSP Residual norm 4.727634711341e-01
     1 KSP Residual norm 1.169178369623e-14
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  2, |residual|_2 = 3.745537e-02
   2 SNES Function norm 3.745536863104e-02
     0 KSP Residual norm 5.571738493521e-02
     1 KSP Residual norm 4.222606354657e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  3, |residual|_2 = 7.370172e-04
   3 SNES Function norm 7.370171737019e-04
     0 KSP Residual norm 6.418014965935e-04
     1 KSP Residual norm 2.132391208383e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  4, |residual|_2 = 1.433233e-07
   4 SNES Function norm 1.433233447799e-07
     0 KSP Residual norm 5.038509414466e-07
     1 KSP Residual norm 1.948639405146e-21
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  5, |residual|_2 = 3.440465e-13
   5 SNES Function norm 3.440465235277e-13
Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 5
Mechanics system solved at nonlinear iteration 5 , final nonlinear 
residual norm: 3.440465e-13, converge reason= 3

Cluster (poor convergence, different from the debug mode):
   NL step  0, |residual|_2 = 1.440000e+00
   0 SNES Function norm 1.440000000000e+00
     0 KSP Residual norm 4.021760102139e+00
     1 KSP Residual norm 2.222981076063e-14
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  1, |residual|_2 = 5.960576e-01
   1 SNES Function norm 5.960575795099e-01
     0 KSP Residual norm 5.724516760789e-01
     1 KSP Residual norm 4.044030898145e-14
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  2, |residual|_2 = 1.482067e-01
   2 SNES Function norm 1.482067440081e-01
     0 KSP Residual norm 1.899198487299e-01
     1 KSP Residual norm 2.132006073251e-15
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  3, |residual|_2 = 7.270703e-02
   3 SNES Function norm 7.270702856431e-02
     0 KSP Residual norm 3.483657684658e-02
     1 KSP Residual norm 3.286054408774e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  4, |residual|_2 = 3.049751e-03
   4 SNES Function norm 3.049750785313e-03
     0 KSP Residual norm 4.812920494029e-03
     1 KSP Residual norm 5.700508014744e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  5, |residual|_2 = 4.513461e-04
   5 SNES Function norm 4.513461421076e-04
     0 KSP Residual norm 1.209655693065e-03
     1 KSP Residual norm 4.945249451450e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  6, |residual|_2 = 1.793522e-04
   6 SNES Function norm 1.793521679461e-04
     0 KSP Residual norm 7.011178005420e-04
     1 KSP Residual norm 8.758630689120e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  7, |residual|_2 = 9.131320e-05
   7 SNES Function norm 9.131320113186e-05
     0 KSP Residual norm 3.544244645037e-04
     1 KSP Residual norm 1.540261136238e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  8, |residual|_2 = 4.648251e-05
   8 SNES Function norm 4.648251104473e-05
     0 KSP Residual norm 2.790752923777e-04
     1 KSP Residual norm 1.079811215541e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step  9, |residual|_2 = 2.515710e-05
   9 SNES Function norm 2.515709920202e-05
     0 KSP Residual norm 1.355995795589e-02
     1 KSP Residual norm 1.111094748172e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 10, |residual|_2 = 2.351012e-05
  10 SNES Function norm 2.351012455292e-05
     0 KSP Residual norm 4.715035865139e-03
     1 KSP Residual norm 5.621155808935e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 11, |residual|_2 = 2.248736e-05
  11 SNES Function norm 2.248736326877e-05
     0 KSP Residual norm 3.557797254429e-03
     1 KSP Residual norm 3.341418332138e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 12, |residual|_2 = 2.128736e-05
  12 SNES Function norm 2.128735852036e-05
     0 KSP Residual norm 7.703099954589e-03
     1 KSP Residual norm 1.154597763933e-16
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 13, |residual|_2 = 2.048972e-05
  13 SNES Function norm 2.048971947471e-05
     0 KSP Residual norm 2.679644378167e-03
     1 KSP Residual norm 8.909261104450e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 14, |residual|_2 = 1.965486e-05
  14 SNES Function norm 1.965485620055e-05
     0 KSP Residual norm 1.479045966367e-02
     1 KSP Residual norm 5.645787321175e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 15, |residual|_2 = 1.737326e-05
  15 SNES Function norm 1.737325739134e-05
     0 KSP Residual norm 5.144625209331e-03
     1 KSP Residual norm 8.043824552860e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 16, |residual|_2 = 1.680721e-05
  16 SNES Function norm 1.680721071217e-05
     0 KSP Residual norm 1.409004856850e-03
     1 KSP Residual norm 1.055045286656e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 17, |residual|_2 = 1.505455e-05
  17 SNES Function norm 1.505455093030e-05
     0 KSP Residual norm 7.771120407446e-03
     1 KSP Residual norm 5.978364642141e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 18, |residual|_2 = 1.413325e-05
  18 SNES Function norm 1.413324859093e-05
     0 KSP Residual norm 2.703906031576e-03
     1 KSP Residual norm 3.156540186874e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 19, |residual|_2 = 1.349265e-05
  19 SNES Function norm 1.349264959991e-05
     0 KSP Residual norm 2.604619849008e-03
     1 KSP Residual norm 4.556754801716e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 20, |residual|_2 = 1.288406e-05
  20 SNES Function norm 1.288406277123e-05
     0 KSP Residual norm 2.408639307512e-03
     1 KSP Residual norm 1.574105435806e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 21, |residual|_2 = 1.228811e-05
  21 SNES Function norm 1.228810775661e-05
     0 KSP Residual norm 2.592267975097e-03
     1 KSP Residual norm 4.275624728417e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 22, |residual|_2 = 1.177481e-05
  22 SNES Function norm 1.177481128032e-05
     0 KSP Residual norm 1.447675648634e-03
     1 KSP Residual norm 1.769626897387e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 23, |residual|_2 = 1.115206e-05
  23 SNES Function norm 1.115206302877e-05
     0 KSP Residual norm 7.987558615343e-03
     1 KSP Residual norm 3.471776844754e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 24, |residual|_2 = 9.968437e-06
  24 SNES Function norm 9.968436832047e-06
     0 KSP Residual norm 2.779729958997e-03
     1 KSP Residual norm 2.539893189750e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 25, |residual|_2 = 9.631119e-06
  25 SNES Function norm 9.631119115197e-06
     0 KSP Residual norm 4.793844577865e-04
     1 KSP Residual norm 1.642814961535e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 26, |residual|_2 = 7.658449e-06
  26 SNES Function norm 7.658449202595e-06
     0 KSP Residual norm 5.463625248925e-03
     1 KSP Residual norm 4.389074959683e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 27, |residual|_2 = 6.852486e-06
  27 SNES Function norm 6.852485650682e-06
     0 KSP Residual norm 1.901774052606e-03
     1 KSP Residual norm 1.166130675804e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 28, |residual|_2 = 6.620653e-06
  28 SNES Function norm 6.620653139738e-06
     0 KSP Residual norm 3.141182445840e-04
     1 KSP Residual norm 6.065443405558e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 29, |residual|_2 = 5.171843e-06
  29 SNES Function norm 5.171843144827e-06
     0 KSP Residual norm 3.871258196172e-03
     1 KSP Residual norm 1.951195061396e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 30, |residual|_2 = 4.576905e-06
  30 SNES Function norm 4.576905178809e-06
     0 KSP Residual norm 1.347692935990e-03
     1 KSP Residual norm 1.446770176793e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 31, |residual|_2 = 4.427166e-06
  31 SNES Function norm 4.427166013787e-06
     0 KSP Residual norm 3.574992580610e-04
     1 KSP Residual norm 7.908091467650e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 32, |residual|_2 = 3.950633e-06
  32 SNES Function norm 3.950633118913e-06
     0 KSP Residual norm 1.970680556721e-03
     1 KSP Residual norm 1.088685998265e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 33, |residual|_2 = 3.721235e-06
  33 SNES Function norm 3.721234938692e-06
     0 KSP Residual norm 6.861687491055e-04
     1 KSP Residual norm 1.953135470491e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 34, |residual|_2 = 3.547135e-06
  34 SNES Function norm 3.547134826276e-06
     0 KSP Residual norm 7.856677852322e-04
     1 KSP Residual norm 1.330652439994e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 35, |residual|_2 = 3.404737e-06
  35 SNES Function norm 3.404737346554e-06
     0 KSP Residual norm 3.073815582292e-04
     1 KSP Residual norm 1.603901159353e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 36, |residual|_2 = 3.075132e-06
  36 SNES Function norm 3.075131676655e-06
     0 KSP Residual norm 1.694745606692e-03
     1 KSP Residual norm 1.791266224670e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 37, |residual|_2 = 2.866068e-06
  37 SNES Function norm 2.866068373248e-06
     0 KSP Residual norm 5.901013004031e-04
     1 KSP Residual norm 3.927275386298e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 38, |residual|_2 = 2.743693e-06
  38 SNES Function norm 2.743693466503e-06
     0 KSP Residual norm 3.838214494089e-04
     1 KSP Residual norm 9.894588963925e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 39, |residual|_2 = 2.673055e-06
  39 SNES Function norm 2.673055265273e-06
     0 KSP Residual norm 2.117297982903e-03
     1 KSP Residual norm 9.292131679050e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 40, |residual|_2 = 2.331440e-06
  40 SNES Function norm 2.331439782776e-06
     0 KSP Residual norm 7.371907291326e-04
     1 KSP Residual norm 8.470282280236e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 41, |residual|_2 = 2.256311e-06
  41 SNES Function norm 2.256310976814e-06
     0 KSP Residual norm 2.567895446571e-04
     1 KSP Residual norm 6.852266839368e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 42, |residual|_2 = 2.105991e-06
  42 SNES Function norm 2.105991050251e-06
     0 KSP Residual norm 1.416238836146e-03
     1 KSP Residual norm 1.550853158673e-17
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 43, |residual|_2 = 1.906527e-06
  43 SNES Function norm 1.906526750558e-06
     0 KSP Residual norm 4.931289500004e-04
     1 KSP Residual norm 5.290867863556e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 44, |residual|_2 = 1.838924e-06
  44 SNES Function norm 1.838923996912e-06
     0 KSP Residual norm 1.405762049962e-05
     1 KSP Residual norm 4.585529421262e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 45, |residual|_2 = 1.058048e-06
  45 SNES Function norm 1.058047908625e-06
     0 KSP Residual norm 7.729636904681e-04
     1 KSP Residual norm 4.133047612862e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 46, |residual|_2 = 9.417074e-07
  46 SNES Function norm 9.417073985418e-07
     0 KSP Residual norm 2.691875835493e-04
     1 KSP Residual norm 1.023461541301e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 47, |residual|_2 = 9.104019e-07
  47 SNES Function norm 9.104018687209e-07
     0 KSP Residual norm 5.848764695778e-05
     1 KSP Residual norm 3.731574398649e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 48, |residual|_2 = 7.821364e-07
  48 SNES Function norm 7.821364289332e-07
     0 KSP Residual norm 4.227277266633e-04
     1 KSP Residual norm 4.822287553611e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 49, |residual|_2 = 7.306530e-07
  49 SNES Function norm 7.306529623165e-07
     0 KSP Residual norm 1.472269887921e-04
     1 KSP Residual norm 3.360940671559e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 50, |residual|_2 = 6.989741e-07
  50 SNES Function norm 6.989741368342e-07
     0 KSP Residual norm 1.079610402578e-04
     1 KSP Residual norm 6.085658602505e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 51, |residual|_2 = 6.986050e-07
  51 SNES Function norm 6.986049538262e-07
     0 KSP Residual norm 5.955220857101e-04
     1 KSP Residual norm 5.588488888499e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 52, |residual|_2 = 5.960243e-07
  52 SNES Function norm 5.960242923830e-07
     0 KSP Residual norm 2.073824147391e-04
     1 KSP Residual norm 5.058556742009e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 53, |residual|_2 = 5.746872e-07
  53 SNES Function norm 5.746872331107e-07
     0 KSP Residual norm 7.224510981546e-05
     1 KSP Residual norm 9.534294596381e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 54, |residual|_2 = 5.467005e-07
  54 SNES Function norm 5.467005265595e-07
     0 KSP Residual norm 3.984467805183e-04
     1 KSP Residual norm 1.600986090410e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 55, |residual|_2 = 4.868459e-07
  55 SNES Function norm 4.868459415833e-07
     0 KSP Residual norm 1.387564478005e-04
     1 KSP Residual norm 9.383394994032e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 56, |residual|_2 = 4.706315e-07
  56 SNES Function norm 4.706315384417e-07
     0 KSP Residual norm 2.944560147317e-05
     1 KSP Residual norm 6.638072600237e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 57, |residual|_2 = 4.016471e-07
  57 SNES Function norm 4.016470859394e-07
     0 KSP Residual norm 2.238135265392e-04
     1 KSP Residual norm 1.576103980007e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 58, |residual|_2 = 3.738898e-07
  58 SNES Function norm 3.738897675506e-07
     0 KSP Residual norm 7.794670768988e-05
     1 KSP Residual norm 5.237500072288e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 59, |residual|_2 = 3.581483e-07
  59 SNES Function norm 3.581482680037e-07
     0 KSP Residual norm 4.670929752150e-05
     1 KSP Residual norm 1.147265035210e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 60, |residual|_2 = 3.433438e-07
  60 SNES Function norm 3.433437845239e-07
     0 KSP Residual norm 2.576227607006e-04
     1 KSP Residual norm 2.498690560261e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 61, |residual|_2 = 3.037047e-07
  61 SNES Function norm 3.037046887097e-07
     0 KSP Residual norm 8.971382982151e-05
     1 KSP Residual norm 3.992428337482e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 62, |residual|_2 = 2.938123e-07
  62 SNES Function norm 2.938123021372e-07
     0 KSP Residual norm 2.434251616706e-05
     1 KSP Residual norm 6.288305083577e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 63, |residual|_2 = 2.629026e-07
  63 SNES Function norm 2.629025543286e-07
     0 KSP Residual norm 1.341713156961e-04
     1 KSP Residual norm 9.005362267699e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 64, |residual|_2 = 2.470770e-07
  64 SNES Function norm 2.470769796053e-07
     0 KSP Residual norm 4.672848087816e-05
     1 KSP Residual norm 5.806073812076e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 65, |residual|_2 = 2.357535e-07
  65 SNES Function norm 2.357534534358e-07
     0 KSP Residual norm 4.758611162337e-05
     1 KSP Residual norm 2.479409167491e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 66, |residual|_2 = 2.255490e-07
  66 SNES Function norm 2.255489784177e-07
     0 KSP Residual norm 3.455253521731e-05
     1 KSP Residual norm 1.848686476485e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 67, |residual|_2 = 2.249190e-07
  67 SNES Function norm 2.249189779176e-07
     0 KSP Residual norm 1.905971621337e-04
     1 KSP Residual norm 1.693538835814e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 68, |residual|_2 = 1.922701e-07
  68 SNES Function norm 1.922701107315e-07
     0 KSP Residual norm 6.637113796734e-05
     1 KSP Residual norm 5.714211105785e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 69, |residual|_2 = 1.854509e-07
  69 SNES Function norm 1.854508680080e-07
     0 KSP Residual norm 2.312103836137e-05
     1 KSP Residual norm 9.855648245687e-20
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 70, |residual|_2 = 1.761273e-07
  70 SNES Function norm 1.761273273016e-07
     0 KSP Residual norm 1.275181139415e-04
     1 KSP Residual norm 4.506070339430e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 71, |residual|_2 = 1.570743e-07
  71 SNES Function norm 1.570743326797e-07
     0 KSP Residual norm 4.440640194736e-05
     1 KSP Residual norm 5.422068035420e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 72, |residual|_2 = 1.518161e-07
  72 SNES Function norm 1.518161153491e-07
     0 KSP Residual norm 8.797907000209e-06
     1 KSP Residual norm 2.075176146890e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 73, |residual|_2 = 1.268831e-07
  73 SNES Function norm 1.268830968047e-07
     0 KSP Residual norm 7.705514125563e-05
     1 KSP Residual norm 1.017793788404e-18
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 74, |residual|_2 = 1.167839e-07
  74 SNES Function norm 1.167838585279e-07
     0 KSP Residual norm 2.683450054484e-05
     1 KSP Residual norm 5.132689633111e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 75, |residual|_2 = 1.122477e-07
  75 SNES Function norm 1.122477442296e-07
     0 KSP Residual norm 7.409290455259e-06
     1 KSP Residual norm 3.645143742976e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 76, |residual|_2 = 9.706797e-08
  76 SNES Function norm 9.706796620796e-08
     0 KSP Residual norm 5.082292041698e-05
     1 KSP Residual norm 3.544425465304e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 77, |residual|_2 = 9.099011e-08
  77 SNES Function norm 9.099010557425e-08
     0 KSP Residual norm 1.769998454096e-05
     1 KSP Residual norm 3.363150042480e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 78, |residual|_2 = 8.692366e-08
  78 SNES Function norm 8.692366224026e-08
     0 KSP Residual norm 1.566736777879e-05
     1 KSP Residual norm 2.819452990843e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 79, |residual|_2 = 8.276834e-08
  79 SNES Function norm 8.276834291448e-08
     0 KSP Residual norm 1.990319801652e-05
     1 KSP Residual norm 1.068580372799e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 80, |residual|_2 = 7.966571e-08
  80 SNES Function norm 7.966570700260e-08
     0 KSP Residual norm 3.003444562499e-06
     1 KSP Residual norm 5.206872525061e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 81, |residual|_2 = 5.608802e-08
  81 SNES Function norm 5.608802450435e-08
     0 KSP Residual norm 5.211286640517e-05
     1 KSP Residual norm 1.336954688010e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 82, |residual|_2 = 4.633277e-08
  82 SNES Function norm 4.633277025854e-08
     0 KSP Residual norm 1.814671884668e-05
     1 KSP Residual norm 1.945186873110e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 83, |residual|_2 = 4.441497e-08
  83 SNES Function norm 4.441496884398e-08
     0 KSP Residual norm 6.320882672742e-06
     1 KSP Residual norm 6.522712340090e-20
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 84, |residual|_2 = 4.345014e-08
  84 SNES Function norm 4.345013744524e-08
     0 KSP Residual norm 3.486532415243e-05
     1 KSP Residual norm 2.820388278092e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 85, |residual|_2 = 3.776562e-08
  85 SNES Function norm 3.776561517098e-08
     0 KSP Residual norm 1.214102392542e-05
     1 KSP Residual norm 4.637953640500e-20
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 86, |residual|_2 = 3.652905e-08
  86 SNES Function norm 3.652904667916e-08
     0 KSP Residual norm 4.229676380623e-06
     1 KSP Residual norm 2.985543935183e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 87, |residual|_2 = 3.421015e-08
  87 SNES Function norm 3.421015322140e-08
     0 KSP Residual norm 2.332617444021e-05
     1 KSP Residual norm 1.899596182975e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 88, |residual|_2 = 3.088663e-08
  88 SNES Function norm 3.088662974125e-08
     0 KSP Residual norm 8.123061571912e-06
     1 KSP Residual norm 9.730785565672e-20
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 89, |residual|_2 = 2.980237e-08
  89 SNES Function norm 2.980236808814e-08
     0 KSP Residual norm 4.955116260996e-07
     1 KSP Residual norm 3.777642296031e-20
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 90, |residual|_2 = 2.461180e-08
  90 SNES Function norm 2.461180426859e-08
     0 KSP Residual norm 3.089926846929e-05
     1 KSP Residual norm 1.152580814246e-19
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 91, |residual|_2 = 1.641444e-08
  91 SNES Function norm 1.641444000238e-08
     0 KSP Residual norm 1.075937296980e-05
     1 KSP Residual norm 9.644931034537e-20
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 92, |residual|_2 = 1.492579e-08
  92 SNES Function norm 1.492578941113e-08
     0 KSP Residual norm 3.746844578568e-06
     1 KSP Residual norm 2.240329961008e-20
   Linear solve converged due to CONVERGED_RTOL iterations 1
   NL step 93, |residual|_2 = 1.438360e-08
  93 SNES Function norm 1.438360232831e-08
Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 93
Mechanics system solved at nonlinear iteration 93 , final nonlinear 
residual norm: 1.438360e-08, converge reason= 3

Thanks,
Dafang

On 01/03/2014 08:51 AM, Jed Brown wrote:
> Dafang Wang <[email protected]> writes:
>
>> Hi Jed,
>>
>> Below are the output details. (Results were obtained with 
>> -snes_converged_reason,
>> -snes_monitor. The -ksp_converged_reason ended up with seg fault, so was not 
>> included. It
>> should not be the key problem though. )
> It should not SEGV.  Can you get a stack trace?
>
>> Output from the desktop (good convergence rate):
>> total constitutive equations = ngauss = 1000
>>    EquationSystems
>>     n_systems()=1
>>      System #0, "Elasticity"
>>       Type "NonlinearImplicit"
>>       Variables={ "ux" "uy" "uz" }
>>       Finite Element Types="LAGRANGE"
>>       Approximation Orders="FIRST"
>>       n_dofs()=648
>>       n_local_dofs()=648
>>       n_constrained_dofs()=108
>>       n_local_constrained_dofs()=108
>>       n_vectors()=1
>>       n_matrices()=1
>>       DofMap Sparsity
>>         Average  On-Processor Bandwidth <= 56.8889
>>         Average Off-Processor Bandwidth <= 0
>>         Maximum  On-Processor Bandwidth <= 81
>>         Maximum Off-Processor Bandwidth <= 0
>>       DofMap Constraints
>>         Number of DoF Constraints = 108
>>         Average DoF Constraint Length= 0
>>
>> *** Warning, This code is deprecated, and likely to be removed in future 
>> library versions!
>> src/mesh/boundary_info.C, line 752, compiled Oct 29 2013 at 08:06:16 ***
> What is this about?
>
>>     NL step  0, |residual|_2 = 1.440000e+00
>>     0 SNES Function norm 1.440000000000e+00
>>     NL step  1, |residual|_2 = 5.960479e-01
>>     1 SNES Function norm 5.960478586457e-01
>>     NL step  2, |residual|_2 = 3.745465e-02
>>     2 SNES Function norm 3.745464932748e-02
>>     NL step  3, |residual|_2 = 7.372288e-04
>>     3 SNES Function norm 7.372287735879e-04
>>     NL step  4, |residual|_2 = 1.426607e-07
>>     4 SNES Function norm 1.426607123296e-07
>>     NL step  5, |residual|_2 = 4.472564e-12
>>     5 SNES Function norm 4.472564229796e-12
>> Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 5
>> Mechanics system solved at nonlinear iteration 5 , final nonlinear residual 
>> norm:
>> 4.472564e-12, converge reason= 3
>> ------------------------------------------------------------------------------------------
>> Output from the cluster (poor convergence rate):
>> total constitutive equations = ngauss = 1000
>>    EquationSystems
>>     n_systems()=1
>>      System #0, "Elasticity"
>>       Type "NonlinearImplicit"
>>       Variables={ "ux" "uy" "uz" }
>>       Finite Element Types="LAGRANGE"
>>       Approximation Orders="FIRST"
>>       n_dofs()=648
>>       n_local_dofs()=648
>>       n_constrained_dofs()=108
>>       n_local_constrained_dofs()=108
>>       n_vectors()=1
>>       n_matrices()=1
>>       DofMap Sparsity
>>         Average  On-Processor Bandwidth <= 56.8889
>>         Average Off-Processor Bandwidth <= 0
>>         Maximum  On-Processor Bandwidth <= 81
>>         Maximum Off-Processor Bandwidth <= 0
>>       DofMap Constraints
>>         Number of DoF Constraints = 108
>>         Average DoF Constraint Length= 0
>>
>> *** Warning, This code is deprecated, and likely to be removed in future 
>> library versions!
>> src/mesh/boundary_info.C, line 752, compiled Dec 19 2013 at 15:59:31 ***
>>     NL step  0, |residual|_2 = 1.440000e+00
>>     0 SNES Function norm 1.440000000000e+00
>>     NL step  1, |residual|_2 = 5.960479e-01
>>     1 SNES Function norm 5.960478586457e-01
>>     NL step  2, |residual|_2 = 1.481775e-01
>>     2 SNES Function norm 1.481775334529e-01
> Can you try with low optimization?  This has the look of an incorrect
> Jacobian (perhaps due to bad optimization or a bad dependent library).
>
> What is the initial state?  Can you try running with -snes_mf_operator?
> (And get the -ksp_monitor -ksp_converged_reason working; we need to know
> what's happening with that solve.)

-- 
Dafang Wang, Ph.D
Postdoctoral Fellow
Institute of Computational Medicine
Department of Biomedical Engineering
Johns Hopkins University
Hackerman Hall Room 218
Baltimore, MD, 21218
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Libmesh-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/libmesh-users

Reply via email to