Dear Matt,

the output files obtained with PETSc-3.4p4 and 3.5p1 versions using the following command line:
-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -mat_mumps_icntl_8 0 -ksp_monitor -ksp_view

are attached below. If skipping flops and memory usage per core, a diff between the two output files reduces to:
diff Output_3.4p4.txt Output_3.5p1.txt
14c14
<         Matrix Object:         64 MPI processes
---
>         Mat Object:         64 MPI processes
18c18
<           total: nonzeros=481059588, allocated nonzeros=481059588
---
>           total: nonzeros=4.8106e+08, allocated nonzeros=4.8106e+08
457c457
<               INFOG(10) (total integer space store the matrix factors after factorization): 26149876 
---
>               INFOG(10) (total integer space store the matrix factors after factorization): 26136333 
461c461
<               INFOG(14) (number of memory compress after factorization): 54 
---
>               INFOG(14) (number of memory compress after factorization): 48 
468,469c468,469
<               INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 338 
<               INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 19782 
---
>               INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 334 
>               INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 19779 
472a473,478
>               INFOG(28) (after factorization: number of null pivots encountered): 0
>               INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 470143172
>               INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 202, 10547
>               INFOG(32) (after analysis: type of analysis done): 1
>               INFOG(33) (value used for ICNTL(8)): 0
>               INFOG(34) (exponent of the determinant if determinant is requested): 0
474c480
<   Matrix Object:   64 MPI processes
---
>   Mat Object:   64 MPI processes
477c483
<     total: nonzeros=63720324, allocated nonzeros=63720324
---
>     total: nonzeros=6.37203e+07, allocated nonzeros=6.37203e+07
481c487
< Norme de U  1 7.37266E-02, L  1  1.00000E+00
---
> Norme de U  1 1.61172E-02, L  1  1.00000E+00
483c489
<  Temps total d execution :   198.373291969299     
---
>  Temps total d execution :   216.934082031250     


Which does not reveal any striking differences, except in the L2 norm of the solution vectors.

I need assistance to help me to overcome this quite bizarre behavior.

Thank you.

Marc MEDALE

=========================================================
Université Aix-Marseille, Polytech'Marseille, Dépt Mécanique Energétique
Laboratoire IUSTI, UMR 7343 CNRS-Université Aix-Marseille
Technopole de Chateau-Gombert, 5 rue Enrico Fermi
13453 MARSEILLE, Cedex 13, FRANCE
---------------------------------------------------------------------------------------------------
Tel  : +33 (0)4.91.10.69.14 ou 38
Fax : +33 (0)4.91.10.69.69
e-mail : [email protected]
=========================================================


KSP Object: 64 MPI processes
  type: preonly
  maximum iterations=10000, initial guess is zero
  tolerances:  relative=1e-20, absolute=1e-50, divergence=10000
  left preconditioning
  using NONE norm type for convergence test
PC Object: 64 MPI processes
  type: lu
    LU: out-of-place factorization
    tolerance for zero pivot 2.22045e-14
    matrix ordering: natural
    factor fill ratio given 0, needed 0
      Factored matrix follows:
        Matrix Object:         64 MPI processes
          type: mpiaij
          rows=2004002, cols=2004002
          package used to perform factorization: mumps
          total: nonzeros=481059588, allocated nonzeros=481059588
          total number of mallocs used during MatSetValues calls =0
            MUMPS run parameters:
              SYM (matrix type):                   0 
              PAR (host participation):            1 
              ICNTL(1) (output for error):         6 
              ICNTL(2) (output of diagnostic msg): 0 
              ICNTL(3) (output for global info):   0 
              ICNTL(4) (level of printing):        0 
              ICNTL(5) (input mat struct):         0 
              ICNTL(6) (matrix prescaling):        7 
              ICNTL(7) (sequentia matrix ordering):7 
              ICNTL(8) (scalling strategy):        0 
              ICNTL(10) (max num of refinements):  0 
              ICNTL(11) (error analysis):          0 
              ICNTL(12) (efficiency control):                         1 
              ICNTL(13) (efficiency control):                         0 
              ICNTL(14) (percentage of estimated workspace increase): 30 
              ICNTL(18) (input mat struct):                           3 
              ICNTL(19) (Shur complement info):                       0 
              ICNTL(20) (rhs sparse pattern):                         0 
              ICNTL(21) (somumpstion struct):                            1 
              ICNTL(22) (in-core/out-of-core facility):               0 
              ICNTL(23) (max size of memory can be allocated locally):0 
              ICNTL(24) (detection of null pivot rows):               0 
              ICNTL(25) (computation of a null space basis):          0 
              ICNTL(26) (Schur options for rhs or solution):          0 
              ICNTL(27) (experimental parameter):                     -8 
              ICNTL(28) (use parallel or sequential ordering):        1 
              ICNTL(29) (parallel ordering):                          0 
              ICNTL(30) (user-specified set of entries in inv(A)):    0 
              ICNTL(31) (factors is discarded in the solve phase):    0 
              ICNTL(33) (compute determinant):                        0 
              CNTL(1) (relative pivoting threshold):      0.01 
              CNTL(2) (stopping criterion of refinement): 1.49012e-08 
              CNTL(3) (absomumpste pivoting threshold):      0 
              CNTL(4) (vamumpse of static pivoting):         -1 
              CNTL(5) (fixation for null pivots):         0 
              RINFO(1) (local estimated flops for the elimination after 
analysis): 
                [0] 9.56571e+09 
                [1] 6.03554e+09 
                [2] 5.28602e+09 
                [3] 4.71458e+09 
                [4] 4.77297e+09 
                [5] 4.62025e+09 
                [6] 4.60478e+09 
                [7] 4.50272e+09 
                [8] 4.83745e+09 
                [9] 5.22409e+09 
                [10] 5.63805e+09 
                [11] 6.02585e+09 
                [12] 6.49054e+09 
                [13] 5.378e+09 
                [14] 4.79934e+09 
                [15] 4.77037e+09 
                [16] 4.84331e+09 
                [17] 4.93195e+09 
                [18] 5.22257e+09 
                [19] 5.25297e+09 
                [20] 5.13269e+09 
                [21] 5.19021e+09 
                [22] 4.75881e+09 
                [23] 4.30376e+09 
                [24] 4.40295e+09 
                [25] 4.36469e+09 
                [26] 4.3722e+09 
                [27] 4.5117e+09 
                [28] 4.6233e+09 
                [29] 4.77495e+09 
                [30] 4.62267e+09 
                [31] 4.51233e+09 
                [32] 4.66611e+09 
                [33] 4.09414e+09 
                [34] 6.08632e+09 
                [35] 6.33603e+09 
                [36] 6.83482e+09 
                [37] 6.10999e+09 
                [38] 4.59808e+09 
                [39] 4.84312e+09 
                [40] 5.93403e+09 
                [41] 4.98752e+09 
                [42] 5.14366e+09 
                [43] 4.4912e+09 
                [44] 4.25993e+09 
                [45] 4.53872e+09 
                [46] 4.2995e+09 
                [47] 4.41656e+09 
                [48] 4.26886e+09 
                [49] 4.9593e+09 
                [50] 5.05505e+09 
                [51] 4.92099e+09 
                [52] 6.05907e+09 
                [53] 4.79346e+09 
                [54] 4.47977e+09 
                [55] 4.69342e+09 
                [56] 4.80231e+09 
                [57] 4.31013e+09 
                [58] 4.46429e+09 
                [59] 4.48957e+09 
                [60] 6.22429e+09 
                [61] 5.87805e+09 
                [62] 7.88576e+09 
                [63] 6.65387e+09 
              RINFO(2) (local estimated flops for the assembly after 
factorization): 
                [0]  1.64392e+07 
                [1]  1.83763e+07 
                [2]  1.62773e+07 
                [3]  1.5397e+07 
                [4]  1.74243e+07 
                [5]  1.71949e+07 
                [6]  1.72873e+07 
                [7]  1.42551e+07 
                [8]  1.39905e+07 
                [9]  1.86527e+07 
                [10]  1.81107e+07 
                [11]  1.74796e+07 
                [12]  2.04681e+07 
                [13]  2.08177e+07 
                [14]  1.87126e+07 
                [15]  1.83848e+07 
                [16]  1.99101e+07 
                [17]  1.80589e+07 
                [18]  2.21854e+07 
                [19]  2.20354e+07 
                [20]  2.02771e+07 
                [21]  2.02525e+07 
                [22]  1.73768e+07 
                [23]  1.35763e+07 
                [24]  1.23504e+07 
                [25]  1.37345e+07 
                [26]  1.42224e+07 
                [27]  1.68279e+07 
                [28]  1.5816e+07 
                [29]  1.67035e+07 
                [30]  1.57229e+07 
                [31]  1.73654e+07 
                [32]  1.69465e+07 
                [33]  1.33815e+07 
                [34]  1.3024e+07 
                [35]  1.38135e+07 
                [36]  1.39399e+07 
                [37]  1.36873e+07 
                [38]  1.39333e+07 
                [39]  1.81841e+07 
                [40]  1.73678e+07 
                [41]  1.38866e+07 
                [42]  1.45561e+07 
                [43]  1.66514e+07 
                [44]  1.77473e+07 
                [45]  1.82926e+07 
                [46]  1.92264e+07 
                [47]  1.95075e+07 
                [48]  1.80275e+07 
                [49]  1.94929e+07 
                [50]  2.01234e+07 
                [51]  1.81616e+07 
                [52]  1.87115e+07 
                [53]  1.72166e+07 
                [54]  1.69523e+07 
                [55]  1.73233e+07 
                [56]  1.76594e+07 
                [57]  1.51023e+07 
                [58]  1.61739e+07 
                [59]  2.0506e+07 
                [60]  1.9926e+07 
                [61]  1.89968e+07 
                [62]  2.02707e+07 
                [63]  1.82821e+07 
              RINFO(3) (local estimated flops for the elimination after 
factorization): 
                [0]  5.41177e+09 
                [1]  5.5562e+09 
                [2]  5.42769e+09 
                [3]  4.81711e+09 
                [4]  5.29209e+09 
                [5]  5.0859e+09 
                [6]  5.31089e+09 
                [7]  5.23122e+09 
                [8]  4.88457e+09 
                [9]  5.07962e+09 
                [10]  5.37388e+09 
                [11]  5.19876e+09 
                [12]  5.17109e+09 
                [13]  4.99276e+09 
                [14]  5.57483e+09 
                [15]  5.32825e+09 
                [16]  5.52462e+09 
                [17]  5.03967e+09 
                [18]  4.95626e+09 
                [19]  5.44048e+09 
                [20]  5.29846e+09 
                [21]  4.96019e+09 
                [22]  5.11429e+09 
                [23]  4.51445e+09 
                [24]  4.36668e+09 
                [25]  4.74672e+09 
                [26]  4.97041e+09 
                [27]  5.12432e+09 
                [28]  5.06271e+09 
                [29]  5.1237e+09 
                [30]  5.14926e+09 
                [31]  4.8076e+09 
                [32]  4.87855e+09 
                [33]  4.78543e+09 
                [34]  4.92575e+09 
                [35]  5.25937e+09 
                [36]  5.16189e+09 
                [37]  5.3965e+09 
                [38]  4.53997e+09 
                [39]  5.47973e+09 
                [40]  5.6047e+09 
                [41]  5.23768e+09 
                [42]  5.18053e+09 
                [43]  5.31341e+09 
                [44]  4.87747e+09 
                [45]  4.7188e+09 
                [46]  4.89911e+09 
                [47]  4.99253e+09 
                [48]  4.86713e+09 
                [49]  5.36102e+09 
                [50]  5.54036e+09 
                [51]  5.06688e+09 
                [52]  5.67456e+09 
                [53]  4.96093e+09 
                [54]  5.19924e+09 
                [55]  5.14885e+09 
                [56]  5.1583e+09 
                [57]  4.89255e+09 
                [58]  5.28688e+09 
                [59]  5.30014e+09 
                [60]  5.25875e+09 
                [61]  5.22888e+09 
                [62]  5.30039e+09 
                [63]  5.23254e+09 
              INFO(15) (estimated size of (in MB) MUMPS internal data for 
running numerical factorization): 
              [0] 350 
              [1] 338 
              [2] 332 
              [3] 324 
              [4] 336 
              [5] 333 
              [6] 329 
              [7] 315 
              [8] 318 
              [9] 340 
              [10] 336 
              [11] 337 
              [12] 365 
              [13] 359 
              [14] 335 
              [15] 335 
              [16] 341 
              [17] 338 
              [18] 366 
              [19] 359 
              [20] 352 
              [21] 354 
              [22] 336 
              [23] 314 
              [24] 309 
              [25] 310 
              [26] 306 
              [27] 330 
              [28] 325 
              [29] 329 
              [30] 321 
              [31] 332 
              [32] 338 
              [33] 309 
              [34] 312 
              [35] 313 
              [36] 315 
              [37] 312 
              [38] 313 
              [39] 338 
              [40] 341 
              [41] 313 
              [42] 317 
              [43] 334 
              [44] 336 
              [45] 341 
              [46] 344 
              [47] 342 
              [48] 338 
              [49] 348 
              [50] 348 
              [51] 345 
              [52] 361 
              [53] 343 
              [54] 336 
              [55] 344 
              [56] 349 
              [57] 331 
              [58] 332 
              [59] 365 
              [60] 375 
              [61] 361 
              [62] 374 
              [63] 353 
              INFO(16) (size of (in MB) MUMPS internal data used during 
numerical factorization): 
                [0] 350 
                [1] 338 
                [2] 332 
                [3] 324 
                [4] 336 
                [5] 333 
                [6] 329 
                [7] 315 
                [8] 318 
                [9] 340 
                [10] 336 
                [11] 337 
                [12] 365 
                [13] 359 
                [14] 335 
                [15] 335 
                [16] 341 
                [17] 338 
                [18] 366 
                [19] 359 
                [20] 352 
                [21] 354 
                [22] 336 
                [23] 314 
                [24] 309 
                [25] 310 
                [26] 306 
                [27] 330 
                [28] 325 
                [29] 329 
                [30] 321 
                [31] 332 
                [32] 338 
                [33] 309 
                [34] 312 
                [35] 313 
                [36] 315 
                [37] 312 
                [38] 313 
                [39] 338 
                [40] 341 
                [41] 313 
                [42] 317 
                [43] 334 
                [44] 336 
                [45] 341 
                [46] 344 
                [47] 342 
                [48] 338 
                [49] 348 
                [50] 348 
                [51] 345 
                [52] 361 
                [53] 343 
                [54] 336 
                [55] 344 
                [56] 349 
                [57] 331 
                [58] 332 
                [59] 365 
                [60] 375 
                [61] 361 
                [62] 374 
                [63] 353 
              INFO(23) (num of pivots eliminated on this processor after 
factorization): 
                [0] 35762 
                [1] 32454 
                [2] 28320 
                [3] 28042 
                [4] 30728 
                [5] 31148 
                [6] 30000 
                [7] 17236 
                [8] 19958 
                [9] 35060 
                [10] 31498 
                [11] 31481 
                [12] 42110 
                [13] 42436 
                [14] 35370 
                [15] 31681 
                [16] 32469 
                [17] 32977 
                [18] 46824 
                [19] 45666 
                [20] 40072 
                [21] 40442 
                [22] 30736 
                [23] 24965 
                [24] 16746 
                [25] 19904 
                [26] 20926 
                [27] 26174 
                [28] 27698 
                [29] 29470 
                [30] 26201 
                [31] 28984 
                [32] 32984 
                [33] 17963 
                [34] 18121 
                [35] 18667 
                [36] 18208 
                [37] 19831 
                [38] 21807 
                [39] 35900 
                [40] 31778 
                [41] 18856 
                [42] 21307 
                [43] 31690 
                [44] 32292 
                [45] 34441 
                [46] 37188 
                [47] 36452 
                [48] 33128 
                [49] 36027 
                [50] 37202 
                [51] 35619 
                [52] 32952 
                [53] 32104 
                [54] 30078 
                [55] 38480 
                [56] 39988 
                [57] 24976 
                [58] 26538 
                [59] 48146 
                [60] 43746 
                [61] 42632 
                [62] 43967 
                [63] 37396 
              RINFOG(1) (global estimated flops for the elimination after 
analysis): 3.28665e+11 
              RINFOG(2) (global estimated flops for the assembly after 
factorization): 1.10276e+09 
              RINFOG(3) (global estimated flops for the elimination after 
factorization): 3.28665e+11 
              (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0,0)*(2^0)
              INFOG(3) (estimated real workspace for factors on all processors 
after analysis): 483073412 
              INFOG(4) (estimated integer workspace for factors on all 
processors after analysis): 26232274 
              INFOG(5) (estimated maximum front size in the complete tree): 
3516 
              INFOG(6) (number of nodes in the complete tree): 220574 
              INFOG(7) (ordering option effectively use after analysis): 5 
              INFOG(8) (structural symmetry in percent of the permuted matrix 
after analysis): 100 
              INFOG(9) (total real/complex workspace to store the matrix 
factors after factorization): 481059588 
              INFOG(10) (total integer space store the matrix factors after 
factorization): 26149876 
              INFOG(11) (order of largest frontal matrix after factorization): 
3516 
              INFOG(12) (number of off-diagonal pivots): 0 
              INFOG(13) (number of delayed pivots after factorization): 0 
              INFOG(14) (number of memory compress after factorization): 54 
              INFOG(15) (number of steps of iterative refinement after 
solution): 0 
              INFOG(16) (estimated size (in MB) of all MUMPS internal data for 
factorization after analysis: value on the most memory consuming processor): 
375 
              INFOG(17) (estimated size of all MUMPS internal data for 
factorization after analysis: sum over all processors): 21525 
              INFOG(18) (size of all MUMPS internal data allocated during 
factorization: value on the most memory consuming processor): 375 
              INFOG(19) (size of all MUMPS internal data allocated during 
factorization: sum over all processors): 21525 
              INFOG(20) (estimated number of entries in the factors): 481059588 
              INFOG(21) (size in MB of memory effectively used during 
factorization - value on the most memory consuming processor): 338 
              INFOG(22) (size in MB of memory effectively used during 
factorization - sum over all processors): 19782 
              INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 
              INFOG(24) (after analysis: value of ICNTL(12) effectively used): 
1 
              INFOG(25) (after factorization: number of pivots modified by 
static pivoting): 0 
  linear system matrix = precond matrix:
  Matrix Object:   64 MPI processes
    type: mpiaij
    rows=2004002, cols=2004002
    total: nonzeros=63720324, allocated nonzeros=63720324
    total number of mallocs used during MatSetValues calls =0
      using I-node (on process 0) routines: found 16687 nodes, limit used is 5
 
Norme de U  1 7.37266E-02, L  1  1.00000E+00
 
 Temps total d execution :   198.373291969299     
KSP Object: 64 MPI processes
  type: preonly
  maximum iterations=10000, initial guess is zero
  tolerances:  relative=1e-20, absolute=1e-50, divergence=10000
  left preconditioning
  using NONE norm type for convergence test
PC Object: 64 MPI processes
  type: lu
    LU: out-of-place factorization
    tolerance for zero pivot 2.22045e-14
    matrix ordering: natural
    factor fill ratio given 0, needed 0
      Factored matrix follows:
        Mat Object:         64 MPI processes
          type: mpiaij
          rows=2004002, cols=2004002
          package used to perform factorization: mumps
          total: nonzeros=4.8106e+08, allocated nonzeros=4.8106e+08
          total number of mallocs used during MatSetValues calls =0
            MUMPS run parameters:
              SYM (matrix type):                   0 
              PAR (host participation):            1 
              ICNTL(1) (output for error):         6 
              ICNTL(2) (output of diagnostic msg): 0 
              ICNTL(3) (output for global info):   0 
              ICNTL(4) (level of printing):        0 
              ICNTL(5) (input mat struct):         0 
              ICNTL(6) (matrix prescaling):        7 
              ICNTL(7) (sequentia matrix ordering):7 
              ICNTL(8) (scalling strategy):        0 
              ICNTL(10) (max num of refinements):  0 
              ICNTL(11) (error analysis):          0 
              ICNTL(12) (efficiency control):                         1 
              ICNTL(13) (efficiency control):                         0 
              ICNTL(14) (percentage of estimated workspace increase): 30 
              ICNTL(18) (input mat struct):                           3 
              ICNTL(19) (Shur complement info):                       0 
              ICNTL(20) (rhs sparse pattern):                         0 
              ICNTL(21) (somumpstion struct):                            1 
              ICNTL(22) (in-core/out-of-core facility):               0 
              ICNTL(23) (max size of memory can be allocated locally):0 
              ICNTL(24) (detection of null pivot rows):               0 
              ICNTL(25) (computation of a null space basis):          0 
              ICNTL(26) (Schur options for rhs or solution):          0 
              ICNTL(27) (experimental parameter):                     -8 
              ICNTL(28) (use parallel or sequential ordering):        1 
              ICNTL(29) (parallel ordering):                          0 
              ICNTL(30) (user-specified set of entries in inv(A)):    0 
              ICNTL(31) (factors is discarded in the solve phase):    0 
              ICNTL(33) (compute determinant):                        0 
              CNTL(1) (relative pivoting threshold):      0.01 
              CNTL(2) (stopping criterion of refinement): 1.49012e-08 
              CNTL(3) (absomumpste pivoting threshold):      0 
              CNTL(4) (vamumpse of static pivoting):         -1 
              CNTL(5) (fixation for null pivots):         0 
              RINFO(1) (local estimated flops for the elimination after 
analysis): 
                [0] 9.56571e+09 
                [1] 6.03554e+09 
                [2] 5.28602e+09 
                [3] 4.71458e+09 
                [4] 4.77297e+09 
                [5] 4.62025e+09 
                [6] 4.60478e+09 
                [7] 4.50272e+09 
                [8] 4.83745e+09 
                [9] 5.22409e+09 
                [10] 5.63805e+09 
                [11] 6.02585e+09 
                [12] 6.49054e+09 
                [13] 5.378e+09 
                [14] 4.79934e+09 
                [15] 4.77037e+09 
                [16] 4.84331e+09 
                [17] 4.93195e+09 
                [18] 5.22257e+09 
                [19] 5.25297e+09 
                [20] 5.13269e+09 
                [21] 5.19021e+09 
                [22] 4.75881e+09 
                [23] 4.30376e+09 
                [24] 4.40295e+09 
                [25] 4.36469e+09 
                [26] 4.3722e+09 
                [27] 4.5117e+09 
                [28] 4.6233e+09 
                [29] 4.77495e+09 
                [30] 4.62267e+09 
                [31] 4.51233e+09 
                [32] 4.66611e+09 
                [33] 4.09414e+09 
                [34] 6.08632e+09 
                [35] 6.33603e+09 
                [36] 6.83482e+09 
                [37] 6.10999e+09 
                [38] 4.59808e+09 
                [39] 4.84312e+09 
                [40] 5.93403e+09 
                [41] 4.98752e+09 
                [42] 5.14366e+09 
                [43] 4.4912e+09 
                [44] 4.25993e+09 
                [45] 4.53872e+09 
                [46] 4.2995e+09 
                [47] 4.41656e+09 
                [48] 4.26886e+09 
                [49] 4.9593e+09 
                [50] 5.05505e+09 
                [51] 4.92099e+09 
                [52] 6.05907e+09 
                [53] 4.79346e+09 
                [54] 4.47977e+09 
                [55] 4.69342e+09 
                [56] 4.80231e+09 
                [57] 4.31013e+09 
                [58] 4.46429e+09 
                [59] 4.48957e+09 
                [60] 6.22429e+09 
                [61] 5.87805e+09 
                [62] 7.88576e+09 
                [63] 6.65387e+09 
              RINFO(2) (local estimated flops for the assembly after 
factorization): 
                [0]  1.74461e+07 
                [1]  1.7763e+07 
                [2]  1.61984e+07 
                [3]  1.57714e+07 
                [4]  1.69917e+07 
                [5]  1.72081e+07 
                [6]  1.71679e+07 
                [7]  1.36888e+07 
                [8]  1.35559e+07 
                [9]  1.84207e+07 
                [10]  1.84914e+07 
                [11]  1.7154e+07 
                [12]  2.05592e+07 
                [13]  2.19054e+07 
                [14]  1.81263e+07 
                [15]  1.73837e+07 
                [16]  1.9493e+07 
                [17]  1.81406e+07 
                [18]  2.25729e+07 
                [19]  2.2107e+07 
                [20]  2.02896e+07 
                [21]  2.0936e+07 
                [22]  1.70773e+07 
                [23]  1.43327e+07 
                [24]  1.32055e+07 
                [25]  1.40204e+07 
                [26]  1.39963e+07 
                [27]  1.65377e+07 
                [28]  1.56423e+07 
                [29]  1.67595e+07 
                [30]  1.54397e+07 
                [31]  1.72123e+07 
                [32]  1.75712e+07 
                [33]  1.29832e+07 
                [34]  1.34651e+07 
                [35]  1.31582e+07 
                [36]  1.38729e+07 
                [37]  1.33971e+07 
                [38]  1.40446e+07 
                [39]  1.7352e+07 
                [40]  1.73378e+07 
                [41]  1.42449e+07 
                [42]  1.39507e+07 
                [43]  1.69381e+07 
                [44]  1.74373e+07 
                [45]  1.9306e+07 
                [46]  1.93248e+07 
                [47]  1.96358e+07 
                [48]  1.80179e+07 
                [49]  1.82471e+07 
                [50]  1.9377e+07 
                [51]  1.78114e+07 
                [52]  1.90581e+07 
                [53]  1.72433e+07 
                [54]  1.71118e+07 
                [55]  1.73639e+07 
                [56]  1.76551e+07 
                [57]  1.53239e+07 
                [58]  1.55162e+07 
                [59]  2.03455e+07 
                [60]  2.0396e+07 
                [61]  1.96719e+07 
                [62]  2.07164e+07 
                [63]  1.92856e+07 
              RINFO(3) (local estimated flops for the elimination after 
factorization): 
                [0]  5.6755e+09 
                [1]  5.46605e+09 
                [2]  5.37302e+09 
                [3]  4.96427e+09 
                [4]  5.24315e+09 
                [5]  5.20644e+09 
                [6]  5.17656e+09 
                [7]  5.21729e+09 
                [8]  4.7574e+09 
                [9]  5.04786e+09 
                [10]  5.27601e+09 
                [11]  5.29835e+09 
                [12]  5.39085e+09 
                [13]  5.52531e+09 
                [14]  4.95803e+09 
                [15]  4.65179e+09 
                [16]  5.74241e+09 
                [17]  4.87817e+09 
                [18]  5.08069e+09 
                [19]  5.42944e+09 
                [20]  4.92564e+09 
                [21]  5.38334e+09 
                [22]  4.78469e+09 
                [23]  4.58722e+09 
                [24]  4.68599e+09 
                [25]  4.7149e+09 
                [26]  4.73439e+09 
                [27]  5.15894e+09 
                [28]  5.13626e+09 
                [29]  5.21199e+09 
                [30]  5.03888e+09 
                [31]  4.8458e+09 
                [32]  4.71502e+09 
                [33]  4.60625e+09 
                [34]  5.22997e+09 
                [35]  5.04259e+09 
                [36]  5.29941e+09 
                [37]  5.16152e+09 
                [38]  4.88186e+09 
                [39]  5.09274e+09 
                [40]  5.27315e+09 
                [41]  5.4348e+09 
                [42]  4.97005e+09 
                [43]  5.41421e+09 
                [44]  4.66093e+09 
                [45]  5.19062e+09 
                [46]  5.16503e+09 
                [47]  5.32782e+09 
                [48]  4.78211e+09 
                [49]  4.82626e+09 
                [50]  5.27645e+09 
                [51]  4.83469e+09 
                [52]  5.86354e+09 
                [53]  5.11462e+09 
                [54]  5.28752e+09 
                [55]  5.30904e+09 
                [56]  5.17419e+09 
                [57]  4.99302e+09 
                [58]  5.06845e+09 
                [59]  4.92094e+09 
                [60]  5.33647e+09 
                [61]  5.55645e+09 
                [62]  5.70385e+09 
                [63]  5.58504e+09 
              INFO(15) (estimated size of (in MB) MUMPS internal data for 
running numerical factorization): 
              [0] 350 
              [1] 338 
              [2] 332 
              [3] 324 
              [4] 336 
              [5] 333 
              [6] 329 
              [7] 315 
              [8] 318 
              [9] 340 
              [10] 336 
              [11] 337 
              [12] 365 
              [13] 359 
              [14] 335 
              [15] 335 
              [16] 341 
              [17] 338 
              [18] 366 
              [19] 359 
              [20] 352 
              [21] 354 
              [22] 336 
              [23] 314 
              [24] 309 
              [25] 310 
              [26] 306 
              [27] 330 
              [28] 325 
              [29] 329 
              [30] 321 
              [31] 332 
              [32] 338 
              [33] 309 
              [34] 312 
              [35] 313 
              [36] 315 
              [37] 312 
              [38] 313 
              [39] 338 
              [40] 341 
              [41] 313 
              [42] 317 
              [43] 334 
              [44] 336 
              [45] 341 
              [46] 344 
              [47] 342 
              [48] 338 
              [49] 348 
              [50] 348 
              [51] 345 
              [52] 361 
              [53] 343 
              [54] 336 
              [55] 344 
              [56] 349 
              [57] 331 
              [58] 332 
              [59] 365 
              [60] 375 
              [61] 361 
              [62] 374 
              [63] 353 
              INFO(16) (size of (in MB) MUMPS internal data used during 
numerical factorization): 
                [0] 350 
                [1] 338 
                [2] 332 
                [3] 324 
                [4] 336 
                [5] 333 
                [6] 329 
                [7] 315 
                [8] 318 
                [9] 340 
                [10] 336 
                [11] 337 
                [12] 365 
                [13] 359 
                [14] 335 
                [15] 335 
                [16] 341 
                [17] 338 
                [18] 366 
                [19] 359 
                [20] 352 
                [21] 354 
                [22] 336 
                [23] 314 
                [24] 309 
                [25] 310 
                [26] 306 
                [27] 330 
                [28] 325 
                [29] 329 
                [30] 321 
                [31] 332 
                [32] 338 
                [33] 309 
                [34] 312 
                [35] 313 
                [36] 315 
                [37] 312 
                [38] 313 
                [39] 338 
                [40] 341 
                [41] 313 
                [42] 317 
                [43] 334 
                [44] 336 
                [45] 341 
                [46] 344 
                [47] 342 
                [48] 338 
                [49] 348 
                [50] 348 
                [51] 345 
                [52] 361 
                [53] 343 
                [54] 336 
                [55] 344 
                [56] 349 
                [57] 331 
                [58] 332 
                [59] 365 
                [60] 375 
                [61] 361 
                [62] 374 
                [63] 353 
              INFO(23) (num of pivots eliminated on this processor after 
factorization): 
                [0] 35762 
                [1] 32454 
                [2] 28320 
                [3] 28042 
                [4] 30728 
                [5] 31148 
                [6] 30000 
                [7] 17236 
                [8] 19958 
                [9] 35060 
                [10] 31498 
                [11] 31481 
                [12] 42110 
                [13] 42436 
                [14] 35370 
                [15] 31681 
                [16] 32469 
                [17] 32977 
                [18] 46824 
                [19] 45666 
                [20] 40072 
                [21] 40442 
                [22] 30736 
                [23] 24965 
                [24] 16746 
                [25] 19904 
                [26] 20926 
                [27] 26174 
                [28] 27698 
                [29] 29470 
                [30] 26201 
                [31] 28984 
                [32] 32984 
                [33] 17963 
                [34] 18121 
                [35] 18667 
                [36] 18208 
                [37] 19831 
                [38] 21807 
                [39] 35900 
                [40] 31778 
                [41] 18856 
                [42] 21307 
                [43] 31690 
                [44] 32292 
                [45] 34441 
                [46] 37188 
                [47] 36452 
                [48] 33128 
                [49] 36027 
                [50] 37202 
                [51] 35619 
                [52] 32952 
                [53] 32104 
                [54] 30078 
                [55] 38480 
                [56] 39988 
                [57] 24976 
                [58] 26538 
                [59] 48146 
                [60] 43746 
                [61] 42632 
                [62] 43967 
                [63] 37396 
              RINFOG(1) (global estimated flops for the elimination after 
analysis): 3.28665e+11 
              RINFOG(2) (global estimated flops for the assembly after 
factorization): 1.10276e+09 
              RINFOG(3) (global estimated flops for the elimination after 
factorization): 3.28665e+11 
              (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0,0)*(2^0)
              INFOG(3) (estimated real workspace for factors on all processors 
after analysis): 483073412 
              INFOG(4) (estimated integer workspace for factors on all 
processors after analysis): 26232274 
              INFOG(5) (estimated maximum front size in the complete tree): 
3516 
              INFOG(6) (number of nodes in the complete tree): 220574 
              INFOG(7) (ordering option effectively use after analysis): 5 
              INFOG(8) (structural symmetry in percent of the permuted matrix 
after analysis): 100 
              INFOG(9) (total real/complex workspace to store the matrix 
factors after factorization): 481059588 
              INFOG(10) (total integer space store the matrix factors after 
factorization): 26136333 
              INFOG(11) (order of largest frontal matrix after factorization): 
3516 
              INFOG(12) (number of off-diagonal pivots): 0 
              INFOG(13) (number of delayed pivots after factorization): 0 
              INFOG(14) (number of memory compress after factorization): 48 
              INFOG(15) (number of steps of iterative refinement after 
solution): 0 
              INFOG(16) (estimated size (in MB) of all MUMPS internal data for 
factorization after analysis: value on the most memory consuming processor): 
375 
              INFOG(17) (estimated size of all MUMPS internal data for 
factorization after analysis: sum over all processors): 21525 
              INFOG(18) (size of all MUMPS internal data allocated during 
factorization: value on the most memory consuming processor): 375 
              INFOG(19) (size of all MUMPS internal data allocated during 
factorization: sum over all processors): 21525 
              INFOG(20) (estimated number of entries in the factors): 481059588 
              INFOG(21) (size in MB of memory effectively used during 
factorization - value on the most memory consuming processor): 334 
              INFOG(22) (size in MB of memory effectively used during 
factorization - sum over all processors): 19779 
              INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 
              INFOG(24) (after analysis: value of ICNTL(12) effectively used): 
1 
              INFOG(25) (after factorization: number of pivots modified by 
static pivoting): 0 
              INFOG(28) (after factorization: number of null pivots 
encountered): 0
              INFOG(29) (after factorization: effective number of entries in 
the factors (sum over all processors)): 470143172
              INFOG(30, 31) (after solution: size in Mbytes of memory used 
during solution phase): 202, 10547
              INFOG(32) (after analysis: type of analysis done): 1
              INFOG(33) (value used for ICNTL(8)): 0
              INFOG(34) (exponent of the determinant if determinant is 
requested): 0
  linear system matrix = precond matrix:
  Mat Object:   64 MPI processes
    type: mpiaij
    rows=2004002, cols=2004002
    total: nonzeros=6.37203e+07, allocated nonzeros=6.37203e+07
    total number of mallocs used during MatSetValues calls =0
      using I-node (on process 0) routines: found 16687 nodes, limit used is 5
 
Norme de U  1 1.61172E-02, L  1  1.00000E+00
 
 Temps total d execution :   216.934082031250     





Le 11 déc. 2014 à 11:43, Matthew Knepley <[email protected]> a écrit :

On Thu, Dec 11, 2014 at 4:38 AM, Marc MEDALE <[email protected]> wrote:
Dear PETSC Users,

I have just updated to PETSc-3.5 my research code that uses PETSc for a while but I'm facing an astonishing difference between PETSc-3.4 to 3.5 versions when solving a very ill conditioned algebraic system with MUMPS (4.10.0 in both cases).

The only differences that arise in my fortran source code are the following:
Loma1-medale% diff ../version_3.5/solvEFL_MAN_SBIF.F ../version_3.4/solvEFL_MAN_SBIF.F
336,337d335
<       CALL MatSetOption(MATGLOB,MAT_KEEP_NONZERO_PATTERN,
<      &       PETSC_TRUE,IER)
749,750c747,748
<       CALL KSPSetTolerances(KSP1,TOL,PETSC_DEFAULT_REAL,
<      &     PETSC_DEFAULT_REAL,PETSC_DEFAULT_INTEGER,IER)
---
>       CALL KSPSetTolerances(KSP1,TOL,PETSC_DEFAULT_DOUBLE_PRECISION,
>      &     PETSC_DEFAULT_DOUBLE_PRECISION,PETSC_DEFAULT_INTEGER,IER)
909c907,908
<           CALL KSPSetOperators(KSP1,MATGLOB,MATGLOB,IER)
---
>           CALL KSPSetOperators(KSP1,MATGLOB,MATGLOB,
>      &                         SAME_NONZERO_PATTERN,IER)

When I run the corresponding program versions on 128 cores of our cluster with the same input data and the following command line arguments:
-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -mat_mumps_icntl_8 0

I get the following outputs:
a)  with PETSc-3.4p4:
        L2 norm of solution vector: 7.39640E-02,

b)  with PETSc-3.5p1:
        L2 norm of solution vector: 1.61325E-02

Do I have change something else in updating my code based on KSP from PETSc-3.4 to 3.5 versions?
Do any default values in the PETSc-MUMPS interface have been changed from PETSc-3.4 to 3.5?
Any hints or suggestions are welcome to help me to recover the right results (obtained with PETSc-3.4).

Send the output from -ksp_monitor -ksp_view for both runs. I am guessing that a MUMPS default changed between versions.

  Thanks,

    Matt
 
Thank you very much.

Marc MEDALE.



--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener

Reply via email to