Just a point of information on this. I recently re-configured PETSc (no external packages) on a local disk here on Shaheen and it took 3 minutes:
xxx=========================================================================xxx Configure stage complete. Now build PETSc libraries with: make PETSC_DIR=/tmp/petsc-3.1-p5 PETSC_ARCH=linux-gnu-c-debug all xxx=========================================================================xxx ./configure_shaheen.py 84.36s user 85.01s system 89% cpu 3:08.26 total At some point we might want to figure out why GPFS is so much slower. A On Thu, Dec 2, 2010 at 11:34 PM, John R. Cary <cary at txcorp.com> wrote: > On 12/2/2010 1:21 PM, Barry Smith wrote: >> >> On Dec 2, 2010, at 1:59 PM, John R. Cary wrote: >> >>> Anyway, given the pokeyness of the FENs on surveyor, it should be some >>> time before midnight! >> >> ? This is completely unacceptable! PETSc takes less than 10 minutes to >> configure and compile on my $2000 laptop (and probably half as long on >> Satish's $1000 laptop). On a $100 million machine it takes one half a day? >> >> ? That is frankly disgusting and indicates extreme problems with the >> configuration of that machine. Why has this problem not been fixed years >> ago? >> > > Well, I may be exaggerating a bit :-), but I started configuring at: > > -rwxrw-r-- ?1 cary users ? ? 924 Dec ?2 13:55 > surveyor.alcf.anl.gov-petsc-ser.sh > > and it is not done at > > Thu Dec ?2 14:33:00 CST 2010 > > This is just the petsc part, not all the packages, so it really does take a > long time, > I find. > > I do find that the XL compilers are both slow and buggy. ?I have submitted > bug > reports but they seem to disappear into a black hole at IBM. ?For example, I > submitted alcf-support > ticket #32756 on August 11, 2009, complaining that the mpi.mod in > /bgsys/drivers/V1R3M0_460_2008-081112P/ppc/comm/default/include/mpi.mod > was built with gfortran and incompatible with XLF, even though it is > called by the XLF compiler wrapper. ?The guys at ALCF contacted IBM about > this, > but here it is, over a year later, and the same problem is still present. > ?It seems > like IBM is just not responsive. > > OTOH, both Ray and Vitali have been very helpful, without their help I might > not have found my way around several compiler bugs and other issues. > > .....John > > > > > > > >>> John >>> >>>> satish >>>> >>>> On Thu, 2 Dec 2010, Matthew Knepley wrote: >>>> >>>>> We always always always always need configure.log. >>>>> >>>>> ? ?Matt >>>>> >>>>> On Thu, Dec 2, 2010 at 8:41 AM, John R. Cary<cary at txcorp.com> ? wrote: >>>>> >>>>>> I am trying to link facets on a FEN of surveyor.alcf.anl.gov. ?It ends >>>>>> with the error, >>>>>> >>>>>> >>>>>> /gpfs/home/projects/facets/surveyor/contrib-xlc-9.0/petsc-3.1-p4-ser/lib/libsuperlu_4.0.a(dldperm.o): >>>>>> In function `dldperm': >>>>>> >>>>>> /gpfs/home/cary/facetspkgs/builds/petsc-3.1-p4/ser/externalpackages/SuperLU_4.0/SRC/dldperm.c:127: >>>>>> undefined reference to `mc64id_' >>>>>> >>>>>> /gpfs/home/cary/facetspkgs/builds/petsc-3.1-p4/ser/externalpackages/SuperLU_4.0/SRC/dldperm.c:134: >>>>>> undefined reference to `mc64ad_' >>>>>> >>>>>> which indicates that the SuperLU compiled with PETSc did >>>>>> not get the fortran underscoring flag correct (which should >>>>>> be no underscore with xlf). ?nm shows >>>>>> >>>>>> login1.surveyor$ nm >>>>>> >>>>>> /gpfs/home/projects/facets/surveyor/contrib-xlc-9.0/petsc-3.1-p4-ser/lib/libsuperlu_4.0.a >>>>>> | grep mc64 >>>>>> ? ? ? ? ? ? ? ?U mc64ad_ >>>>>> ? ? ? ? ? ? ? ?U mc64id_ >>>>>> mc64ad.o: >>>>>> 0000000000000018 D mc64ad >>>>>> 0000000000000030 D mc64bd >>>>>> 0000000000000048 D mc64dd >>>>>> 0000000000000060 D mc64ed >>>>>> 0000000000000078 D mc64fd >>>>>> 0000000000000000 D mc64id >>>>>> 00000000000000c0 D mc64qd >>>>>> 0000000000000090 D mc64rd >>>>>> 00000000000000a8 D mc64sd >>>>>> 00000000000000d8 D mc64ud >>>>>> 00000000000000f0 D mc64wd >>>>>> ? ? ? ? ? ? ? ?U mc64ad_ >>>>>> ? ? ? ? ? ? ? ?U mc64id_ >>>>>> >>>>>> that the underscored symbol is being called, but the >>>>>> underscore-free symbol is what was defined. >>>>>> >>>>>> My PETSc configure line was >>>>>> >>>>>> #!/bin/sh >>>>>> >>>>>> /gpfs/home/cary/facetspkgs/builds-surveyor-xlc/facetspkgs/petsc-3.1-p4/ser/configure >>>>>> \ >>>>>> >>>>>> --prefix=/gpfs/home/projects/facets/surveyor/contrib-xlc-9.0/petsc-3.1-p4-ser >>>>>> \ >>>>>> --with-mpi=0 \ >>>>>> --with-debugging=0 \ >>>>>> --with-x=0 \ >>>>>> --with-cc='xlc_r' \ >>>>>> --with-cxx='xlC_r' \ >>>>>> --with-fc='xlf_r' \ >>>>>> --COPTFLAGS='-O2 -g' \ >>>>>> --download-superlu \ >>>>>> >>>>>> --with-lapack-lib=/home/projects/facets/intrepid/contrib/lapack-ser/lib/liblapack.a >>>>>> \ >>>>>> >>>>>> --with-blas-lib=/home/projects/facets/intrepid/contrib/lapack-ser/lib/libblas.a >>>>>> \ >>>>>> >>>>>> PETSC_DIR=/gpfs/home/cary/facetspkgs/builds-surveyor-xlc/facetspkgs/petsc-3.1-p4/ser >>>>>> \ >>>>>> PETSC_ARCH=facets-ser \ >>>>>> --CFLAGS='-q64 -qlanglvl=redefmac' \ >>>>>> --CXXFLAGS='-q64 -qlanglvl=redefmac' \ >>>>>> --FFLAGS='-q64 -qextname=flush' \ >>>>>> --with-shared=1 >>>>>> >>>>>> so I think that PETSc had enough info to figure out the underscoring. >>>>>> Perhaps this is a bug. >>>>>> >>>>>> But regardless, is there a workaround? >>>>>> >>>>>> Thx....John >>>>>> >>>>> >>>>> >> > >
