Something is very messed up on Crusher. I've never seen this "Cannot allocate memory", but see it for everything:
13:11 1 main= crusher:/gpfs/alpine/csc314/scratch/adams/petsc2$ ll ls: cannot read symbolic link 'configure.log.bkp': Cannot allocate memory ls: cannot read symbolic link 'make.log': Cannot allocate memory ls: cannot read symbolic link 'configure.log': Cannot allocate memory total 25637 drwxr-xr-x 8 adams adams 4096 Jan 20 08:51 arch-olcf-crusher drwxr-xr-x 8 adams adams 4096 Jan 18 20:23 arch-spock-amd drwxrwxr-x 5 adams adams 4096 Jan 26 19:10 arch-summit-dbg-gnu-cuda .... and 13:11 main= crusher:/gpfs/alpine/csc314/scratch/adams/petsc2$ git fetch remote: Enumerating objects: 648, done. remote: Counting objects: 100% (547/547), done. remote: Compressing objects: 100% (238/238), done. remote: Total 648 (delta 379), reused 442 (delta 308), pack-reused 101 Receiving objects: 100% (648/648), 591.28 KiB | 1.38 MiB/s, done. Resolving deltas: 100% (402/402), completed with 114 local objects. error: cannot update the ref 'refs/remotes/origin/main': unable to append to '.git/logs/refs/remotes/origin/main': Cannot allocate memory >From https://gitlab.com/petsc/petsc On Fri, Jan 28, 2022 at 1:07 PM Mark Adams <[email protected]> wrote: > Crusher has been giving me fits and now I get this error (empty log) > > 13:01 main *= crusher:/gpfs/alpine/csc314/scratch/adams/petsc$ > ../arch-olcf-crusher.py > Traceback (most recent call last): > File "../arch-olcf-crusher.py", line 55, in <module> > configure.petsc_configure(configure_options) > AttributeError: module 'configure' has no attribute 'petsc_configure' > > This is a modified version of the repo configure file. This was working > this AM. > Any idea what could cause this? > > Thanks, > Mark > > #!/usr/bin/python3 > > # Modules loaded by default (on login to Crusher): > # > # 1) craype-x86-trento 9) cce/13.0.0 > # 2) libfabric/1.13.0.0 10) craype/2.7.13 > # 3) craype-network-ofi 11) cray-dsmml/0.2.2 > # 4) perftools-base/21.12.0 12) cray-libsci/21.08.1.2 > # 5) xpmem/2.3.2-2.2_1.16__g9ea452c.shasta 13) PrgEnv-cray/8.2.0 > # 6) cray-pmi/6.0.16 14) DefApps/default > # 7) cray-pmi-lib/6.0.16 15) rocm/4.5.0 > # 8) tmux/3.2a 16) cray-mpich/8.1.12 > # > # We use Cray Programming Environment, Cray compilers, Cray-mpich. > # To enable GPU-aware MPI, one has to also set this environment variable > # > # export MPICH_GPU_SUPPORT_ENABLED=1 > # > # Additional note: If "craype-accel-amd-gfx90a" module is loaded (that is > # needed for "OpenMP offload") - it causes link errors when using 'cc or > hipcc' > # with fortran objs, hence not used > # > > if __name__ == '__main__': > import sys > import os > sys.path.insert(0, os.path.abspath('config')) > import configure > configure_options = [ > '--with-cc=cc', > '--with-cxx=CC', > '--with-fc=ftn', > '--COPTFLAGS=-g -ggdb', > '--CXXOPTFLAGS=-g -ggdb', > '--FOPTFLAGS=-g', > '--with-fortran-bindings=0', > 'LIBS=-L{x}/gtl/lib > -lmpi_gtl_hsa'.format(x=os.environ['CRAY_MPICH_ROOTDIR']), > '--with-debugging=1', > #'--with-64-bit-indices=1', > '--with-mpiexec=srun -p batch -N 1 -A csc314_crusher -t 00:10:00', > '--with-hip', > '--with-hipc=hipcc', > '--download-hypre', > #'--download-hypre-commit=HEAD', > #'--download-hypre-configure-arguments=--enable-unified-memory', > #'--with-hypre-gpuarch=gfx90a', > '--with-hip-arch=gfx90a', > '--download-kokkos', > '--download-kokkos-kernels', > #'--with-kokkos-kernels-tpl=1', > > #'--prefix=/gpfs/alpine/world-shared/geo127/petsc/arch-crusher-opt-cray', # > /gpfs/alpine/phy122/proj-shared/petsc/current/arch-opt-amd-hypre', > ] > configure.petsc_configure(configure_options) > > >
