Thanks Rich, I may take you up on your offer but I've handed it over to the apps. It seems to be working, but I would not be surprised if there are more problems.
Note, here is something that I heard from ORNL about cray-petsc: *One of the OLCF consultants sent me this table about testing petsc ex1.f90 using pgi and petsc.* *It shows the combination* *pgi 17.5, cray-petsc 3.7.6.2, cudatoolkit 7.5 worked* *At least they are aware there is a problem.* On Fri, Jun 29, 2018 at 8:30 PM Richard Tran Mills <rtmi...@anl.gov> wrote: > Hi Mark, > > I've got an OLCF account but don't have access to Titan. Would it be > helpful for me to apply to get on whatever allocation you are using so I > can poke around at this stuff? (If so, send me a message off list with > details of what I should apply for.) > > Hopefully I could get on there faster than it took them to enable me on > Summit. I had volunteered to help with issues on Summit some time ago, but > it took so long for them to enable my account on there that I didn't matter > by then. > > --Richard > > On Fri, Jun 29, 2018 at 7:12 AM, Mark Adams <mfad...@lbl.gov> wrote: > >> Thanks all, I knew this but its been a while since I've run on this file >> setup. >> >> On Fri, Jun 29, 2018 at 9:15 AM Seung-Hoe Ku <s...@pppl.gov> wrote: >> >>> Try copying files to scratch file system. >>> Home is not available when executive batch job. Probably interactive >>> she'll too. >>> >>> On Fri, Jun 29, 2018 at 7:47 AM Mark Adams <mfad...@lbl.gov> wrote: >>> >>>> I can not even run a job here. This is from an interactive shell: >>>> >>>> 64-pgi ex19-batch1:~/petsc/src/snes/examples/tutorials> make >>>> PETSC_DIR=/autofs/nccs-svm1_home1/adams/petsc PETSC_ARCH=arch-titan-dbg6 >>>> cc -o ex19.o -c -fast -mp >>>> -I/autofs/nccs-svm1_home1/adams/petsc/include >>>> -I/autofs/nccs-svm1_home1/adams/petsc/arch-titan-dbg64-pgi/include >>>> -I/ccs/proj/env003/petscv3.9-dbg64-pgi/include `pwd`/ex19.c >>>> cc -fast -mp -o ex19 ex19.o >>>> -L/autofs/nccs-svm1_home1/adams/petsc/arch-titan-dbg64-pgi/lib >>>> -Wl,-rpath,/ccs/proj/env003/petscv3.9-dbg64-pgi/lib >>>> -L/ccs/proj/env003/petscv3.9-dbg64-pgi/lib -lpetsc -lHYPRE -lflapack >>>> -lfblas -lparmetis -lmetis -lstdc++ -ldl >>>> /autofs/nccs-svm1_home1/adams/petsc/arch-titan-dbg64-pgi/lib/libpetsc.a(dlimpl.o): >>>> In function `PetscDLOpen': >>>> /autofs/nccs-svm1_home1/adams/petsc/src/sys/dll/dlimpl.c:54: warning: >>>> Using 'dlopen' in statically linked applications requires at runtime the >>>> shared libraries from the glibc version used for linking >>>> /autofs/nccs-svm1_home1/adams/petsc/arch-titan-dbg64-pgi/lib/libpetsc.a(send.o): >>>> In function `PetscOpenSocket': >>>> /autofs/nccs-svm1_home1/adams/petsc/src/sys/classes/viewer/impls/socket/send.c:106: >>>> warning: Using 'gethostbyname' in statically linked applications requires >>>> at runtime the shared libraries from the glibc version used for linking >>>> adams@titan-batch1:~/petsc/src/snes/examples/tutorials> aprun -n 1 >>>> ./ex19 >>>> [NID 02258] 2018-06-29 07:45:09 Exec ./ex19 failed: chdir >>>> /autofs/nccs-svm1_home1/adams/petsc/src/snes/examples/tutorials No such >>>> file or directory >>>> adams@titan-batch1:~/petsc/src/snes/examples/tutorials> >>>> >>>> Seung-Hoe runs here, maybe he can give me advice on what is wrong here. >>>> >>>> >>>> On Thu, Jun 28, 2018 at 7:53 PM Smith, Barry F. <bsm...@mcs.anl.gov> >>>> wrote: >>>> >>>>> >>>>> 1) We need to get this PGI compiler into our regular testing. It >>>>> doesn't like some Fortran code the other compilers are ok with >>>>> >>>>> 2) Mark, likely you just need to submit the examples manually to >>>>> run them. cd src/snes/examples/tutorials; make ex19 then use the batch >>>>> system to submit the example >>>>> >>>>> Barry >>>>> >>>>> >>>>> > On Jun 28, 2018, at 5:35 PM, Mark Adams <mfad...@lbl.gov> wrote: >>>>> > >>>>> > I am trying to run on Titan and I am having problems running the >>>>> tests. I get this output. It says it can not find a directory, but I do an >>>>> ls in after the error and you can see that it is there. >>>>> > >>>>> > I've attached the logs. >>>>> > >>>>> > Any ideas? >>>>> > Thanks, >>>>> > Mark >>>>> > >>>>> > adams@titan-login6:~/petsc> make >>>>> PETSC_DIR=/ccs/proj/env003/petscv3.9-opt64-pgi PETSC_ARCH="" test >>>>> > Running test examples to verify correct installation >>>>> > Using PETSC_DIR=/ccs/proj/env003/petscv3.9-opt64-pgi and PETSC_ARCH= >>>>> > *******************Error detected during compile or >>>>> link!******************* >>>>> > See http://www.mcs.anl.gov/petsc/documentation/faq.html >>>>> > /ccs/home/adams/petsc/src/snes/examples/tutorials ex19 >>>>> > >>>>> ********************************************************************************* >>>>> > cc -o ex19.o -c -fast -mp >>>>> -I/ccs/proj/env003/petscv3.9-opt64-pgi/include >>>>> -I/ccs/proj/env003/petscv3.9-opt64-pgi/include >>>>> -I/ccs/proj/env003/petscv3.9-opt64-pgi/include `pwd`/ex19.c >>>>> > cc -fast -mp -o ex19 ex19.o >>>>> -L/ccs/proj/env003/petscv3.9-opt64-pgi/lib >>>>> -Wl,-rpath,/ccs/proj/env003/petscv3.9-opt64-pgi/lib >>>>> -L/ccs/proj/env003/petscv3.9-opt64-pgi/lib -lpetsc -lHYPRE -lflapack >>>>> -lfblas -lparmetis -lmetis -lstdc++ -ldl >>>>> > /ccs/proj/env003/petscv3.9-opt64-pgi/lib/libpetsc.a(dlimpl.o): In >>>>> function `PetscDLOpen': >>>>> > /autofs/nccs-svm1_home1/adams/petsc/src/sys/dll/dlimpl.c:53: >>>>> warning: Using 'dlopen' in statically linked applications requires at >>>>> runtime the shared libraries from the glibc version used for linking >>>>> > /ccs/proj/env003/petscv3.9-opt64-pgi/lib/libpetsc.a(send.o): In >>>>> function `PetscOpenSocket': >>>>> > >>>>> /autofs/nccs-svm1_home1/adams/petsc/src/sys/classes/viewer/impls/socket/send.c:101: >>>>> warning: Using 'gethostbyname' in statically linked applications requires >>>>> at runtime the shared libraries from the glibc version used for linking >>>>> > /bin/rm -f ex19.o >>>>> > Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 >>>>> MPI process >>>>> > See http://www.mcs.anl.gov/petsc/documentation/faq.html >>>>> > [NID 18525] 2018-06-28 18:25:39 Exec ./ex19 failed: chdir >>>>> /autofs/nccs-svm1_home1/adams/petsc/src/snes/examples/tutorials No such >>>>> file or directory >>>>> > Possible error running C/C++ src/snes/examples/tutorials/ex19 with 2 >>>>> MPI processes >>>>> > See http://www.mcs.anl.gov/petsc/documentation/faq.html >>>>> > [NID 18525] 2018-06-28 18:25:46 Exec ./ex19 failed: chdir >>>>> /autofs/nccs-svm1_home1/adams/petsc/src/snes/examples/tutorials No such >>>>> file or directory >>>>> > 1,5c1 >>>>> > < lid velocity = 0.0016, prandtl # = 1., grashof # = 1. >>>>> > < 0 SNES Function norm 0.0406612 >>>>> > < 1 SNES Function norm 4.12227e-06 >>>>> > < 2 SNES Function norm 6.098e-11 >>>>> > < Number of SNES iterations = 2 >>>>> > --- >>>>> > > [NID 18525] 2018-06-28 18:25:55 Exec ./ex19 failed: chdir >>>>> /autofs/nccs-svm1_home1/adams/petsc/src/snes/examples/tutorials No such >>>>> file or directory >>>>> > /ccs/home/adams/petsc/src/snes/examples/tutorials >>>>> > Possible problem with ex19_hypre, diffs above >>>>> > ========================================= >>>>> > *******************Error detected during compile or >>>>> link!******************* >>>>> > See http://www.mcs.anl.gov/petsc/documentation/faq.html >>>>> > /ccs/home/adams/petsc/src/snes/examples/tutorials ex5f >>>>> > ********************************************************* >>>>> > ftn -c -fast -mp -I/ccs/proj/env003/petscv3.9-opt64-pgi/include >>>>> -I/ccs/proj/env003/petscv3.9-opt64-pgi/include >>>>> -I/ccs/proj/env003/petscv3.9-opt64-pgi/include -o ex5f.o ex5f.F90 >>>>> > PGF90-S-0155-Could not resolve generic procedure dmdacreate2d >>>>> (ex5f.F90: 120) >>>>> > PGF90-S-0450-Argument number 2 to snesgetiterationnumber: kind >>>>> mismatch (ex5f.F90: 180) >>>>> > 0 inform, 0 warnings, 2 severes, 0 fatal for main >>>>> > PGF90-S-0155-Could not resolve generic procedure matsetvalueslocal >>>>> (ex5f.F90: 464) >>>>> > PGF90-S-0155-Could not resolve generic procedure matsetvalueslocal >>>>> (ex5f.F90: 477) >>>>> > 0 inform, 0 warnings, 2 severes, 0 fatal for formjacobianlocal >>>>> > gmake[3]: [ex5f.o] Error 2 (ignored) >>>>> > ftn -fast -mp -o ex5f ex5f.o >>>>> -L/ccs/proj/env003/petscv3.9-opt64-pgi/lib >>>>> -Wl,-rpath,/ccs/proj/env003/petscv3.9-opt64-pgi/lib >>>>> -L/ccs/proj/env003/petscv3.9-opt64-pgi/lib -lpetsc -lHYPRE -lflapack >>>>> -lfblas -lparmetis -lmetis -lstdc++ -ldl >>>>> > /usr/bin/ld: cannot find ex5f.o: No such file or directory >>>>> > /usr/bin/sha1sum: ex5f: No such file or directory >>>>> > gmake[3]: [ex5f] Error 2 (ignored) >>>>> > /bin/rm -f ex5f.o >>>>> > Completed test examples >>>>> > adams@titan-login6:~/petsc> ll >>>>> /autofs/nccs-svm1_home1/adams/petsc/src/snes/examples/tutorials >>>>> > total 1864 >>>>> > -rw-rw-r-- 1 adams adams 11134 May 3 14:26 ex1.c >>>>> > drwxr-xr-x 3 adams adams 4096 May 3 14:26 ex10d >>>>> > -rw-rw-r-- 1 adams adams 92615 May 3 14:26 ex12.c >>>>> > -rw-r--r-- 1 adams adams 19324 Jun 21 17:59 ex13.c >>>>> > -rw-rw-r-- 1 adams adams 20550 May 3 14:26 ex14.c >>>>> > >>>>> > <make.log><configure.log> >>>>> >>>>> -- >>> Sent from Iphone >>> >> >
