Same text also in the attachment. Barry
David,
Nathan Collier has kindly run a test on Titan, Satish on Mira and Hopper,
and Victor on Ranger with a basic optimized build of PETSc (all C code)
Please find below some configure and make timings from the latest PETSc
master.
The Titan times for both configure and make are unacceptable. For total
build time Titan is 3.5 times slower than Mira and Hopper and at least 10 times
slower than laptops. The "time" results on Titan are disturbing
configure
real 14m32.169s (since the user + sys time is much less than real time,
what is it waiting on?)
user 1m51.527s
sys 3m40.734s
make
real 15m56.004s
user 8m8.971s
sys 52m42.734s (why so much?)
which I read as either the filesystem or the compiler system (location of the
compilers, license server of the compilers, ...) is really badly configured.
The Hopper configure time with the default TMPDIR=/scratch/scratchdirs/balay
is is unacceptable but if you actually use the real /tmp it becomes somewhat
reasonable.
Feel free to share this information with local experts,
I suggest you view the below table in a fixed width font editor like Emacs or
Vi so the columns line up.
configure time make time Total compilers
filesystem
Titan 14m32s 15m56s 30m28s Intel 14
/lustre/atlas1/geo103/proj-shared/
41m38s 9m5s 50m43s
/ccs/home/ (no load on login node)
13m
(no load on a different login node)
Mira 6m59s 1m49s 8m48s IBM
/gpfs/mira-home/
Hopper 23m17 1m45s 25m2s
/global/u2/b/balay/petsc.clone default TMPDIR=/scratch/scratchdirs/balay
6m17s 1m39s 7m57s
manually set TMPDIR=/tmp
NSF Ranger UT Austin 5m10s 1m28s 6m38s
default, whatever it is
Linux laptop 53s 1m13s 2m6s Gnu
compile and compiler local
Apple laptop 1m14s 54s 2m8s clang
compile and compiler local
Linux workstation 1m11s 22s 1m33s Gnu
compile and compiler local
1m37s 29s 2m6s Gnu
compile directory local; compiler directory remote
3m11s 25s 3m36s Intel 13
compile directory local; compiler directory remote
PETSc has about 1000 source files that need compiling
The configure is essentially sequential, the make extremely parallel.
During configure the source code is on the listed file system, all .o and
executables are on /tmp
During the make the source code and all .o are on the listed file system
> On Feb 25, 2015, at 11:23 AM, David E. Bernholdt <[email protected]> wrote:
>
> At the kick-off meetings, one of the general complaints I heard
> expressed about the facilities was the slow build times compared to
> personal systems.
>
> If you have this complaint and are an OLCF user, and are willing to work
> with us a little to try to understand your experience in more detail,
> please contact me (individually, not reply-all).
>
> This is a facility thing, not an IDEAS thing, so I can't speak for the
> other facilities. But we've recently received some other similar
> comments, and we're trying to dig into what's happening.
>
> Thanks
> --
> David E. Bernholdt | Email: [email protected]
> Oak Ridge National Laboratory | Phone: +1 865-574-3147
> http://www.csm.ornl.gov/~bernhold | Fax: +1 865-576-5491
> _______________________________________________
> Ideas-team mailing list
> [email protected]
> https://lists.mcs.anl.gov/mailman/listinfo/ideas-team
timings
Description: Binary data
