Performance studies are enormously difficult to do well; which is why there are so few good ones out there. And unless you fall into the LINPACK benchmark or hit upon Streams the rewards of doing an excellent job are pretty thin. Even Streams was not properly maintained for many years, you could not just get it and use it out of the box for a variety of purposes (which is why PETSc has its hacked-up ones). I submit a properly performance study is a full-time job and everyone always has those.
> On Jan 22, 2022, at 2:11 PM, Jed Brown <j...@jedbrown.org> wrote: > > Barry Smith <bsm...@petsc.dev> writes: > >>> On Jan 22, 2022, at 12:15 PM, Jed Brown <j...@jedbrown.org> wrote: >>> Barry, when you did the tech reports, did you make an example to reproduce >>> on other architectures? Like, run this one example (it'll run all the >>> benchmarks across different sizes) and then run this script on the output >>> to make all the figures? >> >> It is documented in >> https://www.overleaf.com/project/5ff8f7aca589b2f7eb81c579 You may need to >> dig through the submit scripts etc to find out exactly. > > This runs a ton of small jobs and each job doesn't really preload, but > instead of loops in job submission scripts, the loops could be inside the C > code and it could directly output tabular data. This would run faster and be > easier to submit and analyze. > > https://gitlab.com/hannah_mairs/summit-performance/-/blob/master/summit-submissions/submit_gpu1.lsf > > It would hopefully also avoid writing the size range manually over here in > the analysis script where it has to match exactly the job submission. > > https://gitlab.com/hannah_mairs/summit-performance/-/blob/master/python/graphs.py#L8-9 > > > We'd make our lives a lot easier understanding new machines if we put into > the design of performance studies just a fraction of the kind of thought we > put into public library interfaces.
