On 5/5/21 5:21 PM, Konstantin Olchanski wrote:
On Tue, May 04, 2021 at 11:00:00PM +0100, Andrew C Aitchison wrote:
On Tue, 4 May 2021, Konstantin Olchanski wrote:
[snip]
(OK, C++20 support in g++ 10.2.1 is "experimental).
And so what?
I can take SL-6 and graft modern versions of all important packages,
one does not even need the devtoolset, GCC is easy to build from sources.
But this is no longer "SL-6", it is "SL-6-KO1", at best.
Same thing, "CentOS-7 with devtoolset, php from webtatic, python from pip, kernel
from ELREPO, etc" is not CentOS-7.
It is an irreproducible Franken-monster-bashed-together-locally thing.
Is this the new standard, the best way to go, "the new thing" for production
environments?
I would say, no. The way forward is to use something like
https://urldefense.proofpoint.com/v2/url?u=https-3A__spack.io_&d=DwICaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=Rcl3bOlhZYTsg6ao7N9s2R8gMaZj5RFHR3ZfE-XUUZg&s=XrKMgW2x7TS-6ye6hlykdflYSWiGTXaDqw3_WO5bTZw&e= for reproducible builds of software. Or better yet,
starting the difficult process of moving user applications into
Singularity containers (https://urldefense.proofpoint.com/v2/url?u=https-3A__sylabs.io_&d=DwICaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=Rcl3bOlhZYTsg6ao7N9s2R8gMaZj5RFHR3ZfE-XUUZg&s=auYx06d5nuVcpRlexBCYemaHd0-4W213prqtSaLByHA&e= ). And getting Spack to build
the Singularity images is even better! Both of those are fully Open
Source tools with really good community support and free online training.
Once you can get the user applications into a container, you can
abstract out the operating system (mostly; still needs to be Linux
kernel - usually). Since Singularity is designed with HPC in mind,
performance is fantastic.
We took an app that was built for RHEL 6, built it in a Singularity
container, and can now run it on any Linux distro. As we move more of
our user apps into Singularity containers we can start upgrading the OS
and tools underneath the HPC environment without users ever knowing
something changed (hopefully they notice the improvements).
Not saying that there isn't a learning curve for those creating the
containers. I'm still not there in understanding it all and the
container world is huge and varied. But it helped to just stick to
Singularity and well establish formats until I got my head around it.
And we haven't gotten to the point of letting users do it themselves
yet. It's still an admin-only creation process. But we are getting there
and the users don't have a clue how the app is installed/tweaked/tuned -
they just know it works.
~Stack~
