Re: Guix on clusters and in HPC

Hi Ben, Ben Woodcroft skribis: > I hope the proposal is/was working out. Making progress; it’s due for the end of the year. Until then you’re welcome to make suggestions. :-) > On 03/11/16 23:44, Ludovic Courtès wrote: >> Hi! >> >> Ben Woodcroft

Re: Guix on clusters and in HPC

Hi Ludo, I hope the proposal is/was working out. On 03/11/16 23:44, Ludovic Courtès wrote: Hi! Ben Woodcroft skribis: I'm a little late here, but please do all of the things on that list :) :-) With this suggestion: + for

Re: Guix on clusters and in HPC

Pjotr Prins skribis: > Wrote down a way to distribute software using containers and tar ;) > > https://github.com/pjotrp/guix-notes/blob/master/DISTRIBUTE.org Pretty cool indeed! Recently I thought we could extract the ‘self-contained-tarball’ function you quoted

Re: Guix on clusters and in HPC

Pjotr Prins writes: > Wrote down a way to distribute software using containers and tar ;) > > https://github.com/pjotrp/guix-notes/blob/master/DISTRIBUTE.org > Wow, awesome stuff! I'm going to play around with this. Kind regards, Roel Janssen

Re: Guix on clusters and in HPC

Pjotr Prins writes: > Wrote down a way to distribute software using containers and tar ;) > > https://github.com/pjotrp/guix-notes/blob/master/DISTRIBUTE.org Neat trick! Thanks for sharing. I see that this relies on undocumented behavior, which is the fact that

Re: Guix on clusters and in HPC

Wrote down a way to distribute software using containers and tar ;) https://github.com/pjotrp/guix-notes/blob/master/DISTRIBUTE.org On Wed, Nov 02, 2016 at 04:03:25PM +, Pjotr Prins wrote: > On Wed, Nov 02, 2016 at 09:25:42AM +1000, Ben Woodcroft wrote: > > guix pull: error: build failed:

Re: Guix on clusters and in HPC

Ben Woodcroft skribis: > Has anyone ever managed to get Guix to work inside docker? I attempted > it as I intend on submitting some applications to kbase[0,1], where > developers submit docker files to run their applications within the > "narrative" interface i.e.

Re: Guix on clusters and in HPC

Hi! Ben Woodcroft skribis: > I'm a little late here, but please do all of the things on that list :) :-) > With this suggestion: > > + for > [[https://lists.gnu.org/archive/html/guix-devel/2016-10/msg5.html][CPU-specific > optimizations]] > + somehow

Re: Guix on clusters and in HPC

On Wed, Nov 02, 2016 at 09:25:42AM +1000, Ben Woodcroft wrote: > guix pull: error: build failed: cloning builder process: Operation not > permitted You can set the permissions to run the daemon. Bruno did some work there: https://hub.docker.com/r/bmpvieira/guix/ > That seems to suggest that

Re: Guix on clusters and in HPC

On 26/10/16 21:51, Ludovic Courtès wrote: Ricardo Wurmus skribis: Ludovic Courtès writes: What they suggest is to add Guix support simply by using Guix inside of Docker… Obviously, I’m not a fan of this because of how inelegant this all seems. When it

Re: Guix on clusters and in HPC

Hi, I'm a little late here, but please do all of the things on that list :) With this suggestion: + for [[https://lists.gnu.org/archive/html/guix-devel/2016-10/msg5.html][CPU-specific optimizations]] + somehow support -mtune=native (and even profile-guided optimizations?)

Re: Guix on clusters and in HPC

myglc2 writes: > On 10/26/2016 at 14:08 Ricardo Wurmus writes: > >> At the MDC we’re using SGE and users specify their software environment >> in the job script. The software environment is a Guix profile, so the >> job script usually contains a line to source the profile’s

Re: Guix on clusters and in HPC

On 10/26/2016 at 14:00 Ludovic Courtès writes: > myglc2 skribis: > >> The scheduler that I am most familiar with, SGE, supports the >> proposition that compute hosts are heterogeneous and that they each have >> a fixed software and/or hardware configuration. As a result, users

Re: Guix on clusters and in HPC

On 10/26/2016 at 14:08 Ricardo Wurmus writes: > At the MDC we’re using SGE and users specify their software environment > in the job script. The software environment is a Guix profile, so the > job script usually contains a line to source the profile’s > “etc/profile”, which has the effect of

Re: Guix on clusters and in HPC

Hi! Eric Bavier skribis: >> - non-root usage > > The Singularity project advertises that it does not use a root-owned > daemon http://singularity.lbl.gov/about#no-root-owned-daemon-processes > but it does not in the same section explain that it uses a setuid >

Re: Guix on clusters and in HPC

- non-root usage + file system virtualization needed * map ~/.local/gnu/store to /gnu/store * user name spaces? * [[https://github.com/proot-me/PRoot/][PRoot]]? but performance problems? * common interface, like “guix enter” spawns a shell where /gnu/store is

Re: Guix on clusters and in HPC

Ricardo Wurmus skribis: > Ludovic Courtès writes: > >> Your thoughts about the point about Galaxy? > > I talked to one of the Galaxy core developers at a conference and they > told me they have implemented Docker support recently. Essentially, > they build

Re: Guix on clusters and in HPC

Hi, myglc2 skribis: > The scheduler that I am most familiar with, SGE, supports the > proposition that compute hosts are heterogeneous and that they each have > a fixed software and/or hardware configuration. As a result, users need > to specify resources, such as SW packages

Re: Guix on clusters and in HPC

myglc2 writes: > While SGE is dated and can be a bear to use, it provides a useful > yardstick for HPC/Cluster functionality. So it is useful to consider how > Guix(SD) might impact this model. Presumably a defining characteristic > of GuixSD clusters is that the software

Re: Guix on clusters and in HPC

On 10/18/2016 at 16:20 Ludovic Courtès writes: > Hello, > > I’m trying to gather a “wish list” of things to be done to facilitate > the use of Guix on clusters and for high-performance computing (HPC). The scheduler that I am most familiar with, SGE, supports the proposition that compute hosts

Re: Guix on clusters and in HPC

Ricardo Wurmus writes: > Roel Janssen writes: > >> * Network-aware guix-daemon >> >> From a user's point of view it would be cool to have a network-aware >> guix-daemon. In our cluster, we have a shared storage, on which we have >> the store, but manipulating the store

Re: Guix on clusters and in HPC

Ludovic Courtès writes: > Your thoughts about the point about Galaxy? I talked to one of the Galaxy core developers at a conference and they told me they have implemented Docker support recently. Essentially, they build software in a minimal Docker system and then extract the

Re: Guix on clusters and in HPC

Thomas Danckaert skribis: > From: l...@gnu.org (Ludovic Courtès) > Subject: Guix on clusters and in HPC > Date: Tue, 18 Oct 2016 16:20:43 +0200 > >> So I’ve come up with an initial list of work items going from the >> immediate needs to crazy ideas (batch scheduler

Re: Guix on clusters and in HPC

Hi Roel, Roel Janssen skribis: > Here are some aspects I think we need: > > * Network-aware guix-daemon Of course! > * Profile management > > The abstraction of profiles is an awesome feature of FPM, but the user > interface is missing. We could do better here. > > Switch

Re: Guix on clusters and in HPC

From: l...@gnu.org (Ludovic Courtès) Subject: Guix on clusters and in HPC Date: Tue, 18 Oct 2016 16:20:43 +0200 So I’ve come up with an initial list of work items going from the immediate needs to crazy ideas (batch scheduler integration!) that hopefully make sense to cluster/HPC people. I’d

Re: Guix on clusters and in HPC

Ludovic Courtès writes: > Hello, > > I’m trying to gather a “wish list” of things to be done to facilitate > the use of Guix on clusters and for high-performance computing (HPC). > > Ricardo and I wrote about the advantages, shortcomings, and perspectives > before: > >

Re: Guix on clusters and in HPC

Christopher Allan Webber skribis: > Great! I wonder how much of the need for cluster / HPC stuff overlaps > with the desiderata of "Guix Ops" / "guix deploy"? I think it’s quite different. Cluster nodes usually run a vendor-provided GNU/Linux distro (often the ancient

Re: Guix on clusters and in HPC

Ludovic Courtès writes: > (The reason I’m asking is that I’m considering submitting a proposal at > Inria to work on some of these things.) Great! I wonder how much of the need for cluster / HPC stuff overlaps with the desiderata of "Guix Ops" / "guix deploy"?