Catonano <caton...@gmail.com> writes: > Ricardo, I don't understand the problem you're raising here (I didn't read > the article yet, though) > > Would you mind to elaborate on that ? > > Why would you want to record the environment ?
I want to record the detected build environment so that I can restore it at execution time. Autoconf provides macros that probe the environment and record the full path to detected tools. For example, I’m looking for Samtools, and the user may provide a particular variant of Samtools at configure time. I record the full path to the executable at configure time and embed that path in a configuration file that is read when the pipeline is run. This works fine for tools, but doesn’t work very well at all for modules in language environments. Take R for example. I can detect and record the location of the R and Rscript executables, but I cannot easily record the location of build-time R packages (such as r-deseq2) in a way that allows me to rebuild the environment at runtime. Instead of writing an Autoconf macro that records the exact location of each of the detected R packages and their dependencies I chose to solve the problem in Guix by wrapping the pipeline executables in R_SITE_LIBS, because I figured that on systems without Guix you aren’t likely to install R packages into separate unique locations anyway — on most systems R packages end up being installed to one and the same directory. I think the desire to restore the configured environment at runtime is valid and we do this all the time when we run binaries that have embedded absolute paths (to libraries or other tools). It’s just that it gets pretty awkward to do this for things like R packages or Python modules (or Guile modules for that matter). The Guix workflow language solves this problem by depending on Guix for software deployment. For PiGx we picked Snakemake early on and it does not have a software deployment solution (it expects to either run inside a suitable environment that the user provides or to have access to pre-built Singularity application bundles). I don’t like to treat pipelines like some sort of collection of scripts that must be invoked in a suitable environment. I like to see pipelines as big software packages that should know about the environment they need, that can be configured like regular tools, and thus only require the packager to assemble the environment, not the end-user. -- Ricardo