Hi Erik, Yes, this has been discussed a few times before and definitely many people would like to see this feature but the implementation is the tricky part. Having said that, the implementation may be getting more feasible. For toolchain resolution we already have a search in place based on the toolchain hierarchy, doing something similar for versions could leverage a lot of the logic in there (https://github.com/easybuilders/easybuild-framework/blob/master/easybuild/framework/easyconfig/easyconfig.py#L1430). We'd also need a syntax for the easyconfigs, which is something that is currently being discussed for https://github.com/easybuilders/easybuild-framework/pull/2357
The issue I see would be that this would be quite slow since the search space is getting very large (particularly for software with a deep dependency chain) and looking for existing installations would probably have to avoid querying the module tool. To keep that under control, we could start getting clever with Lmod and leveraging Lmod spider (and maybe even consider silently using Lmod behind the scenes in EB for this reason?). That way we can quickly find existing installations (who will also have fully resolved deps in the easyconfig archived in the easybuild/reprod dir of the installation so there'll be no need to keep searching down the tree). That would narrow the search a lot since you either want to re-use something appropriate that's already installed or, if not, you want the latest and greatest. Unfortunately there are potential conflicts here, the allowed versions of a direct dep may conflict with an installed version being used as a dep further down the dependency tree. I think it's an interesting discussion for the upcoming User Meeting in Amsterdam. Alan On 11 December 2017 at 16:30, Erik Smeets <[email protected]<mailto:[email protected]>> wrote: Hi, There are also a lot of dependencies to applications that in my opinion don't need a specific toolchain, but just need to be available. Maybe don't add these to a specific toolchain or version, but a generally avaible toolchain (dummy?). Also, now dependencies are really strict on version, while this is not always necessary. Maybe having something like '>' and/or '<' for version dependencies might help to focus on applications that really need updating. Maybe this is already discussed and decided on a long time ago or maybe this will cause other problems. Regards, Erik On Fri, Nov 24, 2017 at 10:48 AM, Åke Sandgren <[email protected]<mailto:[email protected]>> wrote: It's fairly simple. Use whatever is already available for the selected toolchain. Don't forget to check the git develop branch and any pending PRs when deciding on versions. On 11/24/2017 10:16 AM, Joachim Hein wrote: > Hi, > > Because of interoperability of software, I see an increasing need to fix > software versions beyond what is included in the actual toolchains. I am > discussing standard dependencies, like jpeg libraries, HDF5, netcdf, curl. > So, unless there is strong reasons (e.g. bugs), packages build with a certain > toolchain (e.g. foss/2017a) should use a “preferred version” for standard > dependencies. Using the latest and greatest, which is what I did some time > ago has downsides in the form of module load conflicts. > > The question is how would one administer this (selection of version(s), > communication between contributors, …), without driving the administrative > burden even higher. I like to put this for discussion among the regular > contributors. > > Best wishes > Joachim > -- Ake Sandgren, HPC2N, Umea University, S-90187 Umea, Sweden Internet: [email protected]<mailto:[email protected]> Phone: +46 90 7866134<tel:%2B46%2090%207866134> Fax: +46 90-580 14<tel:%2B46%2090-580%2014> Mobile: +46 70 7716134<tel:%2B46%2070%207716134> WWW: http://www.hpc2n.umu.se -- Dr. Alan O'Cais E-CAM Software Manager Juelich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Juelich, Germany Phone: +49 2461 61 5213 Fax: +49 2461 61 6656 E-mail: [email protected]<mailto:[email protected]> WWW: http://www.fz-juelich.de/ias/jsc/EN ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
