On 30/09/16 12:57, Pablo Escobar Lopez wrote:
2016-09-30 12:32 GMT+02:00 Jack Perdue <[email protected]
<mailto:[email protected]>>:
Not saying they are well tested or the best builds but having
repos that you could enable/disable at will to search for easyconfigs
would be nice.
I like the idea! Being able to define an extra list of git repos or
local folders for "eb --search" sounds nice and should help for
collaboration. Many times some easyconfigs are difficult to get merged
to the main repo for technical reason or lack of time and this could
help (e.g. I don't use to search in Jack's published easyconfigs but
if I could add them as an extra "search repo" it would be helpful)
This basically boils down to adding support for URLs rather than only
filepaths to pass to --robot-paths.
It's a good idea, but someone will have to pick up on it.
If you're willing to sync with the upstream repo outside of EasyBuild
(via an rsync cron job for example), this can already be done now: just
list the location in --robot-paths.
On 09/30/2016 05:20 AM, Joachim Hein wrote:
> Looking at your argument, the alternative to my original
> proposal might be an option to not list “attic” items.
>
IMHO this would work better. If anyone gets annoyed because he gets
too many search results from "attic" he would look at the docs to see
if this can be disabled. The other way around I think it wouldn't work
that good because people could just don't notice that there is an
option to get extra search results. I am thinking about new easybuild
users. For we two or anyone else following this discussion both
options would work because we are aware of this detail.
I agree that "eb --search foss" provides too much output. I always try
to search for the app name and not the toolchain name and then do some
grep with the ouput. Btw, side node, whatever is the final decision
about how "eb --search" should work it would be nice to try to keep it
"grep friendly"
Pablo's point about not including the archived (I personally prefer this
term of 'attic') easyconfigs in --search is a good point imho, I agree
that not including them would make people miss them totally.
On the other hand, Joachim's point of not making --search too verbose
makes sense too.
In addition, my concern is about the order in which the archived vs
normal easyconfigs should be printed in the --search output...
This may sound silly, but including the list of matching archived
easyconfigs last in the output will somehow obscure the 'normal'
easyconfigs, because you have to 'scroll' back up to see the normal ones
that are recommended. On the other hand, listing the archived
easyconfigs first in the output, to let the 'normal' easyconfigs be
listed last and thus easier to spot, may be misinterpreted as giving
them preference (since they're mentioned first).
So, the idea of a middle ground popped into my head, something like this:
$ eb --search ^foo
...
* /path/to/foo-1.2.3.eb
* /path/to/foo-1.2.4.eb
Note: additional matching archived easyconfigs were found, use "eb
--search ^foo --consider-archived-easyconfigs" to see those too.
$ eb --search ^foo --consider-archived-easyconfigs
...
* /path/to/foo-1.2.3.eb
* /path/to/foo-1.2.4.eb
Archived easyconfigs:
* /path/to/foo-0.9.8.eb
Note: it is *not* recommended to use these archived easyconfigs,
see http://easybuild.readthedocs.io/en/latest/Archived_easyconfigs.html
for more information.
With this in place, you could chose to always consider archived
easyconfigs (for example by setting
$EASYBUILD_CONSIDER_ARCHIVED_EASYCONFIGS=1).
The configuration option is long enough that you won't do this via
muscle memory though. ;-)
Thoughts?
regards,
Kenneth
regards,
Pablo.
> Anyway, even nowadays I treat EB configs/block that are old as
“as is”
> and do not report them as broken. I have seen a fair number of
> failures in about 1/2 year of easybuilding. I don’t even keep a
> record. Not worth anyone’s time I think.
>
> Best wishes
> Joachim
>
>
>>
>> My fear about not searching by default in legacy easyconfigs is that
>> many people would skip the extra option needed to search in the
>> legacy easyconfigs and they wouldn't notice that it's there. Many
>> times even if the legacy easyconfig doesn't work it can be
useful as
>> a start point to write a new one so it's useful to find them
easily.
>> I know many of us are sysadmins and we like to reply RTFM but
>> easybuild has many many different config flags and it's easy to
skip
>> this one ( even for a sysadmin ;)
>>
>> regards,
>> Pablo.
>>
>> 2016-09-30 11:41 GMT+02:00 Joachim Hein
<[email protected] <mailto:[email protected]>
>> <mailto:[email protected] <mailto:[email protected]>>>:
>>
>> Hi Kenneth,
>>
>> I had a look into the notes and think some extra discussion is
>> required on the “discontinuation business”. Not sure where to
>> start this, so I am replying here.
>>
>> I occasionally get requests for “historic” and “fossile”
versions
>> of packages - not all of them totally unreasonable, e.g. API
>> change in application. Examples include OpenFOAM and Gromacs.
>> Prior to EasyBuild I would have said no, unless you give me a
>> strong reason.
>>
>> With EasyBuild, I check whether a to the user acceptable
version
>> is in the “back catalog” and try to easybuild it. If it builds
>> without much issue I roll it out. If it breaks I am not worse
>> off than without EasyBuild. Typically I build with the “old”
>> machinery of the time (that is what it was tested with) -
>> installing a legacy GCC and OpenMPI in EB is typically easy.
>>
>> I think here are two big selling points of EB, which we perhaps
>> not even push enough.
>>
>> * Better user experience - many requests for legacy versions
>> can easily be supported.It is typically easier to just
>> build it instead of discussing with the user.
>> * Building software with a tried and tested software stack
>> gives stability against bugs introduced in the latest version
>>
>>
>> So, how do you plan to do the discontinuation? Removing
>> toolchain versions will break everything that builds on that
>> toolchain. Keeping them as “unsupported” seems the way
forward to
>> me. That is, it is still available in some form but no longer
>> tested and fixed. If they still work, good, if they are
broken:
>> work around it. Introducing an “attic” type concept might
be an
>> idea. Depreciated packages get to the attic. A typical “eb -S”
>> would not show them, but e.g. an extra “attic” or “antique”
>> option would.
>>
>> Best wishes
>> Joachim
>>
>>
>>
>>> On 29 Sep 2016, at 18:36, Kenneth Hoste
<[email protected] <mailto:[email protected]>
>>> <mailto:[email protected] <mailto:[email protected]>>>
wrote:
>>>
>>> notes are available at
>>>
https://github.com/hpcugent/easybuild/wiki/Conference-call-notes-20160929
<https://github.com/hpcugent/easybuild/wiki/Conference-call-notes-20160929>
>>>
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild_wiki_Conference-2Dcall-2Dnotes-2D20160929&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=N2C0HQpIsDmyOZxHfPYSEoQ0RWTM6m-qFdrDeiDdQLM&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild_wiki_Conference-2Dcall-2Dnotes-2D20160929&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=N2C0HQpIsDmyOZxHfPYSEoQ0RWTM6m-qFdrDeiDdQLM&e=>>
>>>
>>> Next conf call is planned for Wed Oct 12th 5pm CET, cfr.
>>> https://plus.google.com/events/cir0grjke0kbfutgmgp2v9nfm3s
<https://plus.google.com/events/cir0grjke0kbfutgmgp2v9nfm3s>
>>>
<https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_events_cir0grjke0kbfutgmgp2v9nfm3s&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=lqHDMMXzR2b_QOl3MWdeGZ6-K4fq1p9r90SzNOaen7k&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_events_cir0grjke0kbfutgmgp2v9nfm3s&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=lqHDMMXzR2b_QOl3MWdeGZ6-K4fq1p9r90SzNOaen7k&e=>>
>>>
>>> On 27/09/16 21:03, Kenneth Hoste wrote:
>>>> Dear EasyBuilders,
>>>>
>>>> The next EasyBuild conf call is planned for *Thursday*
>>>> September 29th 2016, 5pm CET;
>>>> see also
>>>> https://plus.google.com/events/c8tqca652anjfqblifjam5q9s6k
<https://plus.google.com/events/c8tqca652anjfqblifjam5q9s6k>
>>>>
<https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_events_c8tqca652anjfqblifjam5q9s6k&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=n2yNmrgzxmalXYD5jMGcOalL_vvGx4FsUxv3DSvnIOw&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_events_c8tqca652anjfqblifjam5q9s6k&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=n2yNmrgzxmalXYD5jMGcOalL_vvGx4FsUxv3DSvnIOw&e=>>
>>>> .
>>>>
>>>> (just this once, I've planned it on a Thursday rather than a
>>>> Wednesday due to an agenda conflict)
>>>>
>>>>
>>>> Topics I have in mind include:
>>>>
>>>> * (very early) outlook to EasyBuild v3.0: default config
>>>> changes, (minor) backwards-incompatible changes
>>>>
>>>> * deprecating toolchains: what & how
>>>> * see also
>>>>
https://github.com/hpcugent/easybuild/wiki/Deprecated-toolchains
<https://github.com/hpcugent/easybuild/wiki/Deprecated-toolchains>
>>>>
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild_wiki_Deprecated-2Dtoolchains&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=gSnvv8s_znJmFMPSBIPgAAsZAc8mMg6hAWiO03QUWck&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild_wiki_Deprecated-2Dtoolchains&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=gSnvv8s_znJmFMPSBIPgAAsZAc8mMg6hAWiO03QUWck&e=>>
>>>>
>>>> * update on support for RPATH (WIP)
>>>> *
>>>>
https://github.com/hpcugent/easybuild-framework/compare/develop...boegel:rpath?expand=1
<https://github.com/hpcugent/easybuild-framework/compare/develop...boegel:rpath?expand=1>
>>>>
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild-2Dframework_compare_develop...boegel-3Arpath-3Fexpand-3D1&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=6II9MfeFGUY-xpwQ-GUiv-n-m2x4p8oKd04yCGhzTr0&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild-2Dframework_compare_develop...boegel-3Arpath-3Fexpand-3D1&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=6II9MfeFGUY-xpwQ-GUiv-n-m2x4p8oKd04yCGhzTr0&e=>>
>>>>
>>>> * Q&A
>>>>
>>>> Suggestions for additional topics are welcome.
>>>>
>>>> Please let me know if you're planning to attend this conf
call.
>>>>
>>>> More information about the EasyBuild conf calls is
available at
>>>> https://github.com/hpcugent/easybuild/wiki/Conference-calls
<https://github.com/hpcugent/easybuild/wiki/Conference-calls>
>>>>
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild_wiki_Conference-2Dcalls&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=nABxLrDAlozucg_Duk9GoWHhALQ-rgs7w2oUqJPu7K0&e=
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpcugent_easybuild_wiki_Conference-2Dcalls&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=nABxLrDAlozucg_Duk9GoWHhALQ-rgs7w2oUqJPu7K0&e=>>
>>>> .
>>>>
>>>>
>>>> regards,
>>>>
>>>> Kenneth
>>>
>>
>>
>>
>>
>> --
>> Pablo Escobar López
>> HPC systems engineer
>> sciCORE, University of Basel
>> SIB Swiss Institute of Bioinformatics
>> http://scicore.unibas.ch
>>
<https://urldefense.proofpoint.com/v2/url?u=http-3A__scicore.unibas.ch_&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=nIOTUVJ_iBoI31ySMfWLAJEoa3AdRIR3AZfBUeklBZw&e=
<https://urldefense.proofpoint.com/v2/url?u=http-3A__scicore.unibas.ch_&d=CwMGaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=QycjT3AMp8mFa5J-RDcs2kGxbKac7JJX5bpmHAd9efw&s=nIOTUVJ_iBoI31ySMfWLAJEoa3AdRIR3AZfBUeklBZw&e=>>
>
--
Pablo Escobar López
HPC systems engineer
sciCORE, University of Basel
SIB Swiss Institute of Bioinformatics
http://scicore.unibas.ch
