Hi Bart, On 01 Oct 2013, at 22:48, Fotis Georgatos wrote:
> Hi Bart, > > On Oct 1, 2013, at 10:31 PM, Bart Verleye wrote: >> I have installed a new toolchain, gmpolf-3.0.4. We would *love* to see a pull request being created for this, i.e. the new toolchain definition to the easybuild-framework repo, the easyconfig files for the actual toolchain version to the easybuild-easyconfigs repo. Are you up for that? The version number looks a bit odd to me, but I can't really tell since I have no view on the versions of the toolchain elements. And, I have to admit, the way we version toolchains is very ad hoc/unclear/undocumented, so I'm not blaming you. ;-) >> Now I want to install Paraview, which depends on quite a few other packages. > > To save a few keystrokes, you can find pre-calculated templates for many > of the common building blocks over here (shameless rip from pkgsrc's > Makefiles): > https://github.com/fgeorgatos/easybuild.experimental/tree/master/contrib/pkgsrc/20130506 > (sorry to the rest for being repetitive, but you know how convenient this can > be...) > Paraview is new, i.e. we don't have support for that yet. So, if you end up getting it to work, again, please submit a pull request to include that support in the next EasyBuild release, to avoid that others lose time on the same problems you're solving (which is what EasyBuild is all about, cooperation!). >> During the installation, I came up with following comments: >> >> - --try-toolchain is not recursive. Is that 'not yet', or is there a reason >> behind that? > > "Not yet", there is an open issue somewhere about it, AFAIK. It's "not yet", indeed. I agree this is a very desirable feature, but there's also a 'but'. First, let me try and explain why it's not there yet... Although you would expect this to be a feature that can be easily implemented, it in fact requires quite a bit of work. I started working on it once, and ended up spending a couple of hours on it, without a decent result (i.e., a huge ugly patch, and it didn't even work yet). It seems that the current codebase doesn't easily allow to support this, but it has been a while since I looked at it, so maybe I should just look into this again... We have an open issue for this, see https://github.com/hpcugent/easybuild-framework/issues/450. The 'but' is that --try-X is best effort, and is pretty likely to fail (depending on how different the toolchain you're 'trying' is from the current one). The implementation of this is pretty messy, or to use someone else's words "a steaming pile of crap" (he's right). So, I fully agree with you that this would be very useful, but it's less easy than it may seem at first. Anyway, I'll try and revisit this soon. Do leave a comment in issue #450 to indicate you really want to see this work. >> - --try-toolchain produces a eb file, but does not copy it to the lib >> directory at the end. Perhaps that could be added? > > It's a bugture (contraption of bug-feature): manual work is expected. > In principle that requires heavy editing and is considered more of a template. Are you expecting the easyconfig file to be added to the currently installed easybuild-easyconfigs package? That's a bit weird, since you're then modifying an installed part of EB, which has a particular version (which then no longer makes sense as reference point, by consequence). Actually, what we'd want you to do, is submit a pull request for the easyconfig(s) you obtained this way (if the build worked for you). That way, it will be included in the next version of EasyBuild, and other people can use them without needing to resort to --try-X. As Fotis already mentioned, the automagically tweaked easyconfig file(s) still need to be reviewed by a human (even if it worked) before they can be merged in... We're thinking about allowing people to configure EasyBuild such that pull requests are generated automagically whenever a build is successful and if the easyconfig file that was used is not available yet in our repos, but that's also a bit scary... ;-) > >> - in one of my attemps, at the end of an installation, this error came up: >> (should only have 'files' and 'dirs' keys, values should be lists (at least >> one non-empty)). >> This could be checked before compilation as part of the eb-file sanity check. > > IMHO, the error-level should then be non-zero; I believe then the right thing > will happen, no? Bart is right here, we could/should capture a problem like that a lot earlier, instead of performing the whole build procedure first, and then complain that the sanity check spec is not what we want it to be... Point taken: https://github.com/hpcugent/easybuild-framework/issues/703 > >> During the installation of Qt, some "Project Messages" pop up, and the >> waiting time during the configuration is sometimes large. As a result, eb >> stops and prints: > [...] >> However, this message is not an error, and if I configure myself, without >> eb, the whole process runs through. How can I avoid eb from stopping on >> these messages? > > Either the output has a keyword that makes EasyBuild tick (and requires > tuning), > or the error-level is incorrect or, there is a bug (eh, or all of the above > :). > Which case are we talking about here? The configure script for Qt is an interactive script (eek!), i.e. it asks questions and expects answers, see https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/q/qt.py . EasyBuild easily confuses messages that show up as the last line of output for a while for questions, and then fails when it can't figure out an answer to that 'question'. Hence, EB needs to be told which 'long-hanging' messages are *not* questions, via a list of regex patterns (see the no_qa part in the Qt easyblock). It seems that the current list of no_qa patterns in insufficient on your end, for some reason (maybe slower systems, whatever). Can you try and adjust the Qt easyblock such that these project messages are ignored by extending the no_qa list? If you get that to work, please open a PR to get that fix included. If not, let us know, and we'll help you out. The waiting time during configuration is large for Qt, I remember that from our installations. E.g., in our production build, the configure step took about 7 minutes. Are you saying the configuration is a lot faster if you run it manually (outside of EasyBuild)? Because that would surprise me, and would definitely point to a bug/problem. regards, Kenneth PS: It seems like you guys are really picking up on EasyBuild, great! If you think it's useful to have a conf call of some sort to quickly discuss a couple of things that are not entirely clear to you, please shout, it could be a huge time saver... I'm sure we can find a time that works well on both ends of the wire...
