Hi Geert-Jan, On 26 Jun 2013, at 13:50, Geert Jan BEX wrote:
> Hi Kenneth, > > Thanks for the feedback. > > Meanwhile I got the run_cmd_qa problem solved. I needed an easyblock because > the configuration step prompts to agree with the license. I could have > sidestepped the issue by patching that out of the configuration script, but > that seems a bit unethical ;) The package is Qt, and it is a prerequisite > for UNIFY. There's already a pull request open for a Qt easyblock, see https://github.com/hpcugent/easybuild-easyblocks/pull/210 (and https://github.com/hpcugent/easybuild-easyconfigs/pull/294 for easyconfigs with the goolf and ictce compiler toolchains). It's unfortunate that you were also spending time on it, but maybe you can take that experience, take a look at the pull request and see if you have any suggestions or comments. We like to have at least two pairs of eyes going over PRs (the one who implemented it, and someone else), before we merge stuff in... regards, Kenneth > > Adding the basic stuff to toolchains seems like a good idea. > > Best regards, -gjb- > > > > > On Tue, Jun 25, 2013 at 6:56 PM, Kenneth Hoste <[email protected]> wrote: > Hi Geert-Jan, > > On 25 Jun 2013, at 16:51, Geert Jan BEX wrote: > > > Hi Easybuilders, > > > > A couple of questions: > > • I'm running into trouble using run_cmd_qa in a custom easyblock. > > The command's output is pretty huge, and after a while, the maximum > > hitCount is exceeded. As I understand the inner workings of run_cmd_qa, it > > keeps on reading bits of stdout/stderr from the command it is executing, > > and it tries to match with the questions, answering them when found. > > Failing that, it will check for matches in the no_qa list, and when finding > > one, it resets hitCount, incrementing it otherwise. > > So if I get this right, I've to provide sufficient no_qa elements so that > > the whole of the output of the command is "covered", i.e., making sure that > > there is a no_qa match very n lines? > > > Your view on it is close to correct, but not fully. > The hit count is only increased if the output remains the same and no > question could be matched against the end of the output. > > As soon as the output changes, the hitCount is reset (if not, it's a bug), > because progress was detected without needing to provide an answer. > So, if some operation is taking a while, run_cmd_qa assumes it's a question > (i.e., the command is waiting for input), and will try to match a question to > the output. If it can't, it will die assuming something is wrong. > > So, basically, yes, you'll have to try and come up with no_qa entries (which > can be regex patterns b.t.w.) that match any operation that is printed at the > end of the output for a while, but is not an actual question (e.g. a compile > or copy command). Up until now, we've never run into a procedure in which > this proves impractical (i.e. usually the no_qa list can be kept quite small). > > Can you share an example of what you're running into? > > Also, out of curiosity: what is the custom easyblock for? > > > > • There are a number of modules that are really basic, e.g., git, > > CMake... Perhaps it would make sense to define a very minimal toolchain > > (even one that simply uses the resident gcc) for these kind of modules? > > That would allow to really minimize dependencies. Just a question, maybe > > it is a dumb idea. > > That's one way, yes, but then you're depending on the system-provided tools > again, which is one of the things we want to avoid. > Encouraging to provide modules for all dependencies makes reproducibility > more likely, both across different systems/OSs, and on the same system with > one or multiple minor/major OS updates in between. > > Relying on whatever (version of) tools the system provides is too likely to > change, which may result into builds that suddenly break (we've all seen > that). > > That being said, more stuff should be part of compiler toolchains, yes. There > is a ticket open for exactly that. Because tools/libs like zlib, ncurses, > binutils (ld, ar, ...) are (very) commonly dependencies, they should be (and > will be) part of the compiler toolchains we define. See > https://github.com/hpcugent/easybuild-framework/issues/644 . Maybe also > tar/gzip should be included. > > > regards, > > Kenneth > > > > > -- > - Geert Jan BEX > - HPC analyst/consultant Hasselt University & University of Leuven > - email: [email protected] > - homepage: http://alpha.uhasselt.be/~gjb/ > - PGP public key: http://alpha.uhasselt.be/~gjb/pgp.txt > ---------------------------------------------------------
