Hi Kenneth, Thanks for the feedback.
Meanwhile I got the run_cmd_qa problem solved. I needed an easyblock because the configuration step prompts to agree with the license. I could have sidestepped the issue by patching that out of the configuration script, but that seems a bit unethical ;) The package is Qt, and it is a prerequisite for UNIFY. Adding the basic stuff to toolchains seems like a good idea. Best regards, -gjb- On Tue, Jun 25, 2013 at 6:56 PM, Kenneth Hoste <[email protected]>wrote: > Hi Geert-Jan, > > On 25 Jun 2013, at 16:51, Geert Jan BEX wrote: > > > Hi Easybuilders, > > > > A couple of questions: > > • I'm running into trouble using run_cmd_qa in a custom easyblock. > The command's output is pretty huge, and after a while, the maximum > hitCount is exceeded. As I understand the inner workings of run_cmd_qa, it > keeps on reading bits of stdout/stderr from the command it is executing, > and it tries to match with the questions, answering them when found. > Failing that, it will check for matches in the no_qa list, and when > finding one, it resets hitCount, incrementing it otherwise. > > So if I get this right, I've to provide sufficient no_qa elements so > that the whole of the output of the command is "covered", i.e., making sure > that there is a no_qa match very n lines? > > > Your view on it is close to correct, but not fully. > The hit count is only increased if the output remains the same and no > question could be matched against the end of the output. > > As soon as the output changes, the hitCount is reset (if not, it's a bug), > because progress was detected without needing to provide an answer. > So, if some operation is taking a while, run_cmd_qa assumes it's a > question (i.e., the command is waiting for input), and will try to match a > question to the output. If it can't, it will die assuming something is > wrong. > > So, basically, yes, you'll have to try and come up with no_qa entries > (which can be regex patterns b.t.w.) that match any operation that is > printed at the end of the output for a while, but is not an actual question > (e.g. a compile or copy command). Up until now, we've never run into a > procedure in which this proves impractical (i.e. usually the no_qa list can > be kept quite small). > > Can you share an example of what you're running into? > > Also, out of curiosity: what is the custom easyblock for? > > > > • There are a number of modules that are really basic, e.g., git, > CMake... Perhaps it would make sense to define a very minimal toolchain > (even one that simply uses the resident gcc) for these kind of modules? > That would allow to really minimize dependencies. Just a question, maybe > it is a dumb idea. > > That's one way, yes, but then you're depending on the system-provided > tools again, which is one of the things we want to avoid. > Encouraging to provide modules for all dependencies makes reproducibility > more likely, both across different systems/OSs, and on the same system with > one or multiple minor/major OS updates in between. > > Relying on whatever (version of) tools the system provides is too likely > to change, which may result into builds that suddenly break (we've all seen > that). > > That being said, more stuff should be part of compiler toolchains, yes. > There is a ticket open for exactly that. Because tools/libs like zlib, > ncurses, binutils (ld, ar, ...) are (very) commonly dependencies, they > should be (and will be) part of the compiler toolchains we define. See > https://github.com/hpcugent/easybuild-framework/issues/644 . Maybe also > tar/gzip should be included. > > > regards, > > Kenneth > > -- - Geert Jan BEX - HPC analyst/consultant Hasselt University & University of Leuven - email: [email protected] - homepage: http://alpha.uhasselt.be/~gjb/ - PGP public key: http://alpha.uhasselt.be/~gjb/pgp.txt ---------------------------------------------------------
