Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Yuval Kogman
*cough* if it were a push parser *cough* -- Yuval Kogman <[EMAIL PROTECTED]> http://nothingmuch.woobling.org 0xEBD27418

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Nicholas Clark
On Mon, Feb 05, 2007 at 03:24:37AM +, Andy Armstrong wrote: > On 5 Feb 2007, at 01:08, Eric Wilhelm wrote: > >I was just thinking that since my tests are nicely orthogonal and yet > >there's very little cpu utilization during testing, that I could > >probably get them done about 20x faster if t

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Adam Kennedy
Eric Wilhelm wrote: Hi all, I was just thinking that since my tests are nicely orthogonal and yet there's very little cpu utilization during testing, that I could probably get them done about 20x faster if they were parallelized. Has anyone looked at doing this with TAPx::Harness or otherwis

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Andy Armstrong
On 5 Feb 2007, at 11:41, Adam Kennedy wrote: Short of running each test in some sort of transactional file system (do such things exist? Are they common?) or cloning the build directory before each test, the only solution I can imagine is some sort of thing in the (Makefile|Build).PL that ex

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Andy Armstrong
On 5 Feb 2007, at 11:24, Nicholas Clark wrote: Something I discovered was that processing TAP is slow. Up to about 7 CPUs, parallelising tests feeding TAP into the master "thread" was a win. Beyond that, the process processing the TAP was the bottleneck, and adding more CPUs (and more parall

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Eric Wilhelm
# from Nicholas Clark # on Monday 05 February 2007 03:24 am: >which was sufficient to give a lot of flexibility in how tests were > run (and discover that the core is very unoriginal in its naming of > temporary files) # from Adam Kennedy # on Monday 05 February 2007 03:41 am: >Each test is stil

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Eric Wilhelm
# from Andy Armstrong # on Monday 05 February 2007 04:20 am: >I can't remember whether it's mod_perl or CGI.pm that launches Apache >   a few times during testing but I imagine there might be a problem > with multiple Apaches trying to bind to the same port in that case > too. Ditto anything that

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Andy Armstrong
On 5 Feb 2007, at 18:20, Eric Wilhelm wrote: Unfortunately that disqualifies a whole category of I/O bound tests that would otherwise benefit the most. Well, it disqualifies them from running on the same box/chroot/ whatever. Consider a scheme that uses FAM+rsync to keep another box in lock-s

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Eric Wilhelm
# from Andy Armstrong # on Monday 05 February 2007 10:27 am: >I guess there are already frameworks   >available that'd help with managing that kind of distributed setup?   >The most similar thing I have any experience of is distcc[1]. > >As a matter of interest I wonder if anyone's worked out whet

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Andy Armstrong
On 5 Feb 2007, at 18:47, Eric Wilhelm wrote: From my reading, distcc only ships a single pre-processed file across the wire and gets a binary back from the cc on the other end. That's a very simple and easy to distribute scheme, and I think the claim is that it will work in any build as a dro

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Chris Dolan
On Feb 5, 2007, at 5:41 AM, Adam Kennedy wrote: One thing that strikes me as a problem is transitory resources used by tests. I know I have a number of modules that will create (deleting if they already exist) various files during their run (cleaning up at the end). Each test is still ind

[PATCH] dereference bug in TAPx::Harness exec

2007-02-05 Thread Eric Wilhelm
It seems that exec accumulates arguments, thus running the first test over and over with more and more arguments. http://scratchcomputing.com/tmp/tapx_exec.patch The runtests utility was also passing a string from the '-e' switch. The split(/ /, ...) is a little naive, but demonstrates the i

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Eric Wilhelm
# from Andy Armstrong # on Monday 05 February 2007 11:22 am: >To run tests locally you need to > >* launch the tests in // and gather their output >* display the test progress / results differently >* make it possible to attach some metadata to tests >    that describes such things as which tests

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Chris Dolan
On Feb 5, 2007, at 5:41 AM, Adam Kennedy wrote: Short of running each test in some sort of transactional file system (do such things exist? Are they common?) or cloning the build directory before each test... Here's a terrible, non-portable idea: * create a new, empty temp directory * us

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Andy Armstrong
On 5 Feb 2007, at 19:48, Chris Dolan wrote: A solution might be to use File::Temp::tempdir(CLEANUP => 1) in that case. You can build the whole tree of files per-test and wipe clean upon exit. It seems to me that writing temp files to the build dir should be discouraged. Absolutely. The p

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Andy Armstrong
On 5 Feb 2007, at 19:58, Eric Wilhelm wrote: * launch the tests in // and gather their output * display the test progress / results differently * make it possible to attach some metadata to tests that describes such things as which tests can safely be run concurrently with which other tes

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Eric Wilhelm
# from Nicholas Clark # on Monday 05 February 2007 03:24 am: >Important differences were > >1: you only print out the entire test results as an atomic line when > it has finished >2: You have no idea which test is going to finish next >3: You can't print running progress on tests Ah, there's the

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Andy Armstrong
On 5 Feb 2007, at 20:17, Eric Wilhelm wrote: o TAPx::Harness::test_loop() method as overridable inner part of aggregate_tests(), with some rejuggling of $periods calculation o TAPx::Harness::_do_test(\%args) method as overridable inner part of _runtest(), allowing the subclass to not

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Eric Wilhelm
# from Andy Armstrong # on Monday 05 February 2007 12:23 pm: >We could just ship the raw TAP   >back along with the test summaries I guess. Is that what the 'tap' field is for, or is that intended for input? Maybe the definition of freeze/thaw should recycle that field? --Eric -- You can't wh

Re: Parallelizing TAPx::Harness?

2007-02-05 Thread Andy Armstrong
On 5 Feb 2007, at 21:35, Eric Wilhelm wrote: Is that what the 'tap' field is for, or is that intended for input? Maybe the definition of freeze/thaw should recycle that field? That's how it's documented but TAPx::Parser doesn't currently store the TAP. It does allow the TAP to be spooled to a