The tests were designed to be Responder platform centric. So, if one want to test Pirk on Spark, then they can run the corresponding test suite. The individual tests within each Responder platform scope test test the various cases (and corner cases) of Pirk.
One correction to what you wrote about test input - the Inputs for the tests (see the DistributedTestDriver initialize(fs) method), are created once per run of the driver - if multiple platforms are being tested, the inputs are only created one time. I am not at all opposed to refactoring the tests (they absolutely need a bit of cleanup) On Thu, Oct 6, 2016 at 6:39 PM, Darin Johnson <[email protected]> wrote: > Being pretty familiar with that section of the code base right now. I > think a few things could be done. Regardless I'm refactoring it for > submodules do it'll be easier to iterate these. The main place I'd start is > currently each platform test creates a new input, which means the querier > logic is repeated for the number of platforms tested - setting up the test > inputs once, running all platforms for that test and then tearing down the > inputs would be a start. > > > Maybe have all tests implement an interface like: > > Public interface BaseTest { > > > public void createInputs(...); > > > public boolean checkOutputs(List results); > > } > > > The a runner could do: > > BaseTest myTest=new myTest() > > myTest.createInputs(...) > > For(responder:responders) > > { > > results = responder.runDistributedTest(); > > If(!myTest.checkResults(results) fail(); > > } > > > If nothing else it'd make adding tests easier. > > > > On Oct 6, 2016 9:33 AM, "Walter Ray-Dulany" <[email protected]> wrote: > > > Tim Ellison noted in a recent email to the list (subject: [CANCEL] > [VOTE]: > > Apache Pirk 0.2.0-incubating Realse) the following: > > > > > The distributed tests took a long time (~35mins IIRC), is that normal? > > > > I feel like noting this in a separate thread, and discussing it here, is > > worthwhile. > > > > The distributed tests *do* take a long time. The reason isn't because any > > one test is slow; indeed, most of the individual tests take only a > handful > > of seconds. The slowness is because of the very large number of tests. > This > > vast array of tests, upon inspection, can be seen to be caused by a very > > thorough testing of the large number of different actions that can be > > performed by Pirk and platforms it supports. > > > > I think this is ok. As an example of why this thorough testing is > > necessary, just yesterday, while finishing up PIRK-45, I ran into an > error > > 15 minutes into the distributed tests. Without this thorough testing, > that > > code would have made it to a PR yesterday night, and (without the > > distributed tests) it is likely no one would have caught it (it was a > > subtle "Spark has an ancient package on its classpath pre-empting a newer > > version I'd included, and the old package has a bug" error). > > >
