This would be a great contribution if anyone wants to give it a try. On Thu, Jun 22, 2017 at 9:23 PM, Jean-Baptiste Onofré <[email protected]> wrote:
> Hi Eugene > > I like the idea ! > > Regards > JB > > > On 06/23/2017 12:27 AM, Eugene Kirpichov wrote: > >> Hi folks and especially runner developers, >> >> https://issues.apache.org/jira/browse/BEAM-2506 - quoting from there: >> >> Currently ValidatesRunner test suites run 1 pipeline per unit test. That's >> a lot of small pipelines, and consumes a lot of resources especially in >> case of a pretty heavyweight runner like Dataflow, so tests take a long >> time and can't be run in parallel due to quota issues, etc. >> >> Jason Kuster says he and Davor Bonaci discussed that we could execute >> multiple unit tests in a single TestPipeline. >> >> To further develop it: in case of Java, we could create a custom JUnit >> Runner http://junit.org/junit4/javadoc/4.12/org/junit/runner/Runner.html >> that would apply all the transforms and PAsserts in unit tests to a single >> instance of TestPipeline (per class, rather than per method), and run the >> whole thing at the end. PAssert captures the source location of its >> application, so we could still report which particular test failed. >> >> This obviously has fewer isolation between unit test methods, cause they >> effectively run in parallel instead of in sequence, so things like >> per-method setup and teardown will no longer be applicable. There'll >> probably be other issues. >> >> Anyway, this seems doable and high-impact. >> >> Just bringing this to the attention of the community - it seems worth >> discussing and perhaps someone will be interested in developing this idea >> further or implementing it. >> >> > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
