Re: Policy for Python ValidatesRunner vs IT tests?

2018-08-14 Thread Robin Qiu
Hi Pablo, Beam Java has unit tests, ValidatesRunner tests, and Integration (E2E) test. If I understand correctly, unit tests test the functionality of a component and usually don't require a runner (is this true?); ValidatesRunner tests (marked with @ValidatesRunner annotation) require runners;

Re: Policy for Python ValidatesRunner vs IT tests?

2018-08-14 Thread Pablo Estrada
Ah this is very helpful. I think this is missing in the testing guide. I'll make a short PR to mention it. Best -P. On Tue, Aug 14, 2018 at 3:37 PM Anton Kedin wrote: > IT tests exist in java, similar to unit tests and not marked in a special > way, except they're called *IT.java instead of

Re: Policy for Python ValidatesRunner vs IT tests?

2018-08-14 Thread Anton Kedin
IT tests exist in java, similar to unit tests and not marked in a special way, except they're called *IT.java instead of *Test.java. They're run from corresponding tasks: -

Re: Policy for Python ValidatesRunner vs IT tests?

2018-08-14 Thread Alan Myrvold
The ValidatesContainer test builds the python portable worker container using docker, pushes it, then runs a dataflow pipeline using the container with the flag --worker_harness_container_image. There is only one test that does this, and it is also an 'IT' test. On Tue, Aug 14, 2018 at 3:27 PM

Policy for Python ValidatesRunner vs IT tests?

2018-08-14 Thread Pablo Estrada
Hello, In Python, we tag some test methods with @attr('ValidatesRunner') and @attr('IT'), which marks them to be run as pipeline tests. If I understand correctly: - ValidatesRunner tests are more like a component tests[1] as explained in Beam docs - IT tests are more like a E2E test[2] as