I think it's important to tease apart what why we're trying to mark tests. Generally, nearly all tests should run on all runners. However, there are some exceptions, namely.
1) Some runners don't support all features (especially at the start). 2) Some tests are incompatible with distributed runners (e.g. rely on in-process IO fakes) @RunnableOnService has also been used to mark tests that *should* be run on the service, as it is prohibitively expensive to run all tests on all runners. We should also have the notion of a comprehensive suite of tests a runner should pass to support the full model. This would exclude many tests that are of unmodified composite transforms (that hopefully could run on any runner, but the incremental benefit would be small.) On Wed, Nov 9, 2016 at 12:20 PM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi Mark, > > Generally speaking, I agree. > > As RunnableOnService extends NeedsRunner, @TestsWithRunner or @RunOnRunner > sound clearer. > > Regards > JB > > > On 11/09/2016 09:00 PM, Mark Liu wrote: >> >> Hi all, >> >> I'm working on building RunnableOnService in Python SDK. After having >> discussions with folks, "RunnableOnService" looks like not a very >> intuitive >> name for those unit tests that require runners and build lightweight >> pipelines to test specific components. Especially, they don't have to run >> on a service. >> >> So I want to raise this idea to the community and see if anyone have >> similar thoughts. Maybe we can come up with a name this is tight to >> runner. >> Currently, I have two names in my head: >> >> - TestsWithRunners >> - RunnerExecutable >> >> Any thoughts? >> >> Thanks, >> Mark >> > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com
