Robert, it seems as though setting up the Flink/Spark/Gearpump cluster locally for testing is pretty fast already and typically all the ROS tests run within 5-10 mins. The two outliers are Google Cloud Dataflow which takes over an hour, and also the amount of time we have to wait before there is a free executor available in Jenkins in the afternoon/evening PST time.
On Thu, Oct 20, 2016 at 9:26 PM, Jean-Baptiste Onofré <[email protected]> wrote: > Hi, > > indeed, let me ping INFRA to try to get update. > > Regards > JB > > > On 10/20/2016 09:38 PM, Davor Bonaci wrote: > >> I'd be hugely in favor, however, this is not what the Apache Jenkins >> supports right now, AFAIK. I've asked Infra about this awhile ago, but >> nothing has moved yet. There was also a Jira issue about it, INFRA-11610. >> >> On Thu, Oct 20, 2016 at 12:24 PM, Amit Sela <[email protected]> wrote: >> >> Hi all, >>> >>> I'd like to discuss options to execute ROS tests (per runner) more >>> efficiently, and explore the option of running them on PreCommit, as >>> opposed to PostCommit as they run today. >>> >>> The SDK provides a set of tests called "RunnableOnService" (aka ROS) that >>> can be applied to a runner and validate it (correctly) supports SDK >>> features. >>> It's 300+ tests in total (batch + streaming) and it clearly takes time, >>> and >>> that is why it runs on PostCommit. >>> I think we should look for a configuration where this is executed >>> more efficiently and if possible run on PreCommit since runners are >>> encouraged to rely on those tests and it's better to know of breaking >>> changes before hand. >>> >>> This came up somewhere in this >>> <https://github.com/apache/incubator-beam/pull/1055> conversation, and >>> the >>> highlights are basically: >>> >>> Kenneth Knowles suggested we might parallelize sub-builds in the >>> following >>> manner: >>> >>> 1. Run unit tests. >>> 2. (sub tasks) Run ROS tests for each runner in parallel, skipping >>> unit >>> tests. >>> >>> I was wondering if we could setup Jenkins to run ROS per runner only of >>> there was a code change for that runner - of course SDK changes will >>> probably have to run ROS for all runners, but that might still be an >>> optimization. >>> >>> I think one of Beam's best sell-points is it's extensive testing >>> framework, >>> and the fact that runners can be validated across capabilities, but it >>> would be best to know of runner-braking changes before merging to master. >>> >>> Thoughts ? >>> >>> Thanks, >>> Amit >>> >>> >> > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
