Stephan Ewen commented on FLINK-8745:

+1 for moving the Hadoop / Scala Matrix to Cron builds and

I think that the flip-6 profiles can be dropped for master after the 1.5 
release as well (1.5 still retains the legacy model, AFAIK the plan is to have 
only the Flip-6 mode for 1.6+)

> Reduce travis usage
> -------------------
>                 Key: FLINK-8745
>                 URL: https://issues.apache.org/jira/browse/FLINK-8745
>             Project: Flink
>          Issue Type: Improvement
>          Components: Build System, Travis
>    Affects Versions: 1.4.0, 1.5.0
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>            Priority: Critical
> We've been notified by INFRA that our travis usage is exceedingly high.
> There are various things we could look into short- and long term:
> h2. Short-term
> h3. Reduce number of jobs
> We currently run 12 job for each pr/push.
> The first 10 jobs belong to 2 groups, with each group representing one test 
> run of Flink against a specific hadoop version.
> Given that the majority of changes made to Flink do not impact our 
> compatibility with hadoop we could drop one of the groups and instead rely on 
> daily cron jobs. This alone would cut our travis usage by 40%.
> Once the migration to flip6 is done we can drop the remaining 2 jobs, 
> increasing the reduction to 60%.
> h3. Reduce number of builds
> Travis is run for every PR, regardless of what change was made, even if it 
> was something trivial as removing a trailing space in a documentation file. 
> From time to time it also happens that new commits are pushed in a PR solely 
> to trigger a new build to get that perfect green build.
> Instead we could look into manually triggering travis for pull requests, that 
> is with a bot.
> h2. Long-term
> h3. Incremental builds
> Making the build dependent on the changes made has been brought up a few 
> times now. This would in particular benefit cases where connectors/libraries 
> are modified as they generally have few dependents. We would still have to 
> run everything though if changes are made to the core modules.
> h3. Repository split
> The most painful of them all, but in my opinion also the most promising. With 
> separate repositories for the core flink modules (flink-runtime etc), 
> flink-connectors and flink-libraries would cut downright skip the compilation 
> for a large number of modules.

This message was sent by Atlassian JIRA

Reply via email to