[
https://issues.apache.org/jira/browse/FLINK-8819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-8819:
Description:
This issue is for tracking efforts to rework our Travis scripts to use
[stages|https://docs.travis-ci.com/user/build-stages/].
This feature allows us to define a sequence of jobs that are run one after
another. This implies that we can define dependencies between jobs, in contrast
to our existing jobs that have to be self-contained.
As an example, we could have a compile stage, and a test stage with multiple
jobs.
The main benefit here is that we no longer have to compile modules multiple
times, which would reduce our build times.
The major issue here however is that there is no _proper_ support for passing
build-artifacts from one stage to the next. According to this
[issue|https://github.com/travis-ci/beta-features/issues/28] it is on their
to-do-list however.
In the mean-time we could manually transfer the artifacts between stages by
either using the Travis cache or some other external storage. The cache
solution would work by setting up a cached directory (just like the mvn cache)
and creating build-scope directories within containing the artifacts (I have a
prototype that works like this).
The major concern here is that of cleaning up the cache/storage.
We can clean things up if
* our script fails
* the last stage succeeds.
We can *not* clean things up if
* the build is canceled
* travis fails the build due to a timeout or similar
as apparently there is [no way to run a script at the end of a
build|https://github.com/travis-ci/travis-ci/issues/4221].
Thus we would either have to periodically clear the cache, or encode more
information into the cached files that would allow _other_ builds to clean up
stale date. (For example the build number or date).
was:
This issue is for tracking efforts to rework our Travis scripts to use
[stages|https://docs.travis-ci.com/user/build-stages/].
This feature allows us to define a sequence of jobs that are run one after
another. This implies that we can define dependencies between jobs, in contrast
to our existing jobs that have to be self-contained.
As an example, we could have a compile stage, and a test stage with multiple
jobs.
The main benefit here is that we no longer have to compile modules multiple
times, which would reduce our build times.
The major issue here however is that there is no _proper_ support for passing
build-artifacts from one stage to the next. According to this
[issue|https://github.com/travis-ci/beta-features/issues/28] it is on their
to-do-list however.
In the mean-time we could manually transfer the artifacts between stages by
either using the Travis cache or some other external storage.
The major concern here is that of cleaning up the cache/storage.
We can clean things up if
* our script fails
* the last stage succeeds.
We can *not* clean things up if
* the build is canceled
* travis fails the build due to a timeout or similar
as apparently there is [no way to run a script at the end of a
build|https://github.com/travis-ci/travis-ci/issues/4221].
> Rework travis script to use build stages
>
>
> Key: FLINK-8819
> URL: https://issues.apache.org/jira/browse/FLINK-8819
> Project: Flink
> Issue Type: Sub-task
> Components: Build System, Travis
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Trivial
>
> This issue is for tracking efforts to rework our Travis scripts to use
> [stages|https://docs.travis-ci.com/user/build-stages/].
> This feature allows us to define a sequence of jobs that are run one after
> another. This implies that we can define dependencies between jobs, in
> contrast to our existing jobs that have to be self-contained.
> As an example, we could have a compile stage, and a test stage with multiple
> jobs.
> The main benefit here is that we no longer have to compile modules multiple
> times, which would reduce our build times.
> The major issue here however is that there is no _proper_ support for passing
> build-artifacts from one stage to the next. According to this
> [issue|https://github.com/travis-ci/beta-features/issues/28] it is on their
> to-do-list however.
> In the mean-time we could manually transfer the artifacts between stages by
> either using the Travis cache or some other external storage. The cache
> solution would work by setting up a cached directory (just like the mvn
> cache) and creating build-scope directories within containing the artifacts
> (I have a prototype that works like this).
> The major concern here is that of cleaning up the cache/storage.
> We can clean things up if
> * our script fails
> * the last stage succeeds.
> We can *not* clean things up if
