[
https://issues.apache.org/jira/browse/FLINK-7047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073999#comment-16073999
]
ASF GitHub Bot commented on FLINK-7047:
---------------------------------------
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/4233
WIth the existing surefire configuration we also cannot do this in 2 steps,
one for compilation and one for tests. It is not possible to only run tests
with any maven invocation that includes a life-cycle (i.e verify), which means
that you have to recompile some modules twice which isn't an option because
scala compilation is so utterly horrible.
You _could_ theoretically cd into each project after compilation and
directly invoke the surefire plugin, but this doesn't work as we have 2
executions (1 for unit tests, 1 for IT cases), and when invoking a plugin
directly you can at most run 1 execution and it has to be named either
"default-test" or "default-cli". This would be possible in maven 3.3.1, but we
can't use that without messing with the shading, We could use maven 3.3.X if we
split the repo and add a separate one for flink-dist, but that's not gonna
happen anytime soon.
We could directly invoke the surefire plugin if we merged our surefire
executions, but then we're still left with the problem of having to keep the
travis scripts in sync with the directory structure of the project.
> Reorganize build profiles
> -------------------------
>
> Key: FLINK-7047
> URL: https://issues.apache.org/jira/browse/FLINK-7047
> Project: Flink
> Issue Type: Improvement
> Components: Tests, Travis
> Affects Versions: 1.4.0
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Fix For: 1.4.0
>
>
> With the current build times once again hitting the timeout it is time to
> revisit our approach.
> The current approach of splitting all tests by name, while easy to maintain
> or extend, has the big disadvantage that it's fairly binary in regards to the
> timeout: either we're below the timeout and all builds pass, or we're above
> and the entire merging process stalls. Furthermore, it requires all modules
> to be compiled.
> I propose a different approach by which we bundle several modules, only
> execute the tests of these modules and skip the compilation of some modules
> that are not required for these tests.
> 5 groups are my current suggestion, which will result in 10 build profiles
> total.
> The groups are:
> # *core* - core flink modules like core,runtime,streaming-java,metrics,rocksdb
> # *libraries* - flink-libraries and flink-storm
> # *connectors* - flink-connectors, flink-connector-wikiedits,
> flink-tweet-inputformat
> # *tests* - flink-tests
> # *misc* - flink-yarn, fink-yarn-tests, flink-mesos, flink-examples,
> flink-dist
> To not increase the total number of profiles to ridiculous numbers i also
> propose to only test against 2 combinations of jdk+hadoop+scala:
> # oraclejdk8 + hadoop 2.8.0 + scala 2.11
> # openjdk7 + hadoop 2.4.1 + scala 2.10
> My current estimate is that this will cause profiles to take at most 40
> minutes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
