Joe McDonnell has uploaded a new patch set (#3) to the change originally created by Todd Lipcon. ( http://gerrit.cloudera.org:8080/13290 )
Change subject: Allow running backend tests sharded and in parallel ...................................................................... Allow running backend tests sharded and in parallel This improves the CMake functions for adding a test, adding a NUM_SHARDS parameter and management of the RUN_SERIAL propery. See the newly added documentation for the ADD_BE_TEST() cmake function for details. This patch enables the new functionality for 'expr-test' which is one of the slowest backend tests. To test this, I enabled parallel runs of expr-test and tested that 'ctest -j 8 -R expr-test' runs the shards in parallel, and that different subsets of test cases run in each shard. This command executed in 230 secs vs 871secs serially (so should save about 10 minutes of test time) Along the way I also split up the longest-running test case in expr-test so that it would shard more easily. I also implemented the same functionality for unified tests and used it in a few of the longer-running ones which I could verify were single-threaded and shouldn't have any cross-test conflicts. Change-Id: I0d44475e3f5a45c806f00d5003eeadf59683b009 --- M be/CMakeLists.txt M be/src/catalog/CMakeLists.txt M be/src/codegen/CMakeLists.txt M be/src/common/CMakeLists.txt M be/src/exec/CMakeLists.txt M be/src/exec/parquet/CMakeLists.txt M be/src/experiments/CMakeLists.txt M be/src/exprs/CMakeLists.txt M be/src/exprs/expr-test.cc M be/src/rpc/CMakeLists.txt M be/src/runtime/CMakeLists.txt M be/src/runtime/bufferpool/CMakeLists.txt M be/src/runtime/io/CMakeLists.txt M be/src/scheduling/CMakeLists.txt M be/src/service/CMakeLists.txt M be/src/statestore/CMakeLists.txt M be/src/util/CMakeLists.txt M bin/run-backend-tests.sh 18 files changed, 261 insertions(+), 179 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/13290/3 -- To view, visit http://gerrit.cloudera.org:8080/13290 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0d44475e3f5a45c806f00d5003eeadf59683b009 Gerrit-Change-Number: 13290 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jim Apple <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]>
