Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13290 )
Change subject: Allow running backend tests sharded and in parallel ...................................................................... Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/13290/2/be/CMakeLists.txt File be/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/13290/2/be/CMakeLists.txt@674 PS2, Line 674: "GTEST_OUTPUT=xml:$ENV{IMPALA_BE_TEST_LOGS_DIR}/${TARGET} > The unified test script sets the -gtest_output commandline arg. I think tha Ack http://gerrit.cloudera.org:8080/#/c/13290/2/be/CMakeLists.txt@675 PS2, Line 675: RUN_SERIAL > I haven't use this before. If we have -j 8 on our ctest invocation, and exp right. That's a difference from the way that one of the old "shard and parallelize tests" patches worked. If I recall correctly, that older patch changed each single test to run itself in multiple threads, but didn't let them overlap with each other. In practice, doing it the way this patch does has benefits for the run time in that, in many cases, one shard of a suite is much longer-running than the rest (because one case is an outlier) and it's good to start running other tests at the same time. Our experience in Kudu is that the vast majority of tests don't need RUN_SERIAL and only a few "misbehaved" ones do. That said, we put a fair amount of effort in on the Kudu side to make tests use ephemeral ports and distinct 127.x.y.z loopback addresses based on the PID. So, I took the route of making Impala tests RUN_SERIAL by default and we can flip them over incrementally after checking which tests end up binding ports, etc. -- To view, visit http://gerrit.cloudera.org:8080/13290 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0d44475e3f5a45c806f00d5003eeadf59683b009 Gerrit-Change-Number: 13290 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Comment-Date: Fri, 17 May 2019 17:02:06 +0000 Gerrit-HasComments: Yes
