Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16155 )
Change subject: IMPALA-9887: Add support for sharding end-to-end tests ...................................................................... IMPALA-9887: Add support for sharding end-to-end tests ASAN maintains stacks for each allocation and free of memory. Impala sometimes allocates/frees memory from codegen'd code, so this means that the number of distinct stacks is unbounded. ASAN is storing these stacks in a hash table with a fixed number of buckets (one million). As the stacks accumulate, allocations and frees get slower and slower, because the lookup in this hashtable gets slower. This causes test execution time to degrade over time. Since backend tests and custom cluster tests don't have long running daemons, only the end to end tests are affected. This adds support for breaking end-to-end test execution into shards, restarting Impala between each shard. This uses the preexisting shard_tests pytest functionality introduced for the docker-based tests in IMPALA-6070. The number of shards is configurable via the EE_TEST_SHARDS environment variable. By default, EE_TEST_SHARDS=1 and no sharding is used. Without sharding, an ASAN core job takes about 16-17 hours. With 6 shards, it takes about 9 hours. It is recommended to always use sharding with ASAN. Testing: - Ran core job - Ran ASAN with EE_TEST_SHARDS=6 Change-Id: I0bdbd79940df2bc7b951efdf0f044e6b40a3fda9 Reviewed-on: http://gerrit.cloudera.org:8080/16155 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M bin/run-all-tests.sh M tests/run-tests.py 2 files changed, 64 insertions(+), 10 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0bdbd79940df2bc7b951efdf0f044e6b40a3fda9 Gerrit-Change-Number: 16155 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Laszlo Gaal <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
