This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit ac63e19e0d3c797b08dcf80053fc8e3259d8472d
Author: Joe McDonnell <joemcdonn...@cloudera.com>
AuthorDate: Wed Aug 5 14:17:54 2020 -0700
IMPALA-10043: Keep more logs when using EE_TEST_SHARDS
IMPALA-9887 introduced the EE_TEST_SHARDS setting, which splits
the end-to-end test into shards and restarts Impala in between.
In order to keep the logs from all the shards, the value for
max_log_files needs to be increased so that none get aged out.
This multiplies IMPALA_MAX_LOG_FILES by the number of shards
using EE_TEST_SHARDS.
Testing:
- Ran a test with EE_TEST_SHARDS=6 and verified that the
logs are preserved.
Change-Id: Ie011b892cd2eb1a528012ec5600e72e44f281a88
Reviewed-on: http://gerrit.cloudera.org:8080/16297
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Reviewed-by: Laszlo Gaal <laszlo.g...@cloudera.com>
---
bin/run-all-tests.sh | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/bin/run-all-tests.sh b/bin/run-all-tests.sh
index 5287861..74f65a9 100755
--- a/bin/run-all-tests.sh
+++ b/bin/run-all-tests.sh
@@ -254,6 +254,10 @@ do
# Some test frameworks (e.g. the docker-based tests) use this.
run_ee_tests
else
+ # Increase the maximum number of log files so that the logs from the
shards
+ # don't get aged out. Multiply the default number by the number of
shards.
+ IMPALA_MAX_LOG_FILES_SAVE="${IMPALA_MAX_LOG_FILES:-10}"
+ export IMPALA_MAX_LOG_FILES="$((${EE_TEST_SHARDS} *
${IMPALA_MAX_LOG_FILES_SAVE}))"
# When the EE tests are sharded, it runs 1/Nth of the tests at a time,
restarting
# Impala between the shards. There are two benefits:
# 1. It isolates errors so that if Impala crashes, the next shards will
still run
@@ -268,6 +272,7 @@ do
run_ee_tests "--shard_tests=$shard_idx/${EE_TEST_SHARDS}"
start_impala_cluster
done
+ export IMPALA_MAX_LOG_FILES="${IMPALA_MAX_LOG_FILES_SAVE}"
fi
fi
