Hello Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19324
to look at the new patch set (#8).
Change subject: IMPALA-11476: Test Ozone with erasure coding
......................................................................
IMPALA-11476: Test Ozone with erasure coding
Tweaks to test Ozone with erasure coding.
In slower test environments the Ozone client may end up excluding slow
nodes, causing some tests to fail because there are not enough available
nodes to perform writes (erasure coding 3-2 can reconstruct data during
reads after 2 replica failures, but requires all replicas to write). In
our test environment, we don't provision enough nodes to afford
exclusion, so effectively disable it by setting
ozone.client.exclude.nodes.expiry.time=1 (ms).
Test updates:
- test_exclusive_coordinator_plan: Ozone+EC blocks are 768MB, which is
larger than all tables in our test environment. Use tpch_parquet which
we rely on having 3 files (by loading from snapshot in this case).
- test_new_file_shorter: receives an EOFException when seeking with EC
- test_local_read: erasure-coded-bytes-read is also tied to IMPALA-11697
- test_erasure_coding: Ozone doesn't report files as erasure-coded
(HDDS-7603)
Testing:
- Passes core E2E and custom cluster tests with TARGET_FILESYSTEM=ozone
and ERASURE_CODING=true.
Change-Id: I201e2e33ce94bbc1e81631a0a315884bcc8047d1
---
M bin/impala-config.sh
M testdata/cluster/admin
M testdata/cluster/node_templates/common/etc/hadoop/conf/ozone-site.xml.py
M tests/common/skip.py
M tests/custom_cluster/test_coordinators.py
M tests/query_test/test_io_metrics.py
M tests/query_test/test_scanners.py
7 files changed, 32 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/19324/8
--
To view, visit http://gerrit.cloudera.org:8080/19324
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I201e2e33ce94bbc1e81631a0a315884bcc8047d1
Gerrit-Change-Number: 19324
Gerrit-PatchSet: 8
Gerrit-Owner: Michael Smith <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
