Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19324 )
Change subject: IMPALA-11476: Support Ozone erasure coding ...................................................................... IMPALA-11476: Support Ozone erasure coding Adds support for identifying erasure coding policy with Ozone. Enables testing Ozone with erasure coding. Omits support for identifying erasure coding policy with the o3fs protocol as that protocol is effectively deprecated and its classes don't provide access to the ObjectStore. Refactors volumeBucketPair to use StringTokenizer. Test updates: - test_exclusive_coordinator_plan: Ozone+EC blocks are 768MB, which is larger than all tables in our test environment. Use tpch_parquet which we rely on having 3 files (by loading from snapshot in this case). - test_new_file_shorter: receives an EOFException when seeking with EC - test_local_read: erasure-coded-bytes-read is also tied to IMPALA-11697 - test_erasure_coding: Ozone doesn't report files as erasure-coded (HDDS-7603) Testing: - Passes core E2E and custom cluster tests with TARGET_FILESYSTEM=ozone and ERASURE_CODING=true. Change-Id: I201e2e33ce94bbc1e81631a0a315884bcc8047d1 Reviewed-on: http://gerrit.cloudera.org:8080/19324 Reviewed-by: Joe McDonnell <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M bin/impala-config.sh M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M fe/src/test/java/org/apache/impala/common/FileSystemUtilTest.java M testdata/cluster/admin M testdata/cluster/node_templates/common/etc/hadoop/conf/ozone-site.xml.py M tests/common/impala_test_suite.py M tests/common/skip.py M tests/custom_cluster/test_coordinators.py M tests/query_test/test_io_metrics.py M tests/query_test/test_scanners.py 10 files changed, 107 insertions(+), 31 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/19324 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I201e2e33ce94bbc1e81631a0a315884bcc8047d1 Gerrit-Change-Number: 19324 Gerrit-PatchSet: 13 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]>
