Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21071
Change subject: IMPALA-12827: Fix failures in processing AbortTxnEvent due to aborted write id is cleaned up ...................................................................... IMPALA-12827: Fix failures in processing AbortTxnEvent due to aborted write id is cleaned up HdfsTable tracks the ValidWriteIdList from HMS. When the table is reloaded, the ValidWriteIdList is updated to the latest state. An ABORT_TXN event that is lagging behind could match to aborted write ids that have already been cleaned up by the HMS housekeeping thread. Such write ids can't be found in the cached ValidWriteIdList as opened or aboorted write ids. This hits a Precondition check and fails the event processing. This patch fixes the check to allow this case. Also adds more logs for dealing with write ids. Tests - Add custom-cluster test to start Hive with the housekeeping thread turned on and verify such ABORT_TXN event is processed correctly. Change-Id: I93b6f684d6e4b94961d804a0c022029249873681 --- M bin/create-test-configuration.sh M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/hive/common/MutableValidReaderWriteIdList.java M fe/src/test/resources/hive-site.xml.py M tests/common/impala_test_suite.py M tests/custom_cluster/test_events_custom_configs.py 8 files changed, 78 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/21071/1 -- To view, visit http://gerrit.cloudera.org:8080/21071 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I93b6f684d6e4b94961d804a0c022029249873681 Gerrit-Change-Number: 21071 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <[email protected]>
