Hello k.venureddy2...@gmail.com, Sai Hemanth Gantasala, Csaba Ringhofer, Impala
Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21071
to look at the new patch set (#4).
Change subject: IMPALA-12827: Fix failures in processing AbortTxnEvent due to
aborted write id is cleaned up
......................................................................
IMPALA-12827: Fix failures in processing AbortTxnEvent due to aborted write id
is cleaned up
HdfsTable tracks the ValidWriteIdList from HMS. When the table is
reloaded, the ValidWriteIdList is updated to the latest state. An
ABORT_TXN event that is lagging behind could match to aborted write ids
that have already been cleaned up by the HMS housekeeping thread. Such
write ids can't be found in the cached ValidWriteIdList as opened or
aborted write ids. This hits a Precondition check and fails the event
processing.
This patch fixes the check to allow this case. Also adds more logs for
dealing with write ids.
Tests
- Add custom-cluster test to start Hive with the housekeeping thread
turned on and verified that such ABORT_TXN event is processed
correctly.
Change-Id: I93b6f684d6e4b94961d804a0c022029249873681
---
M bin/create-test-configuration.sh
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M
fe/src/main/java/org/apache/impala/hive/common/MutableValidReaderWriteIdList.java
M fe/src/test/resources/hive-site.xml.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_events_custom_configs.py
8 files changed, 84 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/21071/4
--
To view, visit http://gerrit.cloudera.org:8080/21071
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I93b6f684d6e4b94961d804a0c022029249873681
Gerrit-Change-Number: 21071
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k.venureddy2...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com>
