Hello Sai Hemanth Gantasala, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20145
to look at the new patch set (#3).
Change subject: IMPALA-12256: Fix DDLs losing partition-level create event ids
in reloads
......................................................................
IMPALA-12256: Fix DDLs losing partition-level create event ids in reloads
When HMS event-processor is enabled, each partition will track the
event id of the ADD_PARTITION event that corresponds to its creation.
The create event id is used to skip stale DROP_PARTITION events that
have lower event ids.
However, the partition-level create event ids are not correctly retained
during DDLs. DDLs that modify the partition metadata will reload the
partitions. Since HdfsPartition objects are immutable, the reload is
performed as replacing the old one with the new object created by
HdfsPartition.Builder. The HdfsPartition.Builder is built from the old
HdfsPartition object. But it doesn't copy the create event id. This
patch fixes the issue, also adds logs for comparing partition event ids.
Tests:
- Add e2e tests to verify DDLs won't loss the partition-level create
event ids, so stale ADD_PARTITION events can be skipped as expected.
Change-Id: I052faa093bda69fb16db0d424c1478bba103dad9
---
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M tests/custom_cluster/test_events_custom_configs.py
4 files changed, 59 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/20145/3
--
To view, visit http://gerrit.cloudera.org:8080/20145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I052faa093bda69fb16db0d424c1478bba103dad9
Gerrit-Change-Number: 20145
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
