Hello Quanlong Huang, Sai Hemanth Gantasala, Csaba Ringhofer, Impala Public
Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21029
to look at the new patch set (#3).
Change subject: IMPALA-12543: Deflake test_iceberg_self_events
......................................................................
IMPALA-12543: Deflake test_iceberg_self_events
test_iceberg_self_events has been flaky for not having
tbls_refreshed_before equal to tbls_refreshed_after in-between query
executions. This flakiness happens in ARM, TEST_JDK_VERSION 11, and
TEST_JDK_VERSION 17 environments.
This happen due to db/table level locks is not taken during db/table
self-event check (IMPALA-12461 part1). The order of ALTER TABLE
operation is as follow:
1. alter table starts in CatalogOpExecutor
2. table level lock is taken
3. HMS RPC starts (CatalogOpExecutor.applyAlterTable())
4. HMS generates the event
5. HMS RPC returns
6. table is reloaded
7. catalog version is added to inflight event list
8. table level lock is releases
Meanwhile the event processor thread fetches the new event after 4 and
before 7. Because of IMPALA-12461 (part 1), it can also finish self
event checking before reaching 7. Before IMPALA-12461 it would have
needed to wait for 8.
This patch fix the issue by adding newCatalogVersion to the table's
inflight event list before executing the ALTER TABLE operation. If HMS
update does not complete (ie., an exception is thrown), the new
newCatalogVersion that was added is then removed at finalize block of
CatalogOpExecutor.alterTable().
Testing:
- Pass exhaustive tests.
Change-Id: I8365c934349ad21a4d9327fc11594d2fc3445f79
---
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
A fe/src/main/java/org/apache/impala/catalog/InflightEventParam.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
4 files changed, 121 insertions(+), 50 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/21029/3
--
To view, visit http://gerrit.cloudera.org:8080/21029
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8365c934349ad21a4d9327fc11594d2fc3445f79
Gerrit-Change-Number: 21029
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
