[
https://issues.apache.org/jira/browse/IMPALA-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816289#comment-17816289
]
Csaba Ringhofer commented on IMPALA-12543:
------------------------------------------
[~stigahuang]
Do you think that this can cause correctness issues, or this should only lead
to unnecessary table reloading + failed tests?
If I understand correctly what happens is:
1. alter table starts in CatalogOpExecutor
2. table level lock is taken
3. HMS RPC starts (CatalogOpExecutor.applyAlterTable())
4. HMS generates the event
5. HMS RPC returns
6. table is reloaded
7. catalog version is added to inflight event list
8. table table lock is releases
Meanwhile the event processor thread fetches the new event after 4 and before
7, and because of IMPALA-12461 (part 1), it can also finish self event
checking before reaching 7. Before IMPALA-12461 it would have needed to wait
for 8.
Currently adding to inflight event list happens here:
https://github.com/apache/impala/blob/11d2fe4fc00a1e6ef2d3a45825be9845456adc1d/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1307
Would it be a problem to move this before the HMS RPC, e.g. into
CatalogOpExecutor.applyAlterTable()?
In case the RPC or table loading fails we could remove the inflight event.
> test_iceberg_self_events failed in JDK11 build
> ----------------------------------------------
>
> Key: IMPALA-12543
> URL: https://issues.apache.org/jira/browse/IMPALA-12543
> Project: IMPALA
> Issue Type: Bug
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
> Labels: broken-build
> Attachments: catalogd.INFO, std_err.txt
>
>
> test_iceberg_self_events failed in JDK11 build with following error.
>
> {code:java}
> Error Message
> assert 0 == 1
> Stacktrace
> custom_cluster/test_events_custom_configs.py:637: in test_iceberg_self_events
> check_self_events("ALTER TABLE {0} ADD COLUMN j INT".format(tbl_name))
> custom_cluster/test_events_custom_configs.py:624: in check_self_events
> assert tbls_refreshed_before == tbls_refreshed_after
> E assert 0 == 1 {code}
> This test still pass before IMPALA-11387 merged.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
