Quanlong Huang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19126 )
Change subject: IMPALA-11160: Ignore stale ALTER_PARTITION events on transactional tables ...................................................................... IMPALA-11160: Ignore stale ALTER_PARTITION events on transactional tables When applying ALTER_PARTITION events on transactional tables, we refresh the partition using the metadata in events if hms_event_incremental_refresh_transactional_table is enabled (which is the default). This could be wrong if the ALTER_PARTITION event is stale. The partition metadata will be rolled back to a stale state. This patch compares the eventId with the createEventId of the table and ignores those ALTER_PARTITION events that have older (smaller) event ids. Note that we already do this for many other event types, ALTER_PARTITION is somehow missing the checks. Eventually we should depend on the lastSyncedEventId and replace createEventId with it. The self-event detection can also be replaced since self-events are also stale events. These will be addressed in IMPALA-10976. Tests - Verified locally with local-catalog mode and event-processor enabled and iterated test_acid_compute_stats for 1400 times. Without the fix, the test would fail in tens of runs. Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a Reviewed-on: http://gerrit.cloudera.org:8080/19020 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> Reviewed-on: http://gerrit.cloudera.org:8080/19126 Reviewed-by: Csaba Ringhofer <[email protected]> Tested-by: Quanlong Huang <[email protected]> --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java 1 file changed, 5 insertions(+), 0 deletions(-) Approvals: Csaba Ringhofer: Looks good to me, approved Quanlong Huang: Verified -- To view, visit http://gerrit.cloudera.org:8080/19126 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: branch-4.1.1 Gerrit-MessageType: merged Gerrit-Change-Id: I5bb8cfc213093f3bbd0359c7084b277a3bd5264a Gerrit-Change-Number: 19126 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
