Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17380
Change subject: IMPALA-10692: Fix acid insert when event polling is enabled ...................................................................... IMPALA-10692: Fix acid insert when event polling is enabled IMPALA-10656 broke inserts to acid tables when HMS event polling is enabled. The issue was that the new partitions created during insert were not added to the catalog table yet when createInsertEvents is called, as the table is reloaded only after firing the events and committing the transaction. The fix is to create the INSERT event based on the partition name and the fileset alone for new partitions. Already existing partitions need the Partition object as we add the event to the list of the partition's in-flight events to detect self-events, but luckily new partitions don't need self event-handling because: - new partitions fire events only if the table is ACID - ACID inserts don't fire any INSERT event visible to Impala, so it cannot cause an unnecessary metadata reload ACID inserts from Hive work differently, they always cause an ALTER_TABLE or ALTER_PARTITION event which are detected by Impala and lead to metadata reload. I think that this situation is hacky at best because these events come before COMMIT event (which is currently ignored by Impala), so Impala may reload the table too early (before the commit is finished). Testing: - added acid tables to TestEventProcessing.test_self_events Change-Id: I8c2d0702232538a746410539ad55f87b7fde57e7 --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/custom_cluster/test_event_processing.py 2 files changed, 85 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/17380/2 -- To view, visit http://gerrit.cloudera.org:8080/17380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8c2d0702232538a746410539ad55f87b7fde57e7 Gerrit-Change-Number: 17380 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer <csringho...@cloudera.com>
