Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18066
Change subject: IMPALA-10886: Fix lossing createEventId for INSERT created partitions ...................................................................... IMPALA-10886: Fix lossing createEventId for INSERT created partitions Coordinator calls updateCatalog RPC to catalogd to finalize the INSERT statement. Catalogd will create any new partitions and reload metadata of the updated partitions (including new partitions). After IMPALA-10502, each new partition has a createEventId which is used to detect how "fresh" it is. If the event processor receives a DROP_PARTITION event, it will compare its event id with the createEventId of the partition. Only DROP_PARTITION events happen after the createEventId will be evaluated. There is a bug in CatalogOpExecutor#updateCatalog() that we loss the createEventId of the new partitions. It should be used in the final call of loadTableMetadata(). This bug causes intermittent failures in TestReusePartitionMetadata.test_reuse_partition_meta. The last two DMLs of the test is dropping a partition and then creating it back by an INSERT. If the DROP_PARTITION event is processed after the INSERT finishes, the partition will be dropped incorrectly. Tests - Ran the test 100 times locally. Without the fix, it fails in 10 runs. Change-Id: I2622c28a5ce6084fc77f6ea475d2633445c7f8dd --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/custom_cluster/test_local_catalog.py 2 files changed, 10 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/18066/1 -- To view, visit http://gerrit.cloudera.org:8080/18066 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2622c28a5ce6084fc77f6ea475d2633445c7f8dd Gerrit-Change-Number: 18066 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <[email protected]>
