Quanlong Huang has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/20148 )
Change subject: IMPALA-12257: Fix NPE in createInsertEvents when partitions only exist in HMS ...................................................................... IMPALA-12257: Fix NPE in createInsertEvents when partitions only exist in HMS When files are generated for INSERT statements, coordinator invokes the updateCatalog RPC to let catalogd finalize the HMS changes. Catalogd will fire INSERT events for external tables at the end of this work. This helps other systems that consume HMS events (e.g. Hive replication or other Impala clusters) be notified for the changes. It should be best-effort that any errors in it should not fail the INSERT statement. We've seen a NullPointerException thrown in createInsertEvents() in the case that an updated partition doesn't exist in the metadata cache of catalogd. Note that the partition list cached in catalogd could differ from the actual list in HMS. If an INSERT statement modifies a partition that exists in HMS but not yet synced to catalogd, createInsertEvents() will throw a NPE when finding such a partition in the table. This fixes the NPE by skipping this step and using the partition names directly. This patch also refactors createInsertEvents to make it shorter. Tests - Add e2e tests Change-Id: I7d77844e26283ecb16b3b3aeb9f634bb3113eacd --- M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M tests/custom_cluster/test_insert_behaviour.py 3 files changed, 131 insertions(+), 68 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/20148/4 -- To view, visit http://gerrit.cloudera.org:8080/20148 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7d77844e26283ecb16b3b3aeb9f634bb3113eacd Gerrit-Change-Number: 20148 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang <[email protected]>
