Hello Quanlong Huang, Vihang Karajgaonkar, Yu-Wen Lai, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18083 to look at the new patch set (#9). Change subject: IMPALA-11050: Skip filemetadata reloading in processing AlterPartition event from event processor ...................................................................... IMPALA-11050: Skip filemetadata reloading in processing AlterPartition event from event processor HdfsPartition in catalogD is a collection of files and each file is represented by a FileDescriptor. Currently when processing an AlterPartitionEvent, EventProcessor reloads partition metadata along with its file metadata. File metadata reloading is an expensive operation as it involves listing files in the underlying filesystem. This patch makes an improvement alter partition event processing for external tables. The proposed solution is to compare storage descriptor of HMS partition object received from AlterPartition event with sd of already cached HdfsPartition. Reload file metadata if any of the following is true * HdfsParition's sd does not match sd of HMS partition object * Cache directive is present for HdfsPartition. File metadata reloading is required to fetch latest file blocks info. Testing: * Added new unit tests Change-Id: I238b169f7f1122c62cbeb1434dbb675629a1e5f2 --- A fe/src/main/java/org/apache/impala/catalog/FileMetadataLoadOpts.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 5 files changed, 311 insertions(+), 29 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/18083/9 -- To view, visit http://gerrit.cloudera.org:8080/18083 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I238b169f7f1122c62cbeb1434dbb675629a1e5f2 Gerrit-Change-Number: 18083 Gerrit-PatchSet: 9 Gerrit-Owner: Sourabh Goyal <soura...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Sourabh Goyal <soura...@cloudera.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Yu-Wen Lai <yu-wen....@cloudera.com>