Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18083
to look at the new patch set (#6).
Change subject: [WIP] IMPALA-11050: Skip filemetadata reloading in processing
AlterPartition event from event processor
......................................................................
[WIP] IMPALA-11050: Skip filemetadata reloading in processing AlterPartition
event from event
processor
HdfsPartition in catalogD is a collection of files and each file is represented
by a FileDescriptor.
When the event processor processes an AlterPartitionEvent, currently it reloads
partition metadata
along with its file metadata. Reloading of file metadata is a relatively
expensive operation as it
involves listing files in the underlying filesystem.
This patch makes an improvement in processing alter partition event for
external tables. The
proposed solution is to compare sd of HMS partition object received from
AlterPartition event with
sd of already cached HdfsPartition. Reload file metadata if any of the
following is true
* HdfsParition's sd does not match sd of HMS partition object
* Cache directive is present for HdfsPartition. File metadata reloading is done
to fetch latest
file blocks info.
Testing:
* Added new unit tests
Change-Id: I238b169f7f1122c62cbeb1434dbb675629a1e5f2
---
A fe/src/main/java/org/apache/impala/catalog/FileMetadataLoadOpts.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M
fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
5 files changed, 315 insertions(+), 29 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/18083/6
--
To view, visit http://gerrit.cloudera.org:8080/18083
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I238b169f7f1122c62cbeb1434dbb675629a1e5f2
Gerrit-Change-Number: 18083
Gerrit-PatchSet: 6
Gerrit-Owner: Sourabh Goyal <soura...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
