Hello Quanlong Huang, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19838
to look at the new patch set (#2).
Change subject: IMPALA-11534: Skip reloading file metadata for some ALTER_TABLE
events
......................................................................
IMPALA-11534: Skip reloading file metadata for some ALTER_TABLE events
Reloading file metadata for medium to wide tables is heavy weight
operation in general. So it would be ideal from event processor
perspective to minimize file metadata reloading especially for
ALTER_TABLE statements which are quite common in metastore events.
This patch implements the above optimization by looking at before
and after table objects of an alter event and see if it corresponds
to ALTER TABLE add/change/replace column, set owner, set table
properties. If any of these are changed, the file metadata reloading
can be skipped. For inter-operability purpose this patch introduced a
new start-up flag 'file_metadata_reload_properties' which can be used
to define what table properties need file metadata to be reloaded. If
this value is set to empty, this optimization is not in effect and the
file metadata is always reloaded.
Testing: Added a unit test to confirm that, for certain alter table
statements the file metadata isn't reloaded.
Change-Id: Ia66b96a7c4b7f50fbf46b2e02296cd29a47347b6
---
M be/src/catalog/catalog-server.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M
fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java
8 files changed, 164 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/19838/2
--
To view, visit http://gerrit.cloudera.org:8080/19838
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia66b96a7c4b7f50fbf46b2e02296cd29a47347b6
Gerrit-Change-Number: 19838
Gerrit-PatchSet: 2
Gerrit-Owner: Sai Hemanth Gantasala <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
