Sai Hemanth Gantasala has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19838
Change subject: IMPALA-11534: Skip reloading file metadata for some ALTER_TABLE events ...................................................................... IMPALA-11534: Skip reloading file metadata for some ALTER_TABLE events Reloading file metadata for medium to wide tables is heavy weight operation in general. So it would be ideal from event processor perspective to minimize file metadata reloading especially for ALTER_TABLE statements which are quite common in metastore events. This patch implements the above optimization by looking at before and after table objects of an alter event and see if it corresponds to ALTER TABLE add/change/replace column, set owner, set table properties. If any of these are changed, the file metadata reloading can be skipped. For inter-operability purpose this patch introduced a new start-up flag 'file_metadata_reload_whitelist' which can be used to define what table properties need file metadata to be reloaded. If this value is set to empty, this optimization is not in effect and the file metadata is always reloaded. Testing: Added a unit test to confirm that, for certain alter table statements the file metadata isn't reloaded. Change-Id: Ia66b96a7c4b7f50fbf46b2e02296cd29a47347b6 --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 8 files changed, 161 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/19838/1 -- To view, visit http://gerrit.cloudera.org:8080/19838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia66b96a7c4b7f50fbf46b2e02296cd29a47347b6 Gerrit-Change-Number: 19838 Gerrit-PatchSet: 1 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]>
