Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19838 )
Change subject: IMPALA-11534: Skip reloading file metadata for some ALTER_TABLE events ...................................................................... IMPALA-11534: Skip reloading file metadata for some ALTER_TABLE events Reloading file metadata for medium to wide tables is heavy weight operation in general. So it would be ideal from event processor perspective to minimize file metadata reloading especially for ALTER_TABLE statements which are quite common in metastore events. This patch implements the above optimization by looking at before and after table objects of an alter event and see if it corresponds to ALTER TABLE add/change/replace column, set owner, set table properties. If any of these are changed, the file metadata reloading can be skipped. For inter-operability purpose this patch introduced a new start-up flag 'file_metadata_reload_properties' which can be used to define what table properties need file metadata to be reloaded. If this value is set to empty, this optimization is not in effect and the file metadata is always reloaded. Testing: Added a unit test to confirm that, for certain alter table statements the file metadata isn't reloaded. Change-Id: Ia66b96a7c4b7f50fbf46b2e02296cd29a47347b6 Reviewed-on: http://gerrit.cloudera.org:8080/19838 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/catalog/catalog-server.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 8 files changed, 241 insertions(+), 12 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19838 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia66b96a7c4b7f50fbf46b2e02296cd29a47347b6 Gerrit-Change-Number: 19838 Gerrit-PatchSet: 19 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
