Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17707 )
Change subject: IMPALA-10815: Ignore events on non-default hive catalogs ...................................................................... IMPALA-10815: Ignore events on non-default hive catalogs Hive-3 supports a new type in metastore called catalogs. Even though impala does not support custom catalogs, it is still possible that some external HMS client creates objects within a non-default catalog. This can become problematic when the objects within the custom catalog match with the name of other objects in the default catalog. For example, dropping a custom catalog generates a DROP_DATABASE event on default database of that catalog. When such event is processed, the events processor can remove the default database. This patch adds logic to ignore all the events which are generated on such non-default catalog objects. The default value of catalog is defined in the hive-site.xml of the metastore client which is used by catalogd. If the value is not present it default to "hive". Additionally, it also adds the code to validate that the default catalog name defined in the hive-site.xml of the catalogd is same as in metastore server side. If the values do not match, the events processor does not come up. This patch also removes some config validations which are specific to hive-2 since we don't support hive-2 anymore in master. Testing: 1. Added a new test which creates a custom hive catalog and events on it. The test makes sure that such events do not affect objects in catalogd. Change-Id: Ided463b2a98331d3d305bbe92fbf3a5d2e197acf Reviewed-on: http://gerrit.cloudera.org:8080/17707 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventProcessorConfig.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 5 files changed, 150 insertions(+), 89 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ided463b2a98331d3d305bbe92fbf3a5d2e197acf Gerrit-Change-Number: 17707 Gerrit-PatchSet: 7 Gerrit-Owner: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
