Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19378 )
Change subject: IMPALA:11808: Added support for reload event in catalogD ...................................................................... Patch Set 5: (5 comments) http://gerrit.cloudera.org:8080/#/c/19378/5/be/src/catalog/catalog-server.cc File be/src/catalog/catalog-server.cc: http://gerrit.cloudera.org:8080/#/c/19378/5/be/src/catalog/catalog-server.cc@127 PS5, Line 127: enable_processing_reload_event The name sounds like only applying on the processing side and not the triggering side. I think "enable_reload_events" is better since there is a similar flag of enable_insert_events. http://gerrit.cloudera.org:8080/#/c/19378/5/fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java File fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java: http://gerrit.cloudera.org:8080/#/c/19378/5/fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java@446 PS5, Line 446: FireEventRequestData data = new FireEventRequestData(); : data.setRefreshEvent(isRefresh); : FireEventRequest rqst = new FireEventRequest(true, data); We can't use these if they don't exist in Apache Hive3. We can make this method a no-op and return an empty list. Need to update the comment for this as well. http://gerrit.cloudera.org:8080/#/c/19378/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/19378/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@2444 PS2, Line 2444: Preconditions.checkArgument(MetastoreEventType.RELOAD.equals(getEventType())); : Preconditions.checkNotNull(event.getMessage()); : ReloadMessage reloadMessage = : MetastoreEventsProcessor.getMessageDeserializer() : .getReloadMessage(event.getMessage()); : try { : msTbl_ = Preconditions.checkNotNull(reloadMe > We might need to move this into MetastoreShim if Apache Hive3 doesn't have This still needs to be addressed. http://gerrit.cloudera.org:8080/#/c/19378/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/19378/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@6410 PS5, Line 6410: LOG.warn(String.format("Couldn't obtain a version lock for the table: %s", nit: also mention that reload events won't be fired. http://gerrit.cloudera.org:8080/#/c/19378/5/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/19378/5/tests/custom_cluster/test_events_custom_configs.py@254 PS5, Line 254: .format(unique_database, test_reload_table)) Can we also verify the fired reload event? We have an example here: https://github.com/apache/impala/blob/11068d9aebd83be59773f2b1eaa5682053d4428b/tests/metadata/test_event_processing.py#L406-L412 -- To view, visit http://gerrit.cloudera.org:8080/19378 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic62d58837d356dc2113f3c0904228ac9de484136 Gerrit-Change-Number: 19378 Gerrit-PatchSet: 5 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Wed, 04 Jan 2023 03:43:42 +0000 Gerrit-HasComments: Yes
