Vihang Karajgaonkar created HIVE-25796:
------------------------------------------
Summary: Allow metastore clients to fetch remaining events if some
of the events are cleaned up
Key: HIVE-25796
URL: https://issues.apache.org/jira/browse/HIVE-25796
Project: Hive
Issue Type: Improvement
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
This is the code snippet from HiveMetastoreClient.java's getNextNotification
method
{noformat}
for (NotificationEvent e : rsp.getEvents()) {
LOG.debug("Got event with id : {}", e.getEventId());
if (e.getEventId() != nextEventId) {
if (e.getEventId() == prevEventId) {
LOG.error("NOTIFICATION_LOG table has multiple events with the same
event Id {}. " +
"Something went wrong when inserting notification events.
Bootstrap the system " +
"again to get back teh consistent replicated state.",
prevEventId);
throw new
IllegalStateException(REPL_EVENTS_WITH_DUPLICATE_ID_IN_METASTORE);
} else {
LOG.error("Requested events are found missing in NOTIFICATION_LOG
table. Expected: {}, Actual: {}. "
+ "Probably, cleaner would've cleaned it up. "
+ "Try setting higher value for
hive.metastore.event.db.listener.timetolive. "
+ "Also, bootstrap the system again to get back the
consistent replicated state.",
nextEventId, e.getEventId());
throw new IllegalStateException(REPL_EVENTS_MISSING_IN_METASTORE);
}
}
{noformat}
Consider the case when a client which caches a event id and tries to fetch the
next events since the eventid after long time. In this case, it is possible
that Metastore has cleaned up the events because they were more than 24 hrs
old. In such a case, this API throws an exception. It is possible that client
does not care if the events are not in sequence and hence this exception should
be thrown optionally depending on what the client wants.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
