Quanlong Huang created HIVE-27499:
-------------------------------------
Summary: Add optional table name filter to the HMS notification
fetch API
Key: HIVE-27499
URL: https://issues.apache.org/jira/browse/HIVE-27499
Project: Hive
Issue Type: New Feature
Reporter: Quanlong Huang
The current notification fetch API does not support filters on table name:
{code:java}
struct NotificationEventRequest {
1: required i64 lastEvent,
2: optional i32 maxEvents,
3: optional list<string> eventTypeSkipList,
}
NotificationEventResponse get_next_notification(1:NotificationEventRequest rqst)
{code}
Engines that cache HMS metadata and file metadata might want to know whether
they have synced to the latest events on some given tables.
In order to efficiently fetch events specific to a table, we should enhance the
notification fetch API to include an optional table name in the request so that
only events for that given table are returned. The NOTIFICATION_LOG table in
the HMS should also have an index on table_name so that API calls with the
filter do not do a table scan.
With the index added, we can also extend the NotificationEventsCountRequest
with an optional table name, and extend the response to carry the largest event
id on that table. Currently they are:
{code:java}
struct NotificationEventsCountRequest {
1: required i64 fromEventId,
2: required string dbName,
3: optional string catName,
4: optional i64 toEventId,
5: optional i64 limit
}
struct NotificationEventsCountResponse {
1: required i64 eventsCount,
}
NotificationEventsCountResponse
get_notification_events_count(1:NotificationEventsCountRequest rqst)
{code}
Similar API for multiple tables (given a table lists) will also be helpful
since a query usually involves several tables.
CC [~hemanth619], [~ngangam]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
