Zhihua Deng created HIVE-28546:
----------------------------------
Summary: Optimize fire_listener_event API
Key: HIVE-28546
URL: https://issues.apache.org/jira/browse/HIVE-28546
Project: Hive
Issue Type: Improvement
Security Level: Public (Viewable by anyone)
Reporter: Zhihua Deng
Assignee: Zhihua Deng
The client can use the fire_listener_event to notify the change event to
downstream consumer, the event needs to be sequential, this is implemented by
locking the row in
NOTIFICATION_SEQUENCE when serializing the event to database, so this could be
a bottleneck in concurrent workloads. Currently the Hive client sends this
event per partition for loading dynamic partitions, and the HMS processes them
one by one. It would be way more efficient when sending or processing them in
bulk.
This fix will be incompatible with the older consumer that expect single
partition per InsertEvent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
