[
https://issues.apache.org/jira/browse/HIVE-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022329#comment-16022329
]
anishek edited comment on HIVE-16738 at 5/24/17 5:22 AM:
---------------------------------------------------------
[~mohitsabharwal], i am looking at use cases where we have two separate
instances of HS2 running in different machines(say HS1 and HS2) with both of
them connecting to a common hive metastore and the clients re connecting to one
of the hive server 2 machines randomly,
client 1 connects to HS1
client 2 connects to hs2
threads on HS1 and HS2 should be able to get a lock on
{{NOTIFICATION_TBL_LOCK}} at the same time => hence entering the critical
section to update the notification Event which would read the notification
sequence entry with primary key 1 in both cases and update to the same value
for possibly two events ?
was (Author: anishek):
[~mohitsabharwal], i am looking at use cases where we have two separate
instances of HS2 running in different machines(say HS1 and HS2) with both of
them connecting to a common hive metastore and the clients re connecting to one
of the hive server 2 machines randomly,
client 1 connects to HS1
client 2 connects to hs2
threads on HS1 and HS2 should be able to get a lock on {{NOTIFICATION_TBL_LOCK
}} at the same time => hence entering the critical section to update the
notification Event which would read the notification sequence entry with
primary key 1 in both cases and update to the same value for possibly two
events ?
> Notification ID generation in DBNotification might not be unique across HS2
> instances.
> --------------------------------------------------------------------------------------
>
> Key: HIVE-16738
> URL: https://issues.apache.org/jira/browse/HIVE-16738
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 3.0.0
> Reporter: anishek
> Assignee: anishek
> Fix For: 3.0.0
>
>
> Going to explain the problem in scope of "replication" feature for hive 2
> that is being built, as it is easier to explain:
> To allow replication to work we need to set
> "hive.metastore.transactional.event.listeners" to DBNotificationListener.
> For use cases where there are multiple HiveServer2 Instances running
> {code}
> private void process(NotificationEvent event, ListenerEvent listenerEvent)
> throws MetaException {
> event.setMessageFormat(msgFactory.getMessageFormat());
> synchronized (NOTIFICATION_TBL_LOCK) {
> LOG.debug("DbNotificationListener: Processing : {}:{}",
> event.getEventId(),
> event.getMessage());
> HMSHandler.getMSForConf(hiveConf).addNotificationEvent(event);
> }
> // Set the DB_NOTIFICATION_EVENT_ID for future reference by other
> listeners.
> if (event.isSetEventId()) {
> listenerEvent.putParameter(
> MetaStoreEventListenerConstants.DB_NOTIFICATION_EVENT_ID_KEY_NAME,
> Long.toString(event.getEventId()));
> }
> }
> {code}
> the above code in DBNotificationListner having the object lock wont be
> guarantee enough to make sure that all events get a unique id. The
> transaction isolation level at the db "read-comitted" or "repeatable-read"
> would also not guarantee the same, unless a lock is at the db level
> preferably on table {{NOTIFICATION_SEQUENCE}} which only has one row.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
