tanishq-chugh commented on code in PR #6528:
URL: https://github.com/apache/hive/pull/6528#discussion_r3387549710
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ZookeeperExternalSessionsRegistryClient.java:
##########
@@ -165,4 +225,34 @@ public void childEvent(final CuratorFramework client,
final PathChildrenCacheEve
}
}
}
+
+ private final class ClaimsPathListener implements PathChildrenCacheListener {
+ @Override
+ public void childEvent(CuratorFramework client, PathChildrenCacheEvent
event) {
+ ChildData childData = event.getData();
+ if (childData == null) {
+ return;
+ }
+
+ String applicationId = getApplicationId(childData);
+ synchronized (lock) {
+ switch (event.getType()) {
+ case CHILD_REMOVED:
+ if (!taken.contains(applicationId)) {
+ // if the claim node was released by this particular HS2 itself,
+ // it will be added back to the available list & locks are
notified as part of returnSession()
+ available.add(applicationId);
+ lock.notifyAll();
+ }
+ break;
+ case CHILD_ADDED:
+ // A Tez AM was claimed by another HS2, so remove the AM from the
available list of this particular HS2
+ available.remove(applicationId);
+ break;
Review Comment:
Nice catch @ayushtkn !
In case of `CONNECTION_RECONNECTED` , the cache does replay all the missed
events, but while testing encountered a race condition between two listeners.
Have addressed the same in commit :
[26ef308](https://github.com/apache/hive/pull/6528/commits/26ef30850944bc9ef02e21fa2b3107186e72a73c)
Regarding Connection Lost case, have added logic to kill running DAGs &
clear taken state by the particular HS2 at the same time when ZK deletes its
ephemeral claim nodes, in the same commit:
[26ef308](https://github.com/apache/hive/pull/6528/commits/26ef30850944bc9ef02e21fa2b3107186e72a73c)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
