Thanks for the input, Jordan. My understanding is that the standard watches do but persistent watches don't. Not sure if I miss anything or if this is a bug. Looking forward to any feedback/input on this.
1. We have the following in the standard watch section of Zookeeper documentation and it looks like missing notifications are triggered. When a client reconnects, any previously registered watches will be > reregistered and triggered if needed. https://zookeeper.apache.org/doc/r3.9.3/zookeeperProgrammers.html#sc_WatchSemantics 2. In the code base, Zookeeper client library maintains lastZXid in memory and sends it to the server when resetting watches upon reconnection. The server detects if any missing notifications need to be triggered based on the lastZxid. https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/ClientCnxn.java#L1040-L1041 https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/DataTree.java#L1497 3. The problem is that missing notifications seem only being triggered for standard watches but not for persistent watches when reconnecting. For example, for standard watches, watches.process() is invoked for sending missing notifications. for (String path : dataWatches) { > DataNode node = getNode(path); > if (node == null) { > watcher.process(new WatchedEvent(EventType.NodeDeleted, > KeeperState.SyncConnected, path)); > } else if (node.stat.getMzxid() > relativeZxid) { > watcher.process(new > WatchedEvent(EventType.NodeDataChanged, KeeperState.SyncConnected, path)); > } else { > this.dataWatches.addWatch(path, watcher); > } > } https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/DataTree.java#L1494-L1521 However, for persistence watches, we only register the watches, not detecting and sending missing notifications. for (String path : persistentRecursiveWatches) { > this.dataWatches.addWatch(path, watcher, > WatcherMode.PERSISTENT_RECURSIVE); > } https://github.com/apache/zookeeper/blob/master/zookeeper-server/src/main/java/org/apache/zookeeper/server/DataTree.java#L1494-L1521 Thanks, Li
