I am running Accumulo with a large number of tables (> 10k). Each table sets up a number of watches in ZooKeeper, and we have many (> 50 million watches). We are running 5 ZooKeeper servers and the watches are spread evenly across the servers. We see Accumulo performance degrade sharply when we add more than 6k tables and it appears this is related to the number of watches in ZooKeeper. However, there seems to be a performance fix for a large number of watches, in issue # 1177[1], that was resolved Aug 2018 and merged into master with PR # 590 in release 3.6.0.
My question is how can I get the ZooKeeper 3.6.0 release? I'd like to download a copy of the 3.6.0 jar and test with this version. If all goes well, this will resolve my performance issue with Accumulo watches. A second question would be: is there a way to tune ZooKeeper for better performance with a large number of watches? This does not seem to be a optimal running state for ZooKeeper, but we are constrained by Accumulo and their use of watches. [1] https://issues.apache.org/jira/browse/ZOOKEEPER-1177 [2] https://github.com/apache/zookeeper/pull/590 Regards, Kevin
