clintropolis commented on a change in pull request #9805:
URL: https://github.com/apache/druid/pull/9805#discussion_r449714132
##########
File path: docs/dependencies/zookeeper.md
##########
@@ -74,3 +74,22 @@
${druid.zk.paths.loadQueuePath}/_host_of_historical_process/_segment_identifier
```
This znode will contain a payload that indicates to the Historical process
what it should do with the given segment. When the Historical process is done
with the work, it will delete the znode in order to signify to the Coordinator
that it is complete.
+
+## Common zookeeper related config
+
+### Znode size limit
+When dealing with large data volumn and a lot of segments, errors can be
caused due to Znode size limit exceeded.
Review comment:
```suggestion
When dealing with large data volume and a lot of segments, errors can be
caused due to Znode size limit exceeded.
```
##########
File path: docs/dependencies/zookeeper.md
##########
@@ -74,3 +74,22 @@
${druid.zk.paths.loadQueuePath}/_host_of_historical_process/_segment_identifier
```
This znode will contain a payload that indicates to the Historical process
what it should do with the given segment. When the Historical process is done
with the work, it will delete the znode in order to signify to the Coordinator
that it is complete.
+
+## Common zookeeper related config
+
+### Znode size limit
+When dealing with large data volumn and a lot of segments, errors can be
caused due to Znode size limit exceeded.
+There are at least 3 places the znode size can be configured:
+- Druid nodes runtime property:
+`druid.indexer.runner.maxZnodeBytes` works as druid internal znode limit on
middleManagers and overlods.
Review comment:
```suggestion
`druid.indexer.runner.maxZnodeBytes` works as Druid internal znode limit on
MiddleManagers, Indexers, and Overlords.
```
##########
File path: docs/dependencies/zookeeper.md
##########
@@ -74,3 +74,22 @@
${druid.zk.paths.loadQueuePath}/_host_of_historical_process/_segment_identifier
```
This znode will contain a payload that indicates to the Historical process
what it should do with the given segment. When the Historical process is done
with the work, it will delete the znode in order to signify to the Coordinator
that it is complete.
+
+## Common zookeeper related config
+
+### Znode size limit
+When dealing with large data volumn and a lot of segments, errors can be
caused due to Znode size limit exceeded.
+There are at least 3 places the znode size can be configured:
+- Druid nodes runtime property:
+`druid.indexer.runner.maxZnodeBytes` works as druid internal znode limit on
middleManagers and overlods.
+
+- Zookeeper client side config:
+`jute.maxbuffer` Java system property works as a zookeeper client side limit
on druid nodes.
+
+- Zookeeper server side config:
+`jute.maxbuffer` Java system property works as a zookeeper server side limit
on your zookeeper cluster.
Review comment:
```suggestion
`jute.maxbuffer` Java system property works as a ZooKeeper server side limit
on your ZooKeeper cluster.
```
##########
File path: docs/dependencies/zookeeper.md
##########
@@ -74,3 +74,22 @@
${druid.zk.paths.loadQueuePath}/_host_of_historical_process/_segment_identifier
```
This znode will contain a payload that indicates to the Historical process
what it should do with the given segment. When the Historical process is done
with the work, it will delete the znode in order to signify to the Coordinator
that it is complete.
+
+## Common zookeeper related config
+
+### Znode size limit
+When dealing with large data volumn and a lot of segments, errors can be
caused due to Znode size limit exceeded.
+There are at least 3 places the znode size can be configured:
+- Druid nodes runtime property:
+`druid.indexer.runner.maxZnodeBytes` works as druid internal znode limit on
middleManagers and overlods.
+
+- Zookeeper client side config:
+`jute.maxbuffer` Java system property works as a zookeeper client side limit
on druid nodes.
Review comment:
```suggestion
`jute.maxbuffer` Java system property that works as a ZooKeeper client side
limit on Druid nodes.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
