szetszwo commented on code in PR #5954:
URL: https://github.com/apache/ozone/pull/5954#discussion_r1449102234
##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -1090,15 +1090,41 @@
<value>100</value>
<tag>OZONE, MANAGEMENT, PERFORMANCE</tag>
<description>
- The number of RPC handler threads for each SCM service
- endpoint.
-
- The default is appropriate for small clusters (tens of nodes).
-
- Set a value that is appropriate for the cluster size. Generally, HDFS
- recommends RPC handler count is set to 20 * log2(Cluster Size) with an
- upper limit of 200. However, SCM will not have the same amount of
- traffic as Namenode, so a value much smaller than that will work well
too.
Review Comment:
Let's keep the old description. We may change `Namenode` to `HDFS Namenode`.
##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -1090,15 +1090,41 @@
<value>100</value>
<tag>OZONE, MANAGEMENT, PERFORMANCE</tag>
<description>
- The number of RPC handler threads for each SCM service
- endpoint.
-
- The default is appropriate for small clusters (tens of nodes).
-
- Set a value that is appropriate for the cluster size. Generally, HDFS
- recommends RPC handler count is set to 20 * log2(Cluster Size) with an
- upper limit of 200. However, SCM will not have the same amount of
- traffic as Namenode, so a value much smaller than that will work well
too.
+ Used to set the default number of handler threads for
+ RPC of the SCM service.
+ If you need to set different handlers for a specific RPC,
+ you can refer to the following mapping relationship:
+ ---- RPC type ---- : ---- Configuration items ----
+ SCMClientProtocolServer: 'ozone.scm.client.handler.count.key'
+ SCMBlockProtocolServer: 'ozone.scm.block.handler.count.key'
+ SCMDatanodeProtocolServer: 'ozone.scm.datanode.handler.count.key'
Review Comment:
How about we revise the paragraph as below?
```
To specify handlers for individual RPC servers,
set the following configuration properties instead:
---- RPC type ---- : ---- Configuration properties ----
SCMClientProtocolServer : 'ozone.scm.client.handler.count.key'
SCMBlockProtocolServer : 'ozone.scm.block.handler.count.key'
SCMDatanodeProtocolServer: 'ozone.scm.datanode.handler.count.key'
```
The entire description will look like
```
The number of RPC handler threads for each SCM service
endpoint.
The default is appropriate for small clusters (tens of nodes).
Set a value that is appropriate for the cluster size. Generally, HDFS
recommends RPC handler count is set to 20 * log2(Cluster Size) with an
upper limit of 200. However, Ozone SCM will not have the same amount of
traffic as HDFS Namenode, so a value much smaller than that will work
well too.
To specify handlers for individual RPC servers,
set the following configuration properties instead:
---- RPC type ---- : ---- Configuration properties ----
SCMClientProtocolServer : 'ozone.scm.client.handler.count.key'
SCMBlockProtocolServer : 'ozone.scm.block.handler.count.key'
SCMDatanodeProtocolServer: 'ozone.scm.datanode.handler.count.key'
```
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMBlockProtocolServer.java:
##########
@@ -106,9 +107,9 @@ public SCMBlockProtocolServer(OzoneConfiguration conf,
StorageContainerManager scm) throws IOException {
this.scm = scm;
this.conf = conf;
- final int handlerCount =
+ final int handlerCount = conf.getInt(OZONE_SCM_BLOCK_HANDLER_COUNT_KEY,
conf.getInt(OZONE_SCM_HANDLER_COUNT_KEY,
- OZONE_SCM_HANDLER_COUNT_DEFAULT);
+ OZONE_SCM_HANDLER_COUNT_DEFAULT));
Review Comment:
Let's add a utility method in `OzoneConfiguration`.
```java
//OzoneConfiguration.java
public int getInt(String name, String fallbackName, int defaultValue,
Consumer<String> log) {
String value = this.getTrimmed(name);
if (value == null) {
value = this.getTrimmed(fallbackName);
if (log != null) {
log.accept(name + " is not set. Fallback to " + fallbackName + ",
which is set to " + value);
}
}
if (value == null) {
return defaultValue;
}
return Integer.parseInt(value);
}
```
Then, we can use it here and also for the other cases.
```java
final int handlerCount = conf.getInt(OZONE_SCM_BLOCK_HANDLER_COUNT_KEY,
OZONE_SCM_HANDLER_COUNT_KEY, OZONE_SCM_HANDLER_COUNT_DEFAULT,
LOG::info);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
