[
https://issues.apache.org/jira/browse/HDFS-16107?focusedWorklogId=652544&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-652544
]
ASF GitHub Bot logged work on HDFS-16107:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Sep/21 05:26
Start Date: 18/Sep/21 05:26
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #3170:
URL: https://github.com/apache/hadoop/pull/3170#discussion_r711494952
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
##########
@@ -3192,23 +3196,41 @@ protected Server(String bindAddress, int port,
if (queueSizePerHandler != -1) {
this.maxQueueSize = handlerCount * queueSizePerHandler;
} else {
- this.maxQueueSize = handlerCount * conf.getInt(
- CommonConfigurationKeys.IPC_SERVER_HANDLER_QUEUE_SIZE_KEY,
- CommonConfigurationKeys.IPC_SERVER_HANDLER_QUEUE_SIZE_DEFAULT);
+ this.maxQueueSize = conf.getInt(getQueueClassPrefix() + "." +
+ CommonConfigurationKeys.SERVER_HANDLER_QUEUE_SIZE_KEY, 0);
+ if (this.maxQueueSize < 1) {
+ this.maxQueueSize = handlerCount * conf.getInt(
+ CommonConfigurationKeys.IPC_SERVER_HANDLER_QUEUE_SIZE_KEY,
+ CommonConfigurationKeys.IPC_SERVER_HANDLER_QUEUE_SIZE_DEFAULT);
+ }
+ }
+ int tmpMaxRespSize = conf.getInt(getQueueClassPrefix() + "." +
+ CommonConfigurationKeys.SERVER_RPC_MAX_RESPONSE_SIZE_KEY, 0);
+ if (tmpMaxRespSize < 1) {
+ this.maxRespSize = conf.getInt(
+ CommonConfigurationKeys.IPC_SERVER_RPC_MAX_RESPONSE_SIZE_KEY,
+ CommonConfigurationKeys.IPC_SERVER_RPC_MAX_RESPONSE_SIZE_DEFAULT);
+ } else {
+ this.maxRespSize = tmpMaxRespSize;
}
- this.maxRespSize = conf.getInt(
- CommonConfigurationKeys.IPC_SERVER_RPC_MAX_RESPONSE_SIZE_KEY,
- CommonConfigurationKeys.IPC_SERVER_RPC_MAX_RESPONSE_SIZE_DEFAULT);
if (numReaders != -1) {
this.readThreads = numReaders;
} else {
- this.readThreads = conf.getInt(
- CommonConfigurationKeys.IPC_SERVER_RPC_READ_THREADS_KEY,
- CommonConfigurationKeys.IPC_SERVER_RPC_READ_THREADS_DEFAULT);
+ this.readThreads = conf.getInt(getQueueClassPrefix() + "." +
+ CommonConfigurationKeys.SERVER_RPC_READ_THREADS_KEY, 0);
Review comment:
Hi @jianghuazhu , if we share default value with
```IPC_SERVER_RPC_READ_THREADS_KEY```, maybe we could change
```
this.readThreads = conf.getInt(getQueueClassPrefix() + "." +
CommonConfigurationKeys.SERVER_RPC_READ_THREADS_KEY, 0);
if (this.readThreads < 1) {
this.readThreads = conf.getInt(
CommonConfigurationKeys.IPC_SERVER_RPC_READ_THREADS_KEY,
CommonConfigurationKeys.IPC_SERVER_RPC_READ_THREADS_DEFAULT);
}
```
to
```this.readThreads = conf.getInt(prefix + "." +
CommonConfigurationKeys.SERVER_RPC_READ_THREADS_KEY,
CommonConfigurationKeys.IPC_SERVER_RPC_READ_THREADS_DEFAULT);```.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 652544)
Time Spent: 3h 20m (was: 3h 10m)
> Split RPC configuration to isolate RPC
> --------------------------------------
>
> Key: HDFS-16107
> URL: https://issues.apache.org/jira/browse/HDFS-16107
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> For RPC of different ports, there are some common configurations, such as:
> ipc.server.read.threadpool.size
> ipc.server.read.connection-queue.size
> ipc.server.handler.queue.size
> Once we configure these values, it will affect all requests (including client
> and requests within the cluster).
> It is necessary for us to split these configurations to adapt to different
> ports, such as:
> ipc.8020.server.read.threadpool.size
> ipc.8021.server.read.threadpool.size
> ipc.8020.server.read.connection-queue.size
> ipc.8021.server.read.connection-queue.size
> The advantage of this is to isolate the RPC to deal with the pressure of
> requests from all sides.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
