[jira] [Commented] (HDFS-16134) Optimize RPC throughput
[ https://issues.apache.org/jira/browse/HDFS-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387686#comment-17387686 ] JiangHua Zhu commented on HDFS-16134: - [~hexiaoqiao], thanks for your comment. I will continue to pay attention. > Optimize RPC throughput > --- > > Key: HDFS-16134 > URL: https://issues.apache.org/jira/browse/HDFS-16134 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > > Now RPC has the following phenomena: > 1. Each RPC has 1 Responder, which is used to return the result, but many > client requests are directly returned after the handler is processed; > 2. Call#Connection#responseQueue uses synchronous control every time it is > used; > These phenomena will hinder the performance of RPC, we can try to make some > changes. > 1. Set multiple responders for each RPC, and the user returns the result; > 2. After the Handler processing is completed, add the results that need to be > returned to a queue, and the Responder will process the queue data; > 3. Add an identifier similar to sequenceId to ensure the order of return; -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16134) Optimize RPC throughput
[ https://issues.apache.org/jira/browse/HDFS-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386828#comment-17386828 ] Xiaoqiao He commented on HDFS-16134: Thanks [~jianghuazhu] involve me here. HDFS-15486, HDFS-15869 seems to report the same issue as here. Especially https://github.com/apache/hadoop/pull/2737/files has submitted patch and try to improve it by parallel way but no agreement finally. If you are interested it let us try to push it forward again. In my practice it could improve performance above 5% when high load in my internal production environment. > Optimize RPC throughput > --- > > Key: HDFS-16134 > URL: https://issues.apache.org/jira/browse/HDFS-16134 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: JiangHua Zhu >Priority: Major > > Now RPC has the following phenomena: > 1. Each RPC has 1 Responder, which is used to return the result, but many > client requests are directly returned after the handler is processed; > 2. Call#Connection#responseQueue uses synchronous control every time it is > used; > These phenomena will hinder the performance of RPC, we can try to make some > changes. > 1. Set multiple responders for each RPC, and the user returns the result; > 2. After the Handler processing is completed, add the results that need to be > returned to a queue, and the Responder will process the queue data; > 3. Add an identifier similar to sequenceId to ensure the order of return; -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16134) Optimize RPC throughput
[ https://issues.apache.org/jira/browse/HDFS-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384959#comment-17384959 ] Kihwal Lee commented on HDFS-16134: --- [~daryn] > Optimize RPC throughput > --- > > Key: HDFS-16134 > URL: https://issues.apache.org/jira/browse/HDFS-16134 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: JiangHua Zhu >Priority: Major > > Now RPC has the following phenomena: > 1. Each RPC has 1 Responder, which is used to return the result, but many > client requests are directly returned after the handler is processed; > 2. Call#Connection#responseQueue uses synchronous control every time it is > used; > These phenomena will hinder the performance of RPC, we can try to make some > changes. > 1. Set multiple responders for each RPC, and the user returns the result; > 2. After the Handler processing is completed, add the results that need to be > returned to a queue, and the Responder will process the queue data; > 3. Add an identifier similar to sequenceId to ensure the order of return; -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16134) Optimize RPC throughput
[ https://issues.apache.org/jira/browse/HDFS-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384724#comment-17384724 ] JiangHua Zhu commented on HDFS-16134: - [~elgoiri], [~ayushtkn], [~hexiaoqiao], do you have any different ideas? Welcome to communicate. > Optimize RPC throughput > --- > > Key: HDFS-16134 > URL: https://issues.apache.org/jira/browse/HDFS-16134 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: JiangHua Zhu >Priority: Major > > Now RPC has the following phenomena: > 1. Each RPC has 1 Responder, which is used to return the result, but many > client requests are directly returned after the handler is processed; > 2. Call#Connection#responseQueue uses synchronous control every time it is > used; > These phenomena will hinder the performance of RPC, we can try to make some > changes. > 1. Set multiple responders for each RPC, and the user returns the result; > 2. After the Handler processing is completed, add the results that need to be > returned to a queue, and the Responder will process the queue data; > 3. Add an identifier similar to sequenceId to ensure the order of return; -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
