[jira] [Reopened] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang reopened HBASE-3899: -- Would like to get this backported to 0.90.x branch, as HBASE-5973 depends on Delayable which is from this JIRA. enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.90.6 Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.90.7, 0.92.0 Attachments: HBASE-3899-2.patch, HBASE-3899-amend-v4.patch, HBASE-3899.patch, HBASE-3899.patch, asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-3899) enhance HBase RPC to support free-ing up server handler threads even if response is not ready
[ https://issues.apache.org/jira/browse/HBASE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-3899: --- bq. I can open a new JIRA to address changes, just wanted to get clarification here first. +1 The RPC engine must be fully pluggable to support secure versus non-secure. We can avoid making a hard dependency on security APIs by conditionally building sub-packages. Can't have something breaking that modularity, especially if unused. enhance HBase RPC to support free-ing up server handler threads even if response is not ready - Key: HBASE-3899 URL: https://issues.apache.org/jira/browse/HBASE-3899 Project: HBase Issue Type: Improvement Components: ipc Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.92.0 Attachments: asyncRpc.txt, asyncRpc.txt In the current implementation, the server handler thread picks up an item from the incoming callqueue, processes it and then wraps the response as a Writable and sends it back to the IPC server module. This wastes thread-resources when the thread is blocked for disk IO (transaction logging, read into block cache, etc). It would be nice if we can make the RPC Server Handler threads pick up a call from the IPC queue, hand it over to the application (e.g. HRegion), the application can queue it to be processed asynchronously and send a response back to the IPC server module saying that the response is not ready. The RPC Server Handler thread is now ready to pick up another request from the incoming callqueue. When the queued call is processed by the application, it indicates to the IPC module that the response is now ready to be sent back to the client. The RPC client continues to experience the same behaviour as before. A RPC client is synchronous and blocks till the response arrives. This RPC enhancement allows us to do very powerful things with the RegionServer. In future, we can make enhance the RegionServer's threading model to a message-passing model for better performance. We will not be limited by the number of threads in the RegionServer. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
