[jira] [Commented] (HBASE-6103) Optimize the HBaseServer to deserialize the data for each ipc connection in parallel
[ https://issues.apache.org/jira/browse/HBASE-6103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13554567#comment-13554567 ] Devaraj Das commented on HBASE-6103: Ugh I meant to say HBASE-2941 (and not HBASE-2713). > Optimize the HBaseServer to deserialize the data for each ipc connection in > parallel > > > Key: HBASE-6103 > URL: https://issues.apache.org/jira/browse/HBASE-6103 > Project: HBase > Issue Type: Improvement > Components: Performance >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: HBASE-6103-fb-89.patch > > > Currently HBaseServer is running with a single listener thread, which is > responsible for accepting the connection, reading the data from network > channel, deserializing the data into writable objects and handover to the IPC > handler threads. > When there are multiple hbase clients connecting to the region server > (HBaseServer) and reading/writing a large set of data, the listener and the > respond thread will be performance bottleneck. > So the solution is to deserialize the data for each ipc connection in > parallel for HBaseServer > BTW, it is also one of the reasons that the parallel scanning from multiple > clients is far slower than single client case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6103) Optimize the HBaseServer to deserialize the data for each ipc connection in parallel
[ https://issues.apache.org/jira/browse/HBASE-6103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509311#comment-13509311 ] stack commented on HBASE-6103: -- https://issues.apache.org/jira/browse/HBASE-2941 added a fixed-size pool of Readers. Its like this patch but its fixed size for the pool rather than number of processors and puts stuff on a queue if busy. It looks like we could pick up a few of Liyin's improvements. Worth a bit of study before closing it out. > Optimize the HBaseServer to deserialize the data for each ipc connection in > parallel > > > Key: HBASE-6103 > URL: https://issues.apache.org/jira/browse/HBASE-6103 > Project: HBase > Issue Type: Improvement > Components: Performance >Reporter: Liyin Tang >Assignee: Liyin Tang >Priority: Critical > Fix For: 0.96.0 > > Attachments: HBASE-6103-fb-89.patch > > > Currently HBaseServer is running with a single listener thread, which is > responsible for accepting the connection, reading the data from network > channel, deserializing the data into writable objects and handover to the IPC > handler threads. > When there are multiple hbase clients connecting to the region server > (HBaseServer) and reading/writing a large set of data, the listener and the > respond thread will be performance bottleneck. > So the solution is to deserialize the data for each ipc connection in > parallel for HBaseServer > BTW, it is also one of the reasons that the parallel scanning from multiple > clients is far slower than single client case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6103) Optimize the HBaseServer to deserialize the data for each ipc connection in parallel
[ https://issues.apache.org/jira/browse/HBASE-6103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467132#comment-13467132 ] stack commented on HBASE-6103: -- HBASE-6619 overwrites this patch and technique. Confirm after HBASE-6619 goes in and if so resolve as won't fix. > Optimize the HBaseServer to deserialize the data for each ipc connection in > parallel > > > Key: HBASE-6103 > URL: https://issues.apache.org/jira/browse/HBASE-6103 > Project: HBase > Issue Type: Improvement > Components: Performance >Reporter: Liyin Tang >Assignee: Liyin Tang >Priority: Critical > Fix For: 0.96.0 > > Attachments: HBASE-6103-fb-89.patch > > > Currently HBaseServer is running with a single listener thread, which is > responsible for accepting the connection, reading the data from network > channel, deserializing the data into writable objects and handover to the IPC > handler threads. > When there are multiple hbase clients connecting to the region server > (HBaseServer) and reading/writing a large set of data, the listener and the > respond thread will be performance bottleneck. > So the solution is to deserialize the data for each ipc connection in > parallel for HBaseServer > BTW, it is also one of the reasons that the parallel scanning from multiple > clients is far slower than single client case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
