[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-5776: -- Resolution: Fixed Status: Resolved (was: Patch Available) > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: binlijin > Fix For: 0.96.0 > > Attachments: 5776-trunk-V3.patch, 5776-trunk-V4.patch, > 5776-trunk-V5.patch, 5776-trunk-V6.patch, 5776-trunk-V7.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch, > HBASE-5776-trunk-V2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-5776: -- Attachment: 5776-trunk-V7.patch Patch v7 addresses Ram's comment. TestHTableMultiplexer passes. > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: binlijin > Fix For: 0.96.0 > > Attachments: 5776-trunk-V3.patch, 5776-trunk-V4.patch, > 5776-trunk-V5.patch, 5776-trunk-V6.patch, 5776-trunk-V7.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch, > HBASE-5776-trunk-V2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[
https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-5776:
--
Attachment: 5776-trunk-V6.patch
Patch v6 replaces connection.processBatch() call because it is deprecated.
{code}
[WARNING] Javadoc Warnings
[WARNING] javadoc: warning - No source files for package
org.apache.hadoop.hbase.io.hfile.bucket
[WARNING]
/Users/tyu/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java:117:
warning - @param argument "put" is not a parameter name.
{code}
I don't know where the first warning about hfile.bucket came from.
I have corrected the second warning.
> HTableMultiplexer
> --
>
> Key: HBASE-5776
> URL: https://issues.apache.org/jira/browse/HBASE-5776
> Project: HBase
> Issue Type: Improvement
>Reporter: Liyin Tang
>Assignee: binlijin
> Fix For: 0.96.0
>
> Attachments: 5776-trunk-V3.patch, 5776-trunk-V4.patch,
> 5776-trunk-V5.patch, 5776-trunk-V6.patch,
> ASF.LICENSE.NOT.GRANTED--D2775.1.patch,
> ASF.LICENSE.NOT.GRANTED--D2775.1.patch,
> ASF.LICENSE.NOT.GRANTED--D2775.2.patch,
> ASF.LICENSE.NOT.GRANTED--D2775.2.patch,
> ASF.LICENSE.NOT.GRANTED--D2775.3.patch,
> ASF.LICENSE.NOT.GRANTED--D2775.4.patch,
> ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch,
> HBASE-5776-trunk-V2.patch
>
>
> There is a known issue in HBase client that single slow/dead region server
> could slow down the multiput operations across all the region servers. So the
> HBase client will be as slow as the slowest region server in the cluster.
>
> To solve this problem, HTableMultiplexer will separate the multiput
> submitting threads with the flush threads, which means the multiput operation
> will be a nonblocking operation.
> The submitting thread will shard all the puts into different queues based on
> its destination region server and return immediately. The flush threads will
> flush these puts from each queue to its destination region server.
> Currently the HTableMultiplexer only supports the put operation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-5776: -- Attachment: 5776-trunk-V5.patch Patch v5 adds log statement so that it is easier to read the test output > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: binlijin > Fix For: 0.96.0 > > Attachments: 5776-trunk-V3.patch, 5776-trunk-V4.patch, > 5776-trunk-V5.patch, ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch, > HBASE-5776-trunk-V2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-5776: -- Attachment: 5776-trunk-V4.patch Patch v4 added TEST_UTIL.waitUntilAllRegionsAssigned() following TEST_UTIL.createTable() call. TestHTableMultiplexer passes 3 times locally. > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: binlijin > Fix For: 0.96.0 > > Attachments: 5776-trunk-V3.patch, 5776-trunk-V4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch, > HBASE-5776-trunk-V2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-5776: -- Fix Version/s: 0.96.0 Hadoop Flags: Reviewed > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: binlijin > Fix For: 0.96.0 > > Attachments: 5776-trunk-V3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch, > HBASE-5776-trunk-V2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-5776: -- Attachment: 5776-trunk-V3.patch Patch v3 corrects a problem where TestHTableMultiplexer was referring to "hbase.htablemultiplexer.flush.frequency" but HTableMultiplexer is using "hbase.htablemultiplexer.flush.frequency.ms" I created a constant HTableMultiplexer.TABLE_MULTIPLEXER_FLUSH_FREQ_MS so that test code refers to the same config param. I looped TestHTableMultiplexer 3 times and they passed. > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: binlijin > Attachments: 5776-trunk-V3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch, > HBASE-5776-trunk-V2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-5776: Status: Patch Available (was: Open) > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch, > HBASE-5776-trunk-V2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-5776: Attachment: HBASE-5776-trunk-V2.patch > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch, > HBASE-5776-trunk-V2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-5776: Attachment: HBASE-5776-trunk.patch > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.1.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.2.patch, > ASF.LICENSE.NOT.GRANTED--D2775.3.patch, > ASF.LICENSE.NOT.GRANTED--D2775.4.patch, > ASF.LICENSE.NOT.GRANTED--D2775.5.patch, HBASE-5776-trunk.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5776: --- Attachment: D2775.5.patch Liyin updated the revision "[jira][89-fb][HBASE-5776] HTableMultiplexer". Reviewers: Kannan Addressed Kannan's comments REVISION DETAIL https://reviews.facebook.net/D2775 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/client/HConnection.java src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java src/main/java/org/apache/hadoop/hbase/client/HTable.java src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java src/main/java/org/apache/hadoop/hbase/client/MultiPut.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java To: Kannan, Liyin Cc: JIRA, tedyu > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: D2775.1.patch, D2775.1.patch, D2775.2.patch, > D2775.2.patch, D2775.3.patch, D2775.4.patch, D2775.5.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5776: --- Attachment: D2775.4.patch Liyin updated the revision "[jira][89-fb][HBASE-5776] HTableMultiplexer". Reviewers: Kannan Addressed some offline comments from Kannan about improving the failed put processing. REVISION DETAIL https://reviews.facebook.net/D2775 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/client/HConnection.java src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java src/main/java/org/apache/hadoop/hbase/client/HTable.java src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java src/main/java/org/apache/hadoop/hbase/client/MultiPut.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java To: Kannan, Liyin Cc: JIRA, tedyu > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: D2775.1.patch, D2775.1.patch, D2775.2.patch, > D2775.2.patch, D2775.3.patch, D2775.4.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5776: --- Attachment: D2775.3.patch Liyin updated the revision "[jira][89-fb][HBASE-5776] HTableMultiplexer". Reviewers: Kannan Address Kannan's comments. REVISION DETAIL https://reviews.facebook.net/D2775 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/client/HConnection.java src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java src/main/java/org/apache/hadoop/hbase/client/HTable.java src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java src/main/java/org/apache/hadoop/hbase/client/MultiPut.java src/main/java/org/apache/hadoop/hbase/client/MultiplexablePut.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java To: Kannan, Liyin Cc: JIRA, tedyu > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: D2775.1.patch, D2775.1.patch, D2775.2.patch, > D2775.2.patch, D2775.3.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5776: --- Attachment: D2775.2.patch Liyin updated the revision "[jira][89-fb][HBASE-5776] HTableMultiplexer". Reviewers: Kannan As discussed with Kannan offline, HTableMultiplexer will keep a pool htable instance. So user doesn't need to pass the htable as parameter for the put request. Also keep each put retrying several times before failing and clear the cache location after failing. Add more java doc and license. REVISION DETAIL https://reviews.facebook.net/D2775 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/client/HConnection.java src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java src/main/java/org/apache/hadoop/hbase/client/HTable.java src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java src/main/java/org/apache/hadoop/hbase/client/MultiPut.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: D2775.1.patch, D2775.1.patch, D2775.2.patch, > D2775.2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5776: --- Attachment: D2775.2.patch Liyin updated the revision "[jira][89-fb][HBASE-5776] HTableMultiplexer". Reviewers: Kannan As discussed with Kannan offline, HTableMultiplexer will keep a pool htable instance. So user doesn't need to pass the htable as parameter for the put request. Also keep each put retrying several times before failing and clear the cache location after failing. Add more java doc and license. REVISION DETAIL https://reviews.facebook.net/D2775 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/client/HConnection.java src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java src/main/java/org/apache/hadoop/hbase/client/HTable.java src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java src/main/java/org/apache/hadoop/hbase/client/MultiPut.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: D2775.1.patch, D2775.1.patch, D2775.2.patch, > D2775.2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5776: --- Attachment: D2775.1.patch Liyin requested code review of "[jira][89-fb][HBASE-5776] HTableMultiplexer". Reviewers: Kannan There is a known issue in HBase client that single slow/dead region server could slow down the multiput operations across all the region servers. So the HBase client will be as slow as the slowest region server in the cluster. To solve this problem, HTableMultiplexer will separate the multiput submitting threads with the flush threads, which means the multiput operation will be a nonblocking operation. The submitting thread will shard all the puts into different queues based on its destination region server and return immediately. The flush threads will flush these puts from each queue to its destination region server. Currently the HTableMultiplexer only supports the put operation. TEST PLAN Unit tests REVISION DETAIL https://reviews.facebook.net/D2775 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/client/HConnection.java src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java src/main/java/org/apache/hadoop/hbase/client/HTable.java src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java src/main/java/org/apache/hadoop/hbase/client/MultiPut.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6303/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: D2775.1.patch, D2775.1.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5776: --- Attachment: D2775.1.patch Liyin requested code review of "[jira][89-fb][HBASE-5776] HTableMultiplexer". Reviewers: Kannan There is a known issue in HBase client that single slow/dead region server could slow down the multiput operations across all the region servers. So the HBase client will be as slow as the slowest region server in the cluster. To solve this problem, HTableMultiplexer will separate the multiput submitting threads with the flush threads, which means the multiput operation will be a nonblocking operation. The submitting thread will shard all the puts into different queues based on its destination region server and return immediately. The flush threads will flush these puts from each queue to its destination region server. Currently the HTableMultiplexer only supports the put operation. TEST PLAN Unit tests REVISION DETAIL https://reviews.facebook.net/D2775 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/client/HConnection.java src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java src/main/java/org/apache/hadoop/hbase/client/HTable.java src/main/java/org/apache/hadoop/hbase/client/HTableMultiplexer.java src/main/java/org/apache/hadoop/hbase/client/MultiPut.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6303/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: D2775.1.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
