[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Resolution: Fixed Fix Version/s: 0.94.4 0.96.0 Status: Resolved (was: Patch Available) Integrated into 0.94 and trunk. Thanks all. Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Fix For: 0.96.0, 0.94.4 Attachments: 0.94-6423.patch, 0.94-6423_v4.patch, trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch, trunk-6423_v3.4.patch, trunk-6423_v4.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6423: -- Attachment: 6423.addendum Addendum that fixes TestHeapSize Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Fix For: 0.96.0, 0.94.4 Attachments: 0.94-6423.patch, 0.94-6423_v4.patch, 6423.addendum, trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch, trunk-6423_v3.4.patch, trunk-6423_v4.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423_v3.4.patch 0.94-6423.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: 0.94-6423.patch, trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch, trunk-6423_v3.4.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Open (was: Patch Available) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: 0.94-6423.patch, 0.94-6423_v4.patch, trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch, trunk-6423_v3.4.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: 0.94-6423_v4.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: 0.94-6423.patch, 0.94-6423_v4.patch, trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch, trunk-6423_v3.4.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423_v4.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: 0.94-6423.patch, 0.94-6423_v4.patch, trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch, trunk-6423_v3.4.patch, trunk-6423_v4.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Patch Available (was: Open) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: 0.94-6423.patch, 0.94-6423_v4.patch, trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch, trunk-6423_v3.4.patch, trunk-6423_v4.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Open (was: Patch Available) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423_v3.3.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Patch Available (was: Open) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch, trunk-6423_v3.3.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423_v3.2.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Release Note: Added new configuration parameters to adjust the wait time in getting a lock before acting on a region: hbase.busy.wait.multiplier.max (default 2) hbase.rowlock.wait.duration (same as the default RPC timeout time) For reading, it waits at most hbase.rowlock.wait.duration in getting a lock. For writing, it waits at most hbase.rowlock.wait.duration * min ( #rows affected, hbase.busy.wait.multiplier.max). However, it waits at most the server side call purge timeout time. Hadoop Flags: Reviewed Status: Patch Available (was: Open) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch, trunk-6423_v3.2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Open (was: Patch Available) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Assignee: Jimmy Xiang (was: Amitanand Aiyer) Status: Open (was: Patch Available) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423_v2.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Patch Available (was: Open) Addressed Ted's comments. Fixed TestHeapSize. Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Open (was: Patch Available) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423_v2.1.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Patch Available (was: Open) Rebased to trunk latest. Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Jimmy Xiang Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, trunk-6423_v2.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Attachment: trunk-6423.patch Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Amitanand Aiyer Attachments: trunk-6423.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores
[ https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6423: --- Status: Patch Available (was: Open) Writes should not block reads on blocking updates to memstores -- Key: HBASE-6423 URL: https://issues.apache.org/jira/browse/HBASE-6423 Project: HBase Issue Type: Bug Reporter: Karthik Ranganathan Assignee: Amitanand Aiyer Attachments: trunk-6423.patch We have a big data use case where we turn off WAL and have a ton of reads and writes. We found that: 1. flushing a memstore takes a while (GZIP compression) 2. incoming writes cause the new memstore to grow in an unbounded fashion 3. this triggers blocking memstore updates 4. in turn, this causes all the RPC handler threads to block on writes to that memstore 5. we are not able to read during this time as RPC handlers are blocked At a higher level, we should not hold up the RPC threads while blocking updates, and we should build in some sort of rate control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
