[jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
[ https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368617#comment-14368617 ] Hudson commented on HBASE-13269: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #863 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/863/]) HBASE-13269 Limit result array preallocation to avoid OOME with large scan caching values (apurtell: rev 8fa491e709770ca73701adeb474807ec87b421af) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Limit result array preallocation to avoid OOME with large scan caching values - Key: HBASE-13269 URL: https://issues.apache.org/jira/browse/HBASE-13269 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.0.1, 0.98.12 Attachments: HBASE-13269-0.98.patch, HBASE-13269-0.98.patch, HBASE-13269-1.0.patch, HBASE-13269-1.0.patch Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver with an OOME due to preallocation of the result array according to this parameter. We should limit the preallocation to some sane value. Definitely affects 0.98 (fix needed to HRegionServer) and 1.0.x (fix needed to RsRPCServices), not sure about later versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
[
https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367715#comment-14367715
]
Hadoop QA commented on HBASE-13269:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12705413/HBASE-13269-1.0.patch
against master branch at commit f9a17edc252a88c5a1a2c7764e3f9f65623e0ced.
ATTACHMENT ID: 12705413
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13296//console
This message is automatically generated.
Limit result array preallocation to avoid OOME with large scan caching values
-
Key: HBASE-13269
URL: https://issues.apache.org/jira/browse/HBASE-13269
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 1.0.1, 0.98.12
Attachments: HBASE-13269-0.98.patch, HBASE-13269-0.98.patch,
HBASE-13269-1.0.patch, HBASE-13269-1.0.patch
Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver
with an OOME due to preallocation of the result array according to this
parameter. We should limit the preallocation to some sane value. Definitely
affects 0.98 (fix needed to HRegionServer) and 1.0.x (fix needed to
RsRPCServices), not sure about later versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
[ https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368272#comment-14368272 ] Hudson commented on HBASE-13269: FAILURE: Integrated in HBase-1.0 #812 (See [https://builds.apache.org/job/HBase-1.0/812/]) HBASE-13269 Limit result array preallocation to avoid OOME with large scan caching values (apurtell: rev d443c7096fced912b8d0cc5c63f9f013762e6122) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java Limit result array preallocation to avoid OOME with large scan caching values - Key: HBASE-13269 URL: https://issues.apache.org/jira/browse/HBASE-13269 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.0.1, 0.98.12 Attachments: HBASE-13269-0.98.patch, HBASE-13269-0.98.patch, HBASE-13269-1.0.patch, HBASE-13269-1.0.patch Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver with an OOME due to preallocation of the result array according to this parameter. We should limit the preallocation to some sane value. Definitely affects 0.98 (fix needed to HRegionServer) and 1.0.x (fix needed to RsRPCServices), not sure about later versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
[ https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368100#comment-14368100 ] Lars Hofhansl commented on HBASE-13269: --- +1 Limit result array preallocation to avoid OOME with large scan caching values - Key: HBASE-13269 URL: https://issues.apache.org/jira/browse/HBASE-13269 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.0.1, 0.98.12 Attachments: HBASE-13269-0.98.patch, HBASE-13269-0.98.patch, HBASE-13269-1.0.patch, HBASE-13269-1.0.patch Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver with an OOME due to preallocation of the result array according to this parameter. We should limit the preallocation to some sane value. Definitely affects 0.98 (fix needed to HRegionServer) and 1.0.x (fix needed to RsRPCServices), not sure about later versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
[ https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368496#comment-14368496 ] Hudson commented on HBASE-13269: FAILURE: Integrated in HBase-0.98 #908 (See [https://builds.apache.org/job/HBase-0.98/908/]) HBASE-13269 Limit result array preallocation to avoid OOME with large scan caching values (apurtell: rev 8fa491e709770ca73701adeb474807ec87b421af) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Limit result array preallocation to avoid OOME with large scan caching values - Key: HBASE-13269 URL: https://issues.apache.org/jira/browse/HBASE-13269 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.0.1, 0.98.12 Attachments: HBASE-13269-0.98.patch, HBASE-13269-0.98.patch, HBASE-13269-1.0.patch, HBASE-13269-1.0.patch Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver with an OOME due to preallocation of the result array according to this parameter. We should limit the preallocation to some sane value. Definitely affects 0.98 (fix needed to HRegionServer) and 1.0.x (fix needed to RsRPCServices), not sure about later versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
[ https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367681#comment-14367681 ] Lars Hofhansl commented on HBASE-13269: --- 100 is fine. In most cases we'll have the limit at infinity going forward (since limit is enforced by size). When a limit is specified it is likely going to be small. Limit result array preallocation to avoid OOME with large scan caching values - Key: HBASE-13269 URL: https://issues.apache.org/jira/browse/HBASE-13269 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.0.1, 0.98.12 Attachments: HBASE-13269-0.98.patch, HBASE-13269-1.0.patch Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver with an OOME due to preallocation of the result array according to this parameter. We should limit the preallocation to some sane value. Definitely affects 0.98 (fix needed to HRegionServer) and 1.0.x (fix needed to RsRPCServices), not sure about later versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
[ https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366180#comment-14366180 ] Andrew Purtell commented on HBASE-13269: bq. otherwise it may just be best to remove the preallocation all together. This is exactly what I was thinking too and I'm testing out a patch now (although almost certainly every test will pass) Limit result array preallocation to avoid OOME with large scan caching values - Key: HBASE-13269 URL: https://issues.apache.org/jira/browse/HBASE-13269 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.0.1, 0.98.12 Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver with an OOME due to preallocation of the result array according to this parameter. We should limit the preallocation to some sane value. Definitely affects 0.98 (fix needed to HRegionServer) and 1.0.x (fix needed to RsRPCServices), not sure about later versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
[ https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366142#comment-14366142 ] Jonathan Lawlor commented on HBASE-13269: - After a quick discussion in HBASE-11544 the preallocation was removed because of this issue (there didn't seem to be any way to accurately estimate the number of Results that will be returned since the caching and result size limits work together). This means that in branch-1+ there is no preallocation (this change should have been backported further to prevent the OOM in earlier versions) If a more sophisticated solution is found here, then that change could be added to branch-1+ (HBASE-11544 is only in branch-1+) otherwise it may just be best to remove the preallocation all together. Limit result array preallocation to avoid OOME with large scan caching values - Key: HBASE-13269 URL: https://issues.apache.org/jira/browse/HBASE-13269 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.0.1, 0.98.12 Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver with an OOME due to preallocation of the result array according to this parameter. We should limit the preallocation to some sane value. Definitely affects 0.98 (fix needed to HRegionServer) and 1.0.x (fix needed to RsRPCServices), not sure about later versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
