[
https://issues.apache.org/jira/browse/HBASE-6372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427830#comment-13427830
]
Zhihong Ted Yu commented on HBASE-6372:
---------------------------------------
Lars did some work in HBASE-5659 to improve TestAtomicOperation
>From
>https://builds.apache.org/job/PreCommit-HBASE-Build/2459//testReport/org.apache.hadoop.hbase.regionserver/TestAtomicOperation/testMultiRowMutationMultiThreads/:
{code}
2012-07-31 09:22:36,252 DEBUG [Thread-124]
regionserver.TestAtomicOperation$2(398):
[rowA/colfamily11:qual1/4069/Put/vlen=6/ts=4278,
rowB/colfamily11:qual1/4073/Put/vlen=6/ts=0]
Exception in thread "Thread-124" junit.framework.AssertionFailedError at
junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.fail(Assert.java:56)
at
org.apache.hadoop.hbase.regionserver.TestAtomicOperation$2.run(TestAtomicOperation.java:400)
{code}
This corresponds to the following:
{code}
if (r.size() != 1) {
LOG.debug(r);
failures.incrementAndGet();
fail();
{code}
Maybe Lars has better idea about the above.
About the patch:
{code}
+ + " -Dhbase.mapreduce.export.batch=10");
{code}
Please use EXPORT_BATCHING in place of duplicate config parameter name above.
nit:
{code}
+ final static String EXPORT_BATCHING="hbase.mapreduce.export.batch";
{code}
Introduce spaces around = sign.
> Add scanner batching to Export job
> ----------------------------------
>
> Key: HBASE-6372
> URL: https://issues.apache.org/jira/browse/HBASE-6372
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.2
> Reporter: Lars George
> Assignee: Shengsheng Huang
> Priority: Minor
> Labels: newbie
> Attachments: HBASE-6372.2.patch, HBASE-6372.patch
>
>
> When a single row is too large for the RS heap then an OOME can take out the
> entire RS. Setting scanner batching in custom scans helps avoiding this
> scenario, but for the supplied Export job this is not set.
> Similar to HBASE-3421 we can set the batching to a low number - or if needed
> make it a command line option.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
