[
https://issues.apache.org/jira/browse/HBASE-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451445#comment-15451445
]
Devaraj Das edited comment on HBASE-16255 at 8/31/16 7:30 AM:
--------------------------------------------------------------
Sorry, I was off on my previous guess at the issue. I dug deeper and this seems
like a yarn issue to do with the minimum memory for the containers.
The default heap size (-Xmx200m) for the tasks was too low ([~dspivak],
curious, if you ran other IT tests that do mapreduce, and did/didn't see this
issue)
I added the following in yarn-site.xml:
{noformat}
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>1024</value>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>1024</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>1024</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapred.map.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapred.reduce.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
{noformat}
And in mapred-site.xml, added the following:
{noformat}
<property>
<name>mapred.map.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapred.reduce.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
{noformat}
At the time of this writing, the test was still running (it proceeded beyond
your failure point [~dspivak]). Fingers crossed.
was (Author: devaraj):
Sorry, I was off on my previous guess at the issue. I dug deeper and this seems
like a yarn issue to do with the minimum memory for the containers.
The default heap size (-Xmx200m) for the tasks was too low ([~dspivak],
curious, if you ran other IT tests that do mapreduce, and did/didn't see this
issue)
I added the following in yarn-site.xml:
{noformat}
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>1024</value>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>1024</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>1024</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapred.map.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapred.reduce.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
</configuration>
{noformat}
And in mapred-site.xml, added the following:
{noformat}
<property>
<name>mapred.map.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapred.reduce.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
{noformat}
At the time of this writing, the test was still running (it proceeded beyond
your failure point [~dspivak]). Fingers crossed.
> Backup/Restore IT
> -----------------
>
> Key: HBASE-16255
> URL: https://issues.apache.org/jira/browse/HBASE-16255
> Project: HBase
> Issue Type: Task
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Labels: backup
> Attachments: 16255-addendum.3.txt, 16255.addendum, 16255.addendum2,
> 16255.addendum4, 16255.addendum5, 16255.addendum6, HBASE-16255-v1.patch,
> HBASE-16255-v2.patch, HBASE-16255-v3.patch, HBASE-16255-v4.patch,
> HBASE-16255-v5.patch, HBASE-16255-v6.patch, backup-it-7912-8-30.out,
> backup-it-8-30.out, backup-it-success.out
>
>
> Integration test for backup restore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
