[
https://issues.apache.org/jira/browse/HBASE-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624173#comment-13624173
]
Jonathan Hsieh edited comment on HBASE-8258 at 4/5/13 11:32 PM:
----------------------------------------------------------------
Hm.. On my second try, it looks like hadoop1 has got some failures (essentiall
same as hadoopqa bot), but the hadoop2 build (on an internal jenkins) is good.
On my first run against hadoop2 I had problems with vmem overuse:
{code}
2013-04-05 14:57:45,983 WARN [AsyncDispatcher event handler]
resourcemanager.RMAuditLogger(255): USER=jenkins OPERATION=Application
Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App
failed with state: FAILED PERMISSIONS=Application
application_1365199040071_0001 failed 1 times due to AM Container for
appattempt_1365199040071_0001_000001 exited with exitCode: 143 due to:
Container [pid=24860,containerID=container_1365199040071_0001_01_000001] is
running beyond virtual memory limits. Current usage: 269.3mb of 2.0gb physical
memory used; 6.3gb of 4.2gb virtual memory used. Killing container.
{code}
I've uncommented the pmem/vmem lines in the patch to make hadoop2 happy.
{code}
// Tests were failing because this process used 6GB of virtual memory and
was getting killed.
// we up the VM usable so that processes don't get killed.
conf.setInt("yarn.nodemanager.resource.memory-mb", 8 * 1024);
conf.setFloat("yarn.nodemanager.vmem-pmem-ratio", 8.0f);
{code}
Digging into the hadoop 1 issues now.
was (Author: jmhsieh):
Hm.. On my second try, it looks like hadoop1 has got some failures, but the
hadoop2 build is good.
On my first run against hadoop2 I had problems with vmem overuse:
{code}
2013-04-05 14:57:45,983 WARN [AsyncDispatcher event handler]
resourcemanager.RMAuditLogger(255): USER=jenkins OPERATION=Application
Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App
failed with state: FAILED PERMISSIONS=Application
application_1365199040071_0001 failed 1 times due to AM Container for
appattempt_1365199040071_0001_000001 exited with exitCode: 143 due to:
Container [pid=24860,containerID=container_1365199040071_0001_01_000001] is
running beyond virtual memory limits. Current usage: 269.3mb of 2.0gb physical
memory used; 6.3gb of 4.2gb virtual memory used. Killing container.
{code}
I've uncommented the pmem/vmem lines in the patch to make hadoop2 happy.
{code}
// Tests were failing because this process used 6GB of virtual memory and
was getting killed.
// we up the VM usable so that processes don't get killed.
conf.setInt("yarn.nodemanager.resource.memory-mb", 8 * 1024);
conf.setFloat("yarn.nodemanager.vmem-pmem-ratio", 8.0f);
{code}
Digging into the hadoop 1 issues now.
> Make mapreduce tests pass on hadoop2
> ------------------------------------
>
> Key: HBASE-8258
> URL: https://issues.apache.org/jira/browse/HBASE-8258
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Reporter: stack
> Priority: Blocker
> Fix For: 0.95.1
>
> Attachments: 8258-plain.txt, 8258-v1-hadoop-2.0.txt,
> 8258-v2-hadoop-2.0.txt, 8258-v4-hadoop-2.0.txt, hbase-8258-simple.patch
>
>
> HBASE-7904 was a first attempt at making this work but it got lost in the
> weeds.
> This is a new attempt at making hbase mapreduce jobs run on hadoop2 (w/o
> breaking mapreduce on hadoop1)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
