[jira] [Commented] (YARN-449) HBase test failures when running against Hadoop 2

2013-04-01 Thread Ted Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618891#comment-13618891
 ] 

Ted Yu commented on YARN-449:
-

Based on 
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/49/testReport/
 and http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/50/console, there was no 
mapreduce job failure after HBASE-7904 was checked in.

This JIRA can be resolved.

 HBase test failures when running against Hadoop 2
 -

 Key: YARN-449
 URL: https://issues.apache.org/jira/browse/YARN-449
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Siddharth Seth
Priority: Blocker
 Attachments: 7904-v5.txt, hbase-7904-v3.txt, 
 hbase-TestHFileOutputFormat-wip.txt, hbase-TestingUtility-wip.txt, 
 minimr_randomdir-branch2.txt


 Post YARN-429, unit tests for HBase continue to fail since the classpath for 
 the MRAppMaster is not being set correctly.
 Reverting YARN-129 may fix this, but I'm not sure that's the correct 
 solution. My guess is, as Alexandro pointed out in YARN-129, maven 
 classloader magic is messing up java.class.path.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-449) HBase test failures when running against Hadoop 2

2013-04-01 Thread Vinod Kumar Vavilapalli (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619301#comment-13619301
 ] 

Vinod Kumar Vavilapalli commented on YARN-449:
--

bq. I would not build any argument or deduction on the mess that went on over 
in hbase-7904.
That's good news. Really. Any constructive feedback to avoid the 'mess' in 
future is deeply appreciated though.

Sid and Ted, thanks for being on top of this!

 HBase test failures when running against Hadoop 2
 -

 Key: YARN-449
 URL: https://issues.apache.org/jira/browse/YARN-449
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Siddharth Seth
Priority: Blocker
 Fix For: 2.0.4-alpha

 Attachments: 7904-v5.txt, hbase-7904-v3.txt, 
 hbase-TestHFileOutputFormat-wip.txt, hbase-TestingUtility-wip.txt, 
 minimr_randomdir-branch2.txt


 Post YARN-429, unit tests for HBase continue to fail since the classpath for 
 the MRAppMaster is not being set correctly.
 Reverting YARN-129 may fix this, but I'm not sure that's the correct 
 solution. My guess is, as Alexandro pointed out in YARN-129, maven 
 classloader magic is messing up java.class.path.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-449) HBase test failures when running against Hadoop 2

2013-03-13 Thread Ted Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13601356#comment-13601356
 ] 

Ted Yu commented on YARN-449:
-

Here is OS for Hadoop QA machine:
Linux asf002.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul 20 
17:42:25 UTC 2011 x86_64 GNU/Linux

Here is OS for the machine where I ran unit test manually:
Linux ygridcore.net 2.6.32-220.23.1.el6.YAHOO.20120713.x86_64 #1 SMP Fri Jul 13 
11:40:51 CDT 2012 x86_64 x86_64 x86_64 GNU/Linux

 HBase test failures when running against Hadoop 2
 -

 Key: YARN-449
 URL: https://issues.apache.org/jira/browse/YARN-449
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Siddharth Seth
Priority: Blocker
 Attachments: 7904-v5.txt, hbase-7904-v3.txt, 
 hbase-TestHFileOutputFormat-wip.txt, hbase-TestingUtility-wip.txt, 
 minimr_randomdir-branch2.txt


 Post YARN-429, unit tests for HBase continue to fail since the classpath for 
 the MRAppMaster is not being set correctly.
 Reverting YARN-129 may fix this, but I'm not sure that's the correct 
 solution. My guess is, as Alexandro pointed out in YARN-129, maven 
 classloader magic is messing up java.class.path.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-449) HBase test failures when running against Hadoop 2

2013-03-12 Thread Ted Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13600632#comment-13600632
 ] 

Ted Yu commented on YARN-449:
-

More from TEST-org.apache.hadoop.hbase.mapreduce.TestRowCounter.xml:
{code}
2013-03-12 18:53:39,274 WARN  [Container Monitor] 
monitor.ContainersMonitorImpl(298): Process tree for container: 
container_1363114400920_0001_01_01 has processes older than 1 iteration 
running over the configured limit. Limit=4509715456, current usage = 7007866880
2013-03-12 18:53:39,275 WARN  [Container Monitor] 
monitor.ContainersMonitorImpl$MonitoringThread(444): Container 
[pid=8438,containerID=container_1363114400920_0001_01_01] is running beyond 
virtual memory limits. Current usage: 217.9 MB of 2 GB physical memory used; 
6.5 GB of 4.2 GB virtual memory used. Killing container.
Dump of the process-tree for container_1363114400920_0001_01_01 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) 
SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 8438 7023 8438 8438 (bash) 1 0 108650496 310 /bin/bash -c 
/usr/lib/jvm/java-1.6.0-sun-1.6.0.37.x86_64/bin/java 
-Dlog4j.configuration=container-log4j.properties 
-Dyarn.app.mapreduce.container.log.dir=/homes/hortonzy/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniMRCluster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_1363114400920_0001_01_01
 -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA  
-Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 
1gt;/homes/hortonzy/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniMRCluster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_1363114400920_0001_01_01/stdout
 
2gt;/homes/hortonzy/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniMRCluster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_1363114400920_0001_01_01/stderr
|- 8461 8438 8438 8438 (java) 688 34 6899216384 55478 
/usr/lib/jvm/java-1.6.0-sun-1.6.0.37.x86_64/bin/java 
-Dlog4j.configuration=container-log4j.properties 
-Dyarn.app.mapreduce.container.log.dir=/homes/hortonzy/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster_1035429065/org.apache.hadoop.mapred.MiniMRCluster_1035429065-logDir-nm-1_2/application_1363114400920_0001/container_1363114400920_0001_01_01
 -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA 
-Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster
{code}
Note that 1024m was specified for -Xmx

 HBase test failures when running against Hadoop 2
 -

 Key: YARN-449
 URL: https://issues.apache.org/jira/browse/YARN-449
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Siddharth Seth
Priority: Blocker
 Attachments: 7904-v5.txt, hbase-7904-v3.txt, 
 hbase-TestHFileOutputFormat-wip.txt, hbase-TestingUtility-wip.txt, 
 minimr_randomdir-branch2.txt


 Post YARN-429, unit tests for HBase continue to fail since the classpath for 
 the MRAppMaster is not being set correctly.
 Reverting YARN-129 may fix this, but I'm not sure that's the correct 
 solution. My guess is, as Alexandro pointed out in YARN-129, maven 
 classloader magic is messing up java.class.path.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-449) HBase test failures when running against Hadoop 2

2013-03-12 Thread Ted Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13600650#comment-13600650
 ] 

Ted Yu commented on YARN-449:
-

Here is sample content for /proc/PID/stat
{code}
30873 (sshd) S 30869 30869 30869 0 -1 4202816 360 0 0 0 47 56 0 0 20 0 1 0 
741791881 117960704 516 18446744073709551615 1 1 0 0 0 0 0 4096 65536 
18446744073709551615 0 0 17 2 0 0 0 0 0
{code}
Here is the regex used to parse the stat file:
{code}
  private static final Pattern PROCFS_STAT_FILE_FORMAT = Pattern .compile(
^([0-9-]+)\\s([^\\s]+)\\s[^\\s]\\s([0-9-]+)\\s([0-9-]+)\\s([0-9-]+)\\s +
([0-9-]+\\s){7}([0-9]+)\\s([0-9]+)\\s([0-9-]+\\s){7}([0-9]+)\\s([0-9]+) +
(\\s[0-9-]+){15});
{code}

 HBase test failures when running against Hadoop 2
 -

 Key: YARN-449
 URL: https://issues.apache.org/jira/browse/YARN-449
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Siddharth Seth
Priority: Blocker
 Attachments: 7904-v5.txt, hbase-7904-v3.txt, 
 hbase-TestHFileOutputFormat-wip.txt, hbase-TestingUtility-wip.txt, 
 minimr_randomdir-branch2.txt


 Post YARN-429, unit tests for HBase continue to fail since the classpath for 
 the MRAppMaster is not being set correctly.
 Reverting YARN-129 may fix this, but I'm not sure that's the correct 
 solution. My guess is, as Alexandro pointed out in YARN-129, maven 
 classloader magic is messing up java.class.path.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira