Hadoop-Mapreduce-trunk - Build # 1242 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1242/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 29713 lines...]
Running org.apache.hadoop.mapreduce.lib.partition.TestBinaryPartitioner
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.379 sec
Running org.apache.hadoop.mapreduce.lib.partition.TestMRKeyFieldBasedPartitioner
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.35 sec
Running org.apache.hadoop.mapreduce.TestChild
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.723 sec
Running org.apache.hadoop.mapreduce.filecache.TestURIFragments
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.06 sec
Running org.apache.hadoop.mapreduce.TestMapReduce
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.446 sec

Results :

Failed tests:   testMR(org.apache.hadoop.mapred.TestClusterMRNotification): 
expected:6 but was:4

Tests run: 438, Failures: 1, Errors: 0, Skipped: 15

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] hadoop-mapreduce-client ... SUCCESS [2.070s]
[INFO] hadoop-mapreduce-client-core .. SUCCESS [16.524s]
[INFO] hadoop-mapreduce-client-common  SUCCESS [21.823s]
[INFO] hadoop-mapreduce-client-shuffle ... SUCCESS [1.071s]
[INFO] hadoop-mapreduce-client-app ... SUCCESS [4:25.751s]
[INFO] hadoop-mapreduce-client-hs  SUCCESS [1:10.243s]
[INFO] hadoop-mapreduce-client-jobclient . FAILURE [40:00.302s]
[INFO] Apache Hadoop MapReduce Examples .. SKIPPED
[INFO] hadoop-mapreduce .. SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 46:18.340s
[INFO] Finished at: Wed Oct 31 14:01:35 UTC 2012
[INFO] Final Memory: 21M/124M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on 
project hadoop-mapreduce-client-jobclient: ExecutionException; nested exception 
is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The 
forked VM terminated without saying properly goodbye. VM crash or System.exit 
called ? - [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn goals -rf :hadoop-mapreduce-client-jobclient
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Updating HDFS-4129
Updating HADOOP-8994
Updating HDFS-3573
Updating HDFS-3789
Updating HADOOP-8986
Updating HDFS-3916
Updating HDFS-3695
Updating HDFS-3809
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Created] (MAPREDUCE-4760) Make a version of Counters that is composit for the job and stores the counter values in arrays.

[Robert Joseph Evans (JIRA)]

Robert Joseph Evans created MAPREDUCE-4760:
--

 Summary: Make a version of Counters that is composit for the job 
and stores the counter values in arrays.
 Key: MAPREDUCE-4760
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4760
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Affects Versions: 0.23.4, 2.0.2-alpha
Reporter: Robert Joseph Evans
Priority: Minor


String interning reduced the size of counters a lot.  After that and the fix 
for a memory leak in the IPC server a job with 2 map tasks and 3000 
reducers takes about 200MB to store the state of all of the tasks.  Looking at 
a memory dump of the AM each task attempt has a pointer to a Counters object 
that is about 2kb to 3kb in size.  That means Counters account for about 56MB 
of the 200MB of state.  This job only had about 40 task counters in it.  Each 
counter stores a long value so if we stored them in a long[] instead we should 
only be taking up 7MB.

Also assuming that some of the counters only appear in a map task or a reduce 
task we should be able to have one CompositCounters for map tasks and one for 
reduce tasks so it would reduce the size even further. 

NOTE: without this change I would expect to be able to run a 100,000 task job 
in the default 1024MB AM heap (875MB/200MB * 2300) I reserved 150MB for IPC 
buffers and event data.  With this change we could expect to run about 130,000 
tasks (875MB/150MB * 2300).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-4761) MR AM doesn't handle exceptions from RM correctly

[Vinod Kumar Vavilapalli (JIRA)]

Vinod Kumar Vavilapalli created MAPREDUCE-4761:
--

 Summary: MR AM doesn't handle exceptions from RM correctly
 Key: MAPREDUCE-4761
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4761
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: applicationmaster
Reporter: Vinod Kumar Vavilapalli


The MR AM currently ignores exceptions upto a certain count irrespective of the 
failure type.

Exception handling on the MR AM side is totally broken, it doesn't seem to 
shutdown the AM on getting critical exceptions or reboot commands from the RM - 
another bug.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira