[
https://issues.apache.org/jira/browse/HADOOP-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712569#action_12712569
]
Steve Loughran commented on HADOOP-5906:
----------------------------------------
stack trace. Presumably we just need to set the JVM parameters up right, but
what are the right values?
{code}
[junit] Tests run: 4, Failures: 1, Errors: 0, Time elapsed: 13.164 sec
[junit] Test org.apache.hadoop.streaming.TestStreamingExitStatus FAILED
09/05/24 12:00:11 INFO streaming.PipeMapRed: R/W/S=1/0/0 in:NA [rec/s] out:NA
[rec/s]
09/05/24 12:00:11 INFO streaming.PipeMapRed: Records R/W=1/1
09/05/24 12:00:11 INFO streaming.PipeMapRed: MRErrorThread done
09/05/24 12:00:11 WARN mapred.LocalJobRunner: job_local_0002
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed
with code 1
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:366)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:568)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:360)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:11 INFO streaming.StreamJob: map 0% reduce 0%
09/05/24 12:00:11 INFO streaming.StreamJob: Job running in-process (local
Hadoop)
09/05/24 12:00:11 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:11 INFO streaming.StreamJob: killJob...
09/05/24 12:00:11 WARN streaming.StreamJob: -jobconf option is deprecated,
please use -D instead.
09/05/24 12:00:11 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
09/05/24 12:00:11 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
09/05/24 12:00:12 WARN mapred.JobClient: No job jar file set. User classes may
not be found. See JobConf(Class) or JobConf#setJar(String).
09/05/24 12:00:12 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:13 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:13 INFO streaming.StreamJob: getLocalDirs():
[/Users/slo/Java/Apache/hadoop-branches/hadoop-5847/build/contrib/streaming/test/mapred/local]
09/05/24 12:00:13 INFO streaming.StreamJob: Running job: job_local_0003
09/05/24 12:00:13 INFO streaming.StreamJob: Job running in-process (local
Hadoop)
09/05/24 12:00:13 INFO mapred.MapTask: numReduceTasks: 1
09/05/24 12:00:13 INFO mapred.MapTask: io.sort.mb = 100
09/05/24 12:00:13 WARN mapred.LocalJobRunner: job_local_0003
java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:689)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:14 INFO streaming.StreamJob: map 0% reduce 0%
09/05/24 12:00:14 INFO streaming.StreamJob: Job running in-process (local
Hadoop)
09/05/24 12:00:14 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:14 INFO streaming.StreamJob: killJob...
09/05/24 12:00:14 WARN streaming.StreamJob: -jobconf option is deprecated,
please use -D instead.
09/05/24 12:00:14 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
09/05/24 12:00:14 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
09/05/24 12:00:14 WARN mapred.JobClient: No job jar file set. User classes may
not be found. See JobConf(Class) or JobConf#setJar(String).
09/05/24 12:00:14 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:15 INFO streaming.StreamJob: getLocalDirs():
[/Users/slo/Java/Apache/hadoop-branches/hadoop-5847/build/contrib/streaming/test/mapred/local]
09/05/24 12:00:15 INFO streaming.StreamJob: Running job: job_local_0004
09/05/24 12:00:15 INFO streaming.StreamJob: Job running in-process (local
Hadoop)
09/05/24 12:00:15 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:15 INFO mapred.MapTask: numReduceTasks: 1
09/05/24 12:00:15 INFO mapred.MapTask: io.sort.mb = 100
09/05/24 12:00:16 WARN mapred.LocalJobRunner: job_local_0004
java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:689)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:16 INFO streaming.StreamJob: map 0% reduce 0%
09/05/24 12:00:16 INFO streaming.StreamJob: Job running in-process (local
Hadoop)
09/05/24 12:00:16 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:16 INFO streaming.StreamJob: killJob...
{code}
> Stream test TestStreamingExitStatus fails with Out of Memory
> ------------------------------------------------------------
>
> Key: HADOOP-5906
> URL: https://issues.apache.org/jira/browse/HADOOP-5906
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.21.0
> Environment: OS/X, 64-bit x86 imac, 4GB RAM.
> Reporter: Steve Loughran
> Priority: Minor
>
> contrib/streaming tests are failing a test with an Out of Memory error on an
> OS/X Mac -same problem does not surface on Linux.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
