[
https://issues.apache.org/jira/browse/MAPREDUCE-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167355#comment-13167355
]
Subroto Sanyal commented on MAPREDUCE-3319:
-------------------------------------------
Hi Matt,
I ran the *multifilewc* in trunk (mrv2) and it runs fine.
Output of the run:
{noformat}
2011-12-12 10:35:16,569 INFO mapreduce.Job (Job.java:monitorAndPrintJob(1227))
- map 100% reduce 100%
2011-12-12 10:35:16,574 INFO mapreduce.Job (Job.java:monitorAndPrintJob(1238))
- Job job_1322715021135_0002 completed successfully
2011-12-12 10:35:16,743 INFO mapreduce.Job (Job.java:monitorAndPrintJob(1245))
- Counters: 43
File System Counters
FILE: BYTES_READ=97054
FILE: BYTES_WRITTEN=190175
FILE: READ_OPS=0
FILE: LARGE_READ_OPS=0
FILE: WRITE_OPS=0
HDFS: BYTES_READ=1165096553
HDFS: BYTES_WRITTEN=4394
HDFS: READ_OPS=32
HDFS: LARGE_READ_OPS=0
HDFS: WRITE_OPS=4
org.apache.hadoop.mapreduce.JobCounter
TOTAL_LAUNCHED_MAPS=1
TOTAL_LAUNCHED_REDUCES=1
OTHER_LOCAL_MAPS=1
SLOTS_MILLIS_MAPS=67860
SLOTS_MILLIS_REDUCES=65602
org.apache.hadoop.mapreduce.TaskCounter
MAP_INPUT_RECORDS=265824
MAP_OUTPUT_RECORDS=531648
MAP_OUTPUT_BYTES=1166967360
MAP_OUTPUT_MATERIALIZED_BYTES=4402
SPLIT_RAW_BYTES=989
COMBINE_INPUT_RECORDS=531678
COMBINE_OUTPUT_RECORDS=32
REDUCE_INPUT_GROUPS=2
REDUCE_SHUFFLE_BYTES=4402
REDUCE_INPUT_RECORDS=2
REDUCE_OUTPUT_RECORDS=2
SPILLED_RECORDS=46
SHUFFLED_MAPS=1
FAILED_SHUFFLE=0
MERGED_MAP_OUTPUTS=1
GC_TIME_MILLIS=2566
CPU_MILLISECONDS=69150
PHYSICAL_MEMORY_BYTES=200130560
VIRTUAL_MEMORY_BYTES=756015104
COMMITTED_HEAP_BYTES=137433088
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter
BYTES_READ=0
org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter
BYTES_WRITTEN=4394
{noformat}
I think the problem is resolved in trunk already.
> multifilewc from hadoop examples seems to be broken in 0.20.205.0
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-3319
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3319
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: examples
> Affects Versions: 0.20.205.0
> Reporter: Roman Shaposhnik
> Assignee: Subroto Sanyal
> Priority: Blocker
> Labels: bigtop
> Fix For: 1.1.0
>
> Attachments: MAPREDUCE-3319.patch
>
>
> {noformat}
> /usr/lib/hadoop/bin/hadoop jar
> /usr/lib/hadoop/hadoop-examples-0.20.205.0.22.jar multifilewc examples/text
> examples-output/multifilewc
> 11/10/31 16:50:26 INFO mapred.FileInputFormat: Total input paths to process :
> 2
> 11/10/31 16:50:26 INFO mapred.JobClient: Running job: job_201110311350_0220
> 11/10/31 16:50:27 INFO mapred.JobClient: map 0% reduce 0%
> 11/10/31 16:50:42 INFO mapred.JobClient: Task Id :
> attempt_201110311350_0220_m_000000_0, Status : FAILED
> java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
> to org.apache.hadoop.io.LongWritable
> at
> org.apache.hadoop.mapred.lib.LongSumReducer.reduce(LongSumReducer.java:44)
> at
> org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(Task.java:1431)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1436)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:437)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
