Craig Macdonald created MAPREDUCE-4776:
--
Summary: Reducer Channels
Key: MAPREDUCE-4776
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4776
Project: Hadoop Map/Reduce
Issue Type: Ne
Hi Jiwei,
In trunk (i.e. MR2), the completion events selection + scheduling
logic lies under class EventFetcher's getMapCompletionEvents() method,
as viewable at
http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1249/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 30816 lines...]
Tests run: 4, Failures: 0, Errors: 0
Jiwei,
I think you could use that knowledge to launch reducers closer to the map
output, but I am not sure that it would make much difference. It may even
slow things down. It is a question of several things
1) Can we get enough map tasks close to one another that it will make a
difference?
2) D
Sandy Ryza created MAPREDUCE-4778:
-
Summary: Fair scheduler event log is only written if directory
exists on HDFS
Key: MAPREDUCE-4778
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4778
Project:
Amir Sanjar created MAPREDUCE-4779:
--
Summary: Unit test TestJobTrackerSafeMode fails with ant 1.8.3+
Key: MAPREDUCE-4779
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4779
Project: Hadoop Map/
Chris Nauroth created MAPREDUCE-4780:
Summary: MapReduce distribution build fails on Windows
Key: MAPREDUCE-4780
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4780
Project: Hadoop Map/Reduce
Hi Bobby,
Thank you a lot for your suggestions. My whole idea is to minimize the
aggregate network bandwidth during Shuffle Phase, that is, to limit the
hops to minimum when transmitting data from map node to reduce node.
Usually, Partitioner creates skews that the JobTracker allocates different
a
