Author: ddas
Date: Wed Dec 26 00:39:56 2007
New Revision: 606855
URL: http://svn.apache.org/viewvc?rev=606855&view=rev
Log:
Merge -r 606853:606854 from trunk to 0.15 branch. Fixes HADOOP-2486.
Modified:
lucene/hadoop/branches/branch-0.15/CHANGES.txt
lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java
Modified: lucene/hadoop/branches/branch-0.15/CHANGES.txt
URL:
http://svn.apache.org/viewvc/lucene/hadoop/branches/branch-0.15/CHANGES.txt?rev=606855&r1=606854&r2=606855&view=diff
==============================================================================
--- lucene/hadoop/branches/branch-0.15/CHANGES.txt (original)
+++ lucene/hadoop/branches/branch-0.15/CHANGES.txt Wed Dec 26 00:39:56 2007
@@ -39,6 +39,10 @@
round-robin disk selections randomly. This helps in spreading data across
multiple partitions much better. (acmurhty)
+ HADOOP-2486. When the list of files from the InMemoryFileSystem is obtained
+ for merging, this patch will ensure that only those files whose checksums
+ have also got created (renamed) are returned. (ddas)
+
IMPROVEMENTS
HADOOP-2160. Remove project-level, non-user documentation from
Modified:
lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java
URL:
http://svn.apache.org/viewvc/lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java?rev=606855&r1=606854&r2=606855&view=diff
==============================================================================
---
lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java
(original)
+++
lucene/hadoop/branches/branch-0.15/src/java/org/apache/hadoop/mapred/ReduceTask.java
Wed Dec 26 00:39:56 2007
@@ -1290,8 +1290,11 @@
public void run() {
LOG.info(reduceTask.getTaskId() + " Thread started: " + getName());
try {
+ Path[] inMemClosedFiles;
//initiate merge
- Path[] inMemClosedFiles = inMemFileSys.getFiles(MAP_OUTPUT_FILTER);
+ synchronized (ReduceTask.this) {
+ inMemClosedFiles = inMemFileSys.getFiles(MAP_OUTPUT_FILTER);
+ }
//Note that the above Path[] could be of length 0 if all copies are
//in flight. So we make sure that we have some 'closed' map
//output files to merge to get the benefit of in-memory merge
