Author: ab
Date: Fri Mar  2 00:04:31 2007
New Revision: 513652

URL: http://svn.apache.org/viewvc?view=rev&rev=513652
Log:
HADOOP-1046 - clean up tmp files belonging to stale incomplete blocks.

Modified:
    lucene/hadoop/trunk/CHANGES.txt
    lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java

Modified: lucene/hadoop/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/lucene/hadoop/trunk/CHANGES.txt?view=diff&rev=513652&r1=513651&r2=513652
==============================================================================
--- lucene/hadoop/trunk/CHANGES.txt (original)
+++ lucene/hadoop/trunk/CHANGES.txt Fri Mar  2 00:04:31 2007
@@ -177,6 +177,7 @@
 53. HADOOP-1037.  Fix bin/slaves.sh, which currently only works with
     /bin/bash, to specify /bin/bash rather than /bin/sh.  (cutting)
 
+54. HADOOP-1046. Clean up tmp from partially received stale block files. (ab)
 
 Release 0.11.2 - 2007-02-16
 

Modified: lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java
URL: 
http://svn.apache.org/viewvc/lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java?view=diff&rev=513652&r1=513651&r2=513652
==============================================================================
--- lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java (original)
+++ lucene/hadoop/trunk/src/java/org/apache/hadoop/dfs/FSDataset.java Fri Mar  
2 00:04:31 2007
@@ -435,8 +435,18 @@
             // Is it already in the create process?
             //
             if (ongoingCreates.containsKey(b)) {
+              // check how old is the temp file - wait 1 hour
+              File tmp = (File)ongoingCreates.get(b);
+              if ((System.currentTimeMillis() - tmp.lastModified()) < 3600 * 
1000) {
                 throw new IOException("Block " + b +
                     " has already been started (though not completed), and 
thus cannot be created.");
+              } else {
+                // stale temp file - remove
+                if (!tmp.delete()) {
+                  throw new IOException("Can't write the block - unable to 
remove stale temp file " + tmp);
+                }
+                ongoingCreates.remove(b);
+              }
             }
             FSVolume v = null;
             synchronized ( volumes ) {


Reply via email to