Repository: flume Updated Branches: refs/heads/trunk 209c1e254 -> ba64b1267
FLUME-2950. ReliableSpoolingFileEventReader.rollCurrentFile is broken FLUME-2939 introduced a Java 7 API (nio.file) to ReliableSpoolingFileEventReader. However, the committed patch breaks rollCurrentFile() on Windows machines. File rename might be not atomic, in such cases, and under the assumption that the rolled file and the current file store the same data, Flume only logs a warning and does not assume that the user broke the contract that the spooling directory source demands. In order to determine whether the files store the same data, we used the com.google.common.io.Files.equal() method. However, FLUME-2939 uses java.nio.file.Files.isSameFile() which has different semantics. Consequently, this relaxation does not hold anymore. In addition, TestSpoolingFileLineReader.testDestinationExistsAndSameFileWindows fails. This patch brings back the old check using Files.equal(). (Lior Zeno via Mike Percy) Project: http://git-wip-us.apache.org/repos/asf/flume/repo Commit: http://git-wip-us.apache.org/repos/asf/flume/commit/ba64b126 Tree: http://git-wip-us.apache.org/repos/asf/flume/tree/ba64b126 Diff: http://git-wip-us.apache.org/repos/asf/flume/diff/ba64b126 Branch: refs/heads/trunk Commit: ba64b12670f7b2c9ce079f8898f4601689a51def Parents: 209c1e2 Author: Lior Zeno <[email protected]> Authored: Sun Jul 10 15:03:08 2016 -0700 Committer: Mike Percy <[email protected]> Committed: Sun Jul 10 15:17:45 2016 -0700 ---------------------------------------------------------------------- .../apache/flume/client/avro/ReliableSpoolingFileEventReader.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flume/blob/ba64b126/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java ---------------------------------------------------------------------- diff --git a/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java b/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java index ca5308c..01381a5 100644 --- a/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java +++ b/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java @@ -418,7 +418,7 @@ public class ReliableSpoolingFileEventReader implements ReliableEventReader { * file was already rolled but the rename was not atomic. If that seems * likely, we let it pass with only a warning. */ - if (Files.isSameFile(currentFile.get().getFile().toPath(), dest.toPath())) { + if (com.google.common.io.Files.equal(currentFile.get().getFile(), dest)) { logger.warn("Completed file " + dest + " already exists, but files match, so continuing."); boolean deleted = fileToRoll.delete();
