[
https://issues.apache.org/jira/browse/FLUME-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983967#comment-13983967
]
Hudson commented on FLUME-2357:
-------------------------------
SUCCESS: Integrated in flume-trunk #632 (See
[https://builds.apache.org/job/flume-trunk/632/])
FLUME-2357. HDFS sink should retry closing files that previously had close
errors (mpercy:
http://git-wip-us.apache.org/repos/asf/flume/repo/?p=flume.git&a=commit&h=a94594dd2c5cb980bc6f82b1fa606a922986569e)
*
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFsDataOutputStreamCloseRetryWrapper.java
*
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFsDataOutputStream.java
*
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java
*
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/AbstractHDFSWriter.java
*
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java
*
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java
*
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
*
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFileSystemCloseRetryWrapper.java
*
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
*
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFileSystem.java
*
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockDataStream.java
*
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java
* flume-ng-doc/sphinx/FlumeUserGuide.rst
*
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java
*
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
> HDFS sink should retry closing files that previously had close errors
> ---------------------------------------------------------------------
>
> Key: FLUME-2357
> URL: https://issues.apache.org/jira/browse/FLUME-2357
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.4.0
> Reporter: Patrick Dvorak
> Assignee: Hari Shreedharan
> Fix For: v1.5.0
>
> Attachments: FLUME-2357-1.patch, FLUME-2357-2.patch,
> FLUME-2357-3.patch, FLUME-2357.patch
>
>
> When the AbstractHDFSWriter fails to close a file (due to exceeding the
> callTimeout or other hdfs issues), it will leave the the file open and never
> try again. The only way to close the open files is to restart the flume
> agent. There should be a configurable option to allow the sink to retry to
> close files that had previously failed to close.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
