-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7659/
-----------------------------------------------------------
Review request for Flume.
Description
-------
Added lastWrite to BucketWriter to verify when it was last updated
Added a thread to HDFSEventSink which verifies the last update of each active
bucketWriter and closes them after the configurable timeout
hdfs.closeIdleTimeout has passed.
This addresses bug FLUME-1660.
https://issues.apache.org/jira/browse/FLUME-1660
Diffs
-----
flume-ng-doc/sphinx/FlumeUserGuide.rst 29ead84
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
bce8e11
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
a6d624b
Diff: https://reviews.apache.org/r/7659/diff/
Testing
-------
Local machine testing was performed and the correct closing of files was
confirmed, as well as the correct behavior of the configuration setting
including disabling the feature(by using the default value for
hdfs.closeIdleTimeout of 0)
There is one unrelated test failure which I'm not sure of(if anyone knows
what's causing this, please let me know)
Failed tests: testInOut(org.apache.flume.test.agent.TestFileChannel):
Expected FILE_ROLL sink's dir to have only 1 child, but found 0 children.
expected:<1> but was:<0>
All other tests pass.
Thanks,
Juhani Connolly
