[
https://issues.apache.org/jira/browse/FLUME-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949928#comment-13949928
]
Hudson commented on FLUME-2309:
-------------------------------
UNSTABLE: Integrated in flume-trunk #630 (See
[https://builds.apache.org/job/flume-trunk/630/])
FLUME-2309. Spooling directory should not always consume the oldest file first.
(hshreedharan:
http://git-wip-us.apache.org/repos/asf/flume/repo/?p=flume.git&a=commit&h=61b9bcbb69ae3d19f72276b3aaa78ff3679cecfc)
*
flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java
*
flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java
*
flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java
* flume-ng-doc/sphinx/FlumeUserGuide.rst
*
flume-ng-core/src/test/java/org/apache/flume/client/avro/TestReliableSpoolingFileEventReader.java
* flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java
> Spooling directory should not always consume the oldest file first.
> -------------------------------------------------------------------
>
> Key: FLUME-2309
> URL: https://issues.apache.org/jira/browse/FLUME-2309
> Project: Flume
> Issue Type: New Feature
> Affects Versions: v1.4.0
> Reporter: Muhammad Ehsan ul Haque
> Assignee: Muhammad Ehsan ul Haque
> Priority: Minor
> Labels: feature, patch
> Fix For: v1.5.0
>
> Attachments: FLUME-2309-0.patch, FLUME-2309-0.patch,
> FLUME-2309-1.patch, FLUME-2309-commit.patch
>
>
> The ReliableSpoolingFileEventReader reads the oldest file in the spooling
> directory first. This is done by listing the directory contents and then
> sorting file list based on timestamp. This may be very slow if there are a
> lot of files (of the order of 100K or more) in the directory.
> However, this is not always needed, there can be simple cases in which the
> order to consume the file is not important.
> There should be an option of consuming the files in arbitrary order, allowing
> the files to be consumed quickly without any delay.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
