[
https://issues.apache.org/jira/browse/FLUME-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2498:
-------------------------------
Attachment: FLUME-2498-5.patch
Uploading Patch.
- Changed the file consume order to be based on modification time (let me if
you feel this is inappropriate choice, can easily switch to creation time if
needed).
- Also added to unit test to check file consume order
- it includes changes from patch v4 submitted by Johny (can u please verify
this [~jrufus] ?)
-corrected the description of 'filegroups' setting in doc. [~iijima_satoshi]
can you please validate ?
Although i have a unit test to verify the consume order, would be good if
someone does a review of it to ensure i didnt mess up. [~iijima_satoshi] johny,
ashish or others ? I have kept my changes to bare minimal to reduce churn.
> Implement Taildir Source
> ------------------------
>
> Key: FLUME-2498
> URL: https://issues.apache.org/jira/browse/FLUME-2498
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Satoshi Iijima
> Fix For: v1.7.0
>
> Attachments: FLUME-2498-2.patch, FLUME-2498-3.patch,
> FLUME-2498-4.patch, FLUME-2498-5.patch, FLUME-2498.patch
>
>
> This is the proposal of implementing a new tailing source.
> This source watches the specified files, and tails them in nearly real-time
> once appends are detected to these files.
> * This source is reliable and will not miss data even when the tailing files
> rotate.
> * It periodically writes the last read position of each file in a position
> file using the JSON format.
> * If Flume is stopped or down for some reason, it can restart tailing from
> the position written on the existing position file.
> * It can add event headers to each tailing file group.
> A attached patch includes a config documentation of this.
> This source requires Unix-style file system and Java 1.7 or later.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
