[
https://issues.apache.org/jira/browse/FLUME-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Lanza García updated FLUME-3219:
---------------------------------------
Summary: Taildir source: if file is renamed, it is consumed again (was:
Taildir source: if file is rename, it is consumed again)
> Taildir source: if file is renamed, it is consumed again
> --------------------------------------------------------
>
> Key: FLUME-3219
> URL: https://issues.apache.org/jira/browse/FLUME-3219
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: 1.8.0
> Reporter: Daniel Lanza García
> Priority: Major
>
> Current behavior of Taildir is such that if a file is renamed (eg log
> rotated) it is consumed again.
> https://github.com/apache/flume/blob/d1f24f56ce9714bb3e1edc671da290c75a17dead/flume-ng-sources/flume-taildir-source/src/main/java/org/apache/flume/source/taildir/ReliableTaildirEventReader.java#L247
> Would not be better if the inode is followed, and if that inode has been
> consumed do not consume it again? With current implementation, once file is
> rotated, you get duplicates in the case you include in the path previous
> days's data (you want to do that if agent fails and needs to consume data
> from previous days).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
