[
https://issues.apache.org/jira/browse/CHUKWA-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785503#action_12785503
]
Ari Rabkin commented on CHUKWA-421:
-----------------------------------
My idea is as follows:
1) Adaptors should keep track of the last-modified date of the file they're
tailing.
2) Each time through, get the list of files whose names match the file in
question, ignoring trailing stuff after a final dot.
(Possible rotation results)
3) Process them, oldest-first. Break ties using longest-first, or some
name-based heuristic.
The end goal of this is that if file Foo is being tailed, and then when we look
next, Foo is shorter and there's a Foo.1, we'll finish tailing Foo.1, and then
start tailing Foo.
> new logfile rotation detection scheme
> -------------------------------------
>
> Key: CHUKWA-421
> URL: https://issues.apache.org/jira/browse/CHUKWA-421
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Components: data collection
> Reporter: Ari Rabkin
>
> Currently, we keep files open in order to detect rotation. This is wasteful
> of file handles, doesn't work correctly across reboots, and I think we can do
> better.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
