[
https://issues.apache.org/jira/browse/CHUKWA-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785505#action_12785505
]
Jerome Boulon commented on CHUKWA-421:
--------------------------------------
We can keep track of current offset and the last record and compare it.
This should work assuming that the date should not be the same and if it is and
the record is the same then for us it will be a duplicate anyway.
> new logfile rotation detection scheme
> -------------------------------------
>
> Key: CHUKWA-421
> URL: https://issues.apache.org/jira/browse/CHUKWA-421
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Components: data collection
> Reporter: Ari Rabkin
>
> Currently, we keep files open in order to detect rotation. This is wasteful
> of file handles, doesn't work correctly across reboots, and I think we can do
> better.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
