[
https://issues.apache.org/jira/browse/CHUKWA-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ari Rabkin reopened CHUKWA-421:
-------------------------------
There's a subtle bug in how the new adaptor writes checkpoints. The last-mod
date in the checkpoint reflects data SENT, not data WRITTEN. So after a
collector crash, this adaptor will resume at the wrong place. Fix is for the
adaptor to keep a mapping from send offset to last-mod-date, listen to commit
reports, and write accordingly.
If other adaptors start needing to do this, we might need to pull this
functionality out into a utility class.
> new logfile rotation detection scheme
> -------------------------------------
>
> Key: CHUKWA-421
> URL: https://issues.apache.org/jira/browse/CHUKWA-421
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Components: data collection
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-421.patch
>
>
> Currently, we keep files open in order to detect rotation. This is wasteful
> of file handles, doesn't work correctly across reboots, and I think we can do
> better.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
