[
https://issues.apache.org/jira/browse/CHUKWA-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719735#action_12719735
]
Eric Yang commented on CHUKWA-185:
----------------------------------
If we go with time window approach, DTA will only work on files that have
active updates. What if the user want to stream over files that were
previously archived and no longer receiving updates? This is not in the
previous identified use case, but it may make sense to include this use case.
If we specify start time (as processed time flag), and time window size, the
system could process data in a queue and try to closing the gap between past
and present. The processed time flag could be used as an indicator for
resuming agent crash as well.
> ability to tail a whole directory
> ---------------------------------
>
> Key: CHUKWA-185
> URL: https://issues.apache.org/jira/browse/CHUKWA-185
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: data collection
> Affects Versions: 0.1.2, 0.2.0
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Fix For: 0.3.0
>
> Attachments: CHUKWA-185.patch
>
>
> Right now, FileTailingAdaptors watch particular files. It'd be great to be
> able to watch a whole path: to say something like /var/logs/*, where new logs
> created in that directory get picked up.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
