[
https://issues.apache.org/jira/browse/CHUKWA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533671#comment-13533671
]
Sourygna Luangsay commented on CHUKWA-675:
------------------------------------------
I haven't tried it in production yet. Nonetheless, I have included this change
in our benchmarks configuration:
- without the change, the agent needs more or less 30 minutes to send one 490MB
file with the file tailer adaptor.
- with the change, it only needs 45 seconds
I don't really understand what you mean by "rate limiting". The code already
offers some protection
to ensure that one (big) file to tail may not prevent other smaller files to be
tailed (see chukwaAgent.fileTailingAdaptor.maxReadSize).
What is more, I think my change is correct if you consider the comment at the
beginning of the class:
* For now, it tries each file in succession. If it gets through every file
* within two seconds, and no more data remain, it will sleep.
*
* If there was still data available in any file, the adaptor will loop again.
> Bad sleeping condition in FileTailer.java
> -----------------------------------------
>
> Key: CHUKWA-675
> URL: https://issues.apache.org/jira/browse/CHUKWA-675
> Project: Chukwa
> Issue Type: Bug
> Components: Data Collection
> Affects Versions: 0.6.0
> Reporter: Sourygna Luangsay
> Assignee: Sourygna Luangsay
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: chukwa-675.patch
>
>
> Current "if condition" in the run() method is wrong.
> As a consequence, a big file to tail will take a lot of time.
> See:
> http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201212.mbox/%3ccaotz1wd-abkjwh5gbnui3o6xj2kiqyf7hvgdg80mdek4h6y...@mail.gmail.com%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
