[
https://issues.apache.org/jira/browse/FLUME-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638461#comment-13638461
]
Hari Shreedharan commented on FLUME-1819:
-----------------------------------------
Venkatesh,
The patch generally looks good, but like you said, it will still wait for a new
event to be available. You could have a 2nd thread to which the event list is
passed which can check if the list is empty and if it is not it can flush it.
This thread can be scheduled (using a scheduled executor service) after timeout
seconds. You will need to be careful about locking while the events are being
flushed (the main runnable and the flush runnable should not execute the
flushing code simultaneously). In this case you could even remove the timeout()
method. Does that make sense?
> ExecSource don't flush the cache if there is no input entries
> -------------------------------------------------------------
>
> Key: FLUME-1819
> URL: https://issues.apache.org/jira/browse/FLUME-1819
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Reporter: Fengdong Yu
> Assignee: Venkatesh Sivasubramanian
> Fix For: v1.4.0
>
> Attachments: FLUME-1819.patch, FLUME-1819.patch.1
>
>
> ExecSource has a default batchSize: 20, exec source read data from the
> source, then put it into the cache, after the cache is full, push it to the
> channel.
> but if exec source's cache is not full, and there isn't any input for a long
> time, then these entries always kept in the cache, there is no chance to the
> channel until the source's cache is full.
> so, the patch added a new config line: batchTimeout for ExecSource, and
> default is 3 seconds, if batchTimeout exceeded, push all cached data to the
> channel even the cache is not full.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
