[
https://issues.apache.org/jira/browse/FLUME-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253545#comment-13253545
]
Hari Shreedharan commented on FLUME-1121:
-----------------------------------------
Brock:
I noticed more than one problem. One of the issues was that even though the
source got exceptions when trying to write more events than the capacity of the
channel(N), the source got exceptions(there was no sink attached), for the
extra events(K). Later when I removed the source, and simply attached a sink,
then the replay function would create a transaction for the internal memory
channel to load all events from the file. Though this should only have been N,
it tries to load N+K events, and fails right away. In the above example, N was
100 and K was 7. K can be anything, the only thing that determines K is the
time you stop sending events.
I believe the replay function, and the way we load events from the file to
memory channel need to be refactored. I think we should not accept new events
from the source, until and unless we have replayed every event from the file
and got commits from the sink for all those transactions. This way we can
guarantee that all events will eventually be replayed and no events are
lost(since the source would send failure back to the previous hop).
That said, things seem to look OK if number of events put into the channel <
total channel capacity.
> Recoverable Memory Channel cannot recover data
> -----------------------------------------------
>
> Key: FLUME-1121
> URL: https://issues.apache.org/jira/browse/FLUME-1121
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.2.0
> Reporter: Hari Shreedharan
>
> Run an agent with a source, like seq with a recoverable memory channel and no
> sink. kill the process with ctrl-C. Run an agent now with no source, but
> logger sink and the same recoverable memory channel. I expected the events
> generated by the source earlier should be written to the sink, but no events
> with picked up.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
