[
https://issues.apache.org/jira/browse/FLUME-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162393#comment-14162393
]
Gwen Shapira commented on FLUME-2495:
-------------------------------------
ok, I think I see what you mean. Basically, clear list immediately after
writing to channel. Then if we fail to commit to Kafka, most of the time we
will not have duplicates (unless the source actually dies)
> Kafka Source may miss events when channel is not available
> ----------------------------------------------------------
>
> Key: FLUME-2495
> URL: https://issues.apache.org/jira/browse/FLUME-2495
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Reporter: Gwen Shapira
> Assignee: Gwen Shapira
> Attachments: FLUME-2495.0.patch, FLUME-2495.1.patch
>
>
> Because the Kafka consumer itself tracks offsets, and we don't restart the
> consumer when we get channel errors, the consumer will skip messages that we
> couldn't write to channel, even though we did not advance offset in Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
