jweinste created FLUME-2862:
-------------------------------
Summary: KafkaChannel rollback can reorder events
Key: FLUME-2862
URL: https://issues.apache.org/jira/browse/FLUME-2862
Project: Flume
Issue Type: Bug
Components: Channel
Reporter: jweinste
Failed events can be appended to the end of queue during rollback, thus
reordering the events seen.
doTake()...
Event e;
if (!consumerAndIter.get().failedEvents.isEmpty()) {
e = consumerAndIter.get().failedEvents.removeFirst();
}
...
eventTaken = true;
events.get().add(e);
doRollback()
else {
counter.addToRollbackCounter(Long.valueOf(events.get().size()));
consumerAndIter.get().failedEvents.addAll(events.get());
events.get().clear();
}
Failed events (case of failedEvents.removeFirst()) should be added to the front
of queue along lines of:
addAll(0, events.get());
While *new* events should be added to end (--not sure this combo exists--)
Otherwise
failedEvents = [0,1,2,3,4]
doTake()
failedEvents = [1,2,3,4]
doRollback()
failedEvents = [1,2,3,4,0]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
