@hadoop.apache.org
Subject: Re: Distributed queue: how to ensure no lost items?
At least once is generally the case in queuing systems unless you can
do a distributed transaction with your consumer. What comes in handy
in an at least once case, is letting the consumer know that a message
may have
At least once is generally the case in queuing systems unless you can
do a distributed transaction with your consumer. What comes in handy
in an at least once case, is letting the consumer know that a message
may have 'potentially' already been processed. That way he can double
check first before
We should expand that section. the current queue recipe guarantees that things
are consumed at most once. to guarantee at least the consumer creates an
ephemeral node queue-X-inprocess to indicate that the node is being processed.
once the queue element has been processed the consumer deletes qu
You can't simply leave an element in the queue until a consumer
finishes processing it, otherwise multiple consumers may end up
processing it. What about the following:
- Use a failure detector to detect which consumers are up;
- Before removing an element from the queue, a consumer creates a
