Guy, This is really an issue with changing # of partitions. If # of partitions changes for a topic, in the transition phase, messages used to be delivered to the same partition could be delivered to different partitions and their consumption ordering is non-deterministic (since ordered consumption is only guaranteed within a partition).
In 0.7, # of partitions increases as new brokers are added. In 0.8, # of partitions is set at topic creation time and will stay the same when new brokers are added. Thanks, Jun On Wed, Oct 31, 2012 at 4:12 AM, Guy Peleg <guy.pe...@gmail.com> wrote: > Hi, > > As I learn and plan to use Kafka, I'm concirned about possible race > condition when brokers/consumers are added or removed. > > Say I have a topic that is devide into two partitions, where consumers are > deviding the mssages between those two partitions by ,say, modulo event-id, > where events with the same event ids should be processed by the order of > their arrival, that will work since as I said, I will devide the incoming > events by their event-id % number_of_partitions > > Now, when a new paratition is added, there might be situations where events > with event-id 'x', will still be in the first broker, while new ones, with > event-id 'x', are added to the new paratition > which may result in those events being processed in parallel, what am i > missing? > > Thanks, > > Guy >
