Re: Ordering of messages in the same kafka streams sub-topology with multiple sinks for the same topic

2019-12-04 Thread Vasily Sulatskov
Hi John, Thanks for you explanation. I guess I'll have to adapt my code to assume there's no ordering guarantee between different "branches". On 2019/12/03 15:59:59, "John Roesler" wrote: > Hi Vasily,> > > Probably in this case, with the constraints you're providing, the first > branch wou

Re: Ordering of messages in the same kafka streams sub-topology with multiple sinks for the same topic

2019-12-03 Thread John Roesler
Hi Vasily, Probably in this case, with the constraints you’re providing, the first branch would output first, but I wouldn’t depend on it. Any small change in your program could mess this up, and also any change in Streams could alter the exact execution order also. The right way to think abo

Re: Ordering in a Kafka Pipe line

2018-07-06 Thread Emmett Butler
Kafka provides total ordering only within individual partitions. A topic with multiple partitions is considered a "partial order", in which multiple subsets of the topic are considered well-ordered but the topic as a whole is not. The tradeoff between scalability via partitioning and message orderi

Re: Ordering guarantee after branching a stream

2018-04-20 Thread Matthias J. Sax
As long as no repartitioning happens yes. If one of both sub-streams is repartitioned, there is no guarantee. -Matthias On 4/20/18 4:31 PM, Botuck, Jacob (STL) - contr wrote: > If I call branch on a kStream, and I send record A into the trunkStream > followed by record B. If records A and B go

Re: ordering

2013-08-22 Thread Jun Rao
-Original Message- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Thursday, August 22, 2013 12:01 AM > To: users@kafka.apache.org > Subject: Re: ordering > > Actually, I am not sure if I understand the trouble that you mentioned. > Could you elaborate that a bit more? &

RE: ordering

2013-08-22 Thread Yu, Libo
] Sent: Thursday, August 22, 2013 12:01 AM To: users@kafka.apache.org Subject: Re: ordering Actually, I am not sure if I understand the trouble that you mentioned. Could you elaborate that a bit more? Thanks, Jun On Wed, Aug 21, 2013 at 12:30 PM, Yu, Libo wrote: > Hi, > > This is f

Re: ordering

2013-08-21 Thread Jun Rao
Actually, I am not sure if I understand the trouble that you mentioned. Could you elaborate that a bit more? Thanks, Jun On Wed, Aug 21, 2013 at 12:30 PM, Yu, Libo wrote: > Hi, > > This is from kafka faq: > > > * Each partition is not consumed by more than one consumer > thread/process in

Re: ordering

2013-08-21 Thread Philip O'Toole
No, there isn't, not at the very start when there is no state in Zookeeper. Once there is state the Kafka team have told me that rebalancing will not result in any dupes. However, if there is no state in Zookeeper and your partitions are empty, simply wait until all consumers have balanced before