Hi Antonio, AFAIK, there are two reasons for this:
1. Rebalancing itself brings latency because it takes time to redistribute the elements. 2. Rebalancing also messes up the order in the Kafka topic partitions, and often makes a event-time window wait longer to trigger in case you’re using event time characteristic. Best Regards, Paul Lam > 在 2018年8月10日，05:49，antonio saldivar <ansal...@gmail.com> 写道： > > Hello > > Sending ~450 elements per second ( the values are in milliseconds start to > end) > I went from: > with Rebalance > +------------+ > | AVGWINDOW | > +------------+ > | 32131.0853 | > +------------+ > > to this without rebalance > > +------------+ > | AVGWINDOW | > +------------+ > | 70.2077 | > +------------+ > > El jue., 9 ago. 2018 a las 17:42, Elias Levy (<fearsome.lucid...@gmail.com > <mailto:fearsome.lucid...@gmail.com>>) escribió: > What do you consider a lot of latency? The rebalance will require > serializing / deserializing the data as it gets distributed. Depending on > the complexity of your records and the efficiency of your serializers, that > could have a significant impact on your performance. > > On Thu, Aug 9, 2018 at 2:14 PM antonio saldivar <ansal...@gmail.com > <mailto:ansal...@gmail.com>> wrote: > Hello > > Does anyone know why when I add "rebalance()" to my .map steps is adding a > lot of latency rather than not having rebalance. > > > I have kafka partitions in my topic 44 and 44 flink task manager > > execution plan looks like this when I add rebalance but it is adding a lot of > latency > > kafka-src -> rebalance -> step1 -> rebalance ->step2->rebalance -> kafka-sink > > Thank you > regards >
