Re: balanced of Stream Codec

2016-10-15 Thread Thomas Weise
Without knowing the operations following the indeterministic partitioning, assume that you cannot have exactly-once results because processing won't be idempotent. If there are only stateless operations, then it should be OK. If there are stateful operations (windowing with any form of aggregation

Re: balanced of Stream Codec

2016-10-15 Thread Amol Kekre
Sunil, Round robin in an internal operator could be used in exactly once writes to external system for certain operations. I do know what your business logic is, but in case it can be split into partitions and then unified (for example aggregates), you have a situation where you can use round

Re: balanced of Stream Codec

2016-10-15 Thread Munagala Ramanath
If you want round-robin distribution which will give you uniform load across all partitions you can use a StreamCodec like this (provided the number of partitions is known and static): *public class CatagoryStreamCodec extends KryoSerializableStreamCodec {* * private int n = 0;* * @Override* *