One way is to have a pass-through operator X that is parallel partitioned like your B currently. Then, connect the output port of X to B and use a suitable partitioner for B to create as many partitions as you want: A -> X -> B -> C.
Ram On Mon, Jul 25, 2016 at 9:41 AM, Yogi Devendra <[email protected]> wrote: > Hi, > > I have a DAG A->B->C. > > 1. A is kafka input operator reading from 4 different topics configured > with ONE_TO_ONE strategy. Thus creating 4 partitons of A. > 2. B and C are configured to have parallel partitions w.r.t. their input > port. Thus, currently both B, C have 4 partitions. > > I am observing B significant latency in B operator. Thus, would like to > have 2 partitions for B per partition of A. Since, application is latency > intensive, I want to avoid unifiers as far as possible. > > How to achieve this partitioning? > > > ~ Yogi >
