Re: Sub Partitioning the parallel partitions

@Yogi, If you are up for the small experiment and share some latency number with us, It will help improve the unifiers. 1. Avoid M*1 unifiers 2. Use the mentioned PR to launch your application with the unifiers, https://github.com/apache/apex-core/pull/358/ On Mon, Jul 25, 2016 at 11:47 PM

Re: Sub Partitioning the parallel partitions

Yogi, What is the reason for avoiding unifiers? Regards, Ashwin. On Mon, Jul 25, 2016 at 9:41 AM, Yogi Devendra wrote: > Hi, > > I have a DAG A->B->C. > > 1. A is kafka input operator reading from 4 different topics configured > with ONE_TO_ONE strategy. Thus creating

Re: Sub Partitioning the parallel partitions

Since the partitions of the Kafka input operator are based on (topic, partition) combination, can we increase the number of Kafka partitions per topic? ~ Bhupesh On Mon, Jul 25, 2016 at 10:21 PM, Munagala Ramanath wrote: > One way is to have a pass-through operator X that

Re: Sub Partitioning the parallel partitions

One way is to have a pass-through operator X that is parallel partitioned like your B currently. Then, connect the output port of X to B and use a suitable partitioner for B to create as many partitions as you want: A -> X -> B -> C. Ram On Mon, Jul 25, 2016 at 9:41 AM, Yogi Devendra

Sub Partitioning the parallel partitions

Hi, I have a DAG A->B->C. 1. A is kafka input operator reading from 4 different topics configured with ONE_TO_ONE strategy. Thus creating 4 partitons of A. 2. B and C are configured to have parallel partitions w.r.t. their input port. Thus, currently both B, C have 4 partitions. I am observing