Priyanka Why not just set 25 on operator A and during dynamic partitioning if any you can aggregate old value and distribute among new partitions
Thanks -Gaurav On Sep 22, 2015 10:43 PM, "Priyanka Gugale" <[email protected]> wrote: > Thanks for suggestions. > > Ashwin, The option you proposed is doable, but then each operator which > needs to address such use case will have to track the number of partitions. > So we were looking for some common mechanism. > Chandni, Let me try the res.operatorRequests, that could solve the problem. > > -Priyanka > > On Tue, Sep 22, 2015 at 8:05 PM, Chandni Singh <[email protected]> > wrote: > > > Hi Priyanka, > > > > As Isha mentioned you will can implement a StatsListener. However instead > > of setting res.repartitionRequired = true, you can add a > > StatsListener.OperatorRequest to res.operatorsRequests. This will not > > re-deploy the instances. > > > > The OperatorRequest that you will need to implement will have to change > the > > property value. > > > > You can refer to AutoMetricTest for an example. > > > > - Chandni > > > > On Tue, Sep 22, 2015 at 7:11 AM, Ashwin Chandra Putta < > > [email protected]> wrote: > > > > > Why not maintain a variable containing the number of partitions in each > > > partition; and when you set the property, just divide it by number of > > > partitions variable? Does it not solve the problem? > > > > > > Regards, > > > Ashwin. > > > > > > On Tue, Sep 22, 2015 at 5:37 AM, Priyanka Gugale < > > [email protected] > > > > > > > wrote: > > > > > > > I have an integer property named "intProp". This property is > applicable > > > to > > > > an operator A. I can set the property from dtcli using > > > > "set-operator-property operator-name property-name property-value" > and > > > this > > > > will get applied to all instances of opeartor A. > > > > > > > > Now, I want to apply this property in a way that it's equally divided > > > > amongst all instances of operator A. For example, if intProp=100 and > > > > number of partitions=4 then intProp=25 should be set on each instance > > of > > > > operator A. > > > > > > > > Is there a way to achieve this? > > > > > > > > I have tried following approach to do this: I have overriden > > > > definePartitions of Operator, once we do partitioning I am setting > > right > > > > value on each operator instance based on new partition count. The > > > > limitation here is that the value is applied during definePartitions. > > If > > > > the value is updated dynamically it won't be applied till > > > definePartitions > > > > is called again. > > > > > > > > -Priyanka > > > > > > > > > > > > > > > > -- > > > > > > Regards, > > > Ashwin. > > > > > >
