Isha, What you are suggesting should be a property on the partitioner instead. Not every partitioner needs such property configuration. You could have one that partitions itself based on the external system, for example.
Thanks, Thomas On Fri, Nov 20, 2015 at 12:47 PM, Isha Arkatkar <[email protected]> wrote: > Hi, > > Actually, I was thinking some operator attribute like this in populateDag: > dag.setAttribute(operator, OperatorContext.NUM_PARTITIONS, 2); > > which could translate internally to: > dag.setAttribute(operator, OperatorContext.PARTITIONER, new > StatelessPartitioner<GenericTestOperator>(2)); > > But just setting number of partitions would be easier in populateDag and > user does not need to worry about which partitioner to use. > > Thanks, > Isha > > > On Fri, Nov 20, 2015 at 4:29 AM, Pramod Immaneni <[email protected]> > wrote: > > > Is the suggestion that operators implement the partitioner interface and > > pass-through the calls to the partitioner implementation set as a > property > > or is it something else? > > > > On Thu, Nov 19, 2015 at 8:54 PM, Gaurav Gupta <[email protected]> > > wrote: > > > > > I also think property works better because as Thomas mentioned it can > be > > > configured like any other operator component. > > > > > > Thanks > > > - Gaurav > > > > > > > On Nov 19, 2015, at 5:24 PM, Isha Arkatkar <[email protected]> > > wrote: > > > > > > > > I also think property works better, though, can be an easier API to > > 'set > > > > number of partitions for operator to N' And internally it can pick up > > > > Stateless Partitioner as default? > > > > > > > > Thanks, > > > > Isha > > > > > > > > > > > > On Thu, Nov 19, 2015 at 5:15 PM, Thomas Weise < > [email protected]> > > > > wrote: > > > > > > > >> I actually think a property works better as it will let you > configure > > > the > > > >> partitioner like any other operator component. Like the Kafka > > consumer. > > > >> > > > >> -- > > > >> sent from mobile > > > >> On Nov 19, 2015 4:51 PM, "Siyuan Hua" <[email protected]> > wrote: > > > >> > > > >>> Thomas, this is what I'm going to do. But I like Pramod's idea to > > have > > > an > > > >>> annotation > > > >>> > > > >>> On Thu, Nov 19, 2015 at 4:34 PM, Thomas Weise < > > [email protected]> > > > >>> wrote: > > > >>> > > > >>>> Why not implement the interface on the operator and delegate the > > call > > > >> to > > > >>>> the partitioner implementation? > > > >>>> > > > >>>> On Thu, Nov 19, 2015 at 4:09 PM, Pramod Immaneni < > > > >> [email protected] > > > >>>> > > > >>>> wrote: > > > >>>> > > > >>>>> Idea for an annotation maybe? > > > >>>>> > > > >>>>> On Thu, Nov 19, 2015 at 3:49 PM, Siyuan Hua < > > [email protected]> > > > >>>>> wrote: > > > >>>>> > > > >>>>>> Sometimes I want to separate the Partitioner logic from operator > > > >> and > > > >>>>> still > > > >>>>>> I want to have default partitioner for the operator (with > > > >> specifying > > > >>> it > > > >>>>>> from application) > > > >>>>>> > > > >>>>>> > > > >>>>>> Best, > > > >>>>>> Siyuan > > > >>>>>> > > > >>>>> > > > >>>> > > > >>> > > > >> > > > > > > > > >
