Instead of creating a new space for evolving operators, why not create a new space for mature operators and graduate existing mature operators?
That way, it solves both the items of discussion - having separation between evolving and mature operators. - not having evolving operators in mature space. Regards, Ashwin. On Fri, May 27, 2016 at 6:01 PM, Thomas Weise <[email protected]> wrote: > It is important that the discussion happens, regardless in which thread. If > we want to establish criteria for operators to be promoted out of contrib > then we need to make sure those already out there match the bar. Today > there are many operators that don't. > > Also, the incubator space needs to be managed. If things don't make it / > are abandoned then there needs to be a policy to deal with it. That's part > of the same package, criteria for going in needs to be complemented by > criteria for removal. > > Thomas > > > On Fri, May 27, 2016 at 4:30 PM, Amol Kekre <[email protected]> wrote: > > > Agreed > > > > Thks > > Amol > > > > On Fri, May 27, 2016 at 4:14 PM, Pramod Immaneni <[email protected] > > > > wrote: > > > > > Amol, > > > > > > I would suggest starting a separate thread for that discussion. > > > > > > Thanks > > > > > > On Fri, May 27, 2016 at 4:05 PM, Amol Kekre <[email protected]> > > wrote: > > > > > > > Yes there are two conflicting threads now. The original thread was to > > > open > > > > up a way for contributors to submit code in a dir (contrib?) as long > as > > > > license part of taken care of. > > > > > > > > On the thread of removing non-used operators -> How do we know what > is > > > > being used? > > > > > > > > Thks, > > > > Amol > > > > > > > > > > > > On Fri, May 27, 2016 at 3:40 PM, Sandesh Hegde < > > [email protected]> > > > > wrote: > > > > > > > > > +1 for removing the not-used operators. > > > > > > > > > > So we are creating a process for operator writers who don't want to > > > > > understand the platform, yet wants to contribute? How big is that > > set? > > > > > If we tell the app-user, here is the code which has not passed all > > the > > > > > checklist, will they be ready to use that in production? > > > > > > > > > > This thread has 2 conflicting forces, reduce the operators and make > > it > > > > easy > > > > > to add more operators. > > > > > > > > > > > > > > > > > > > > On Fri, May 27, 2016 at 3:03 PM Pramod Immaneni < > > > [email protected]> > > > > > wrote: > > > > > > > > > > > On Fri, May 27, 2016 at 2:30 PM, Gaurav Gupta < > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Pramod, > > > > > > > > > > > > > > By that logic I would say let's put all partitionable operators > > > into > > > > > one > > > > > > > folder, non-partitionable operators in another and so on... > > > > > > > > > > > > > > > > > > > Remember the original goal of making it easier for new members to > > > > > > contribute and managing those contributions to maturity. It is > not > > a > > > > > > functional level separation. > > > > > > > > > > > > > > > > > > > When I look at hadoop code I see these annotations being used > at > > > > class > > > > > > > level and not at package/folder level. > > > > > > > > > > > > > > > > > > I had a typo in my email, I meant to say "think of this like a > > > > folder..." > > > > > > as an analogy and not literally. > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > On Fri, May 27, 2016 at 2:10 PM, Pramod Immaneni < > > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > On Fri, May 27, 2016 at 1:05 PM, Gaurav Gupta < > > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Can same goal not be achieved by > > > > > > > > > using > > > > org.apache.hadoop.classification.InterfaceStability.Evolving > > > > > / > > > > > > > > > > org.apache.hadoop.classification.InterfaceStability.Unstable > > > > > > > annotation? > > > > > > > > > > > > > > > > > > > > > > > > > I think it is important to localize the additions in one > place > > so > > > > > that > > > > > > it > > > > > > > > becomes clearer to users about the maturity level of these, > > > easier > > > > > for > > > > > > > > developers to track them towards the path to maturity and > also > > > > > > provides a > > > > > > > > clearer directive for committers and contributors on > acceptance > > > of > > > > > new > > > > > > > > submissions. Relying on the annotations alone makes them > spread > > > all > > > > > > over > > > > > > > > the place and adds an additional layer of difficulty in > > > > > identification > > > > > > > not > > > > > > > > just for users but also for developers who want to find such > > > > > operators > > > > > > > and > > > > > > > > improve them. This of this like a folder level annotation > where > > > > > > > everything > > > > > > > > under this folder is unstable or evolving. > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, May 27, 2016 at 12:35 PM, David Yan < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Malhar in its current state, has way too many > > operators > > > > > that > > > > > > > fall > > > > > > > > > in > > > > > > > > > > > the > > > > > > > > > > > > > "non-production quality" category. We should make > it > > > > > obvious > > > > > > to > > > > > > > > > users > > > > > > > > > > > > that > > > > > > > > > > > > > which operators are up to par, and which operators > > are > > > > not, > > > > > > and > > > > > > > > > maybe > > > > > > > > > > > > even > > > > > > > > > > > > > remove those that are likely not ever used in a > real > > > use > > > > > > case. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am ambivalent about revisiting older operators and > > > doing > > > > > this > > > > > > > > > > exercise > > > > > > > > > > > as > > > > > > > > > > > > this can cause unnecessary tensions. My original > intent > > > is > > > > > for > > > > > > > > > > > > contributions going forward. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IMO it is important to address this as well. Operators > > > > outside > > > > > > the > > > > > > > > play > > > > > > > > > > > area should be of well known quality. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think this is important, and I don't anticipate much > > > tension > > > > if > > > > > > we > > > > > > > > > > establish clear criteria. > > > > > > > > > > It's not helpful if we let the old subpar operators stay > > and > > > > put > > > > > up > > > > > > > the > > > > > > > > > > bars for new operators. > > > > > > > > > > > > > > > > > > > > David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Regards, Ashwin.
