Good point. If some authors raise PRs without noticing the need for a proposal, we shouldn't ask them to close their PRs only because of the absence of the proposal.
"Design review" without a proposal for simple PRs would be good if we can determine well what PRs need and what don't. But, how do we know? Even for the same PR, someone may think it needs a proposal but another may not. If someone don't notice the need for a proposal and raise a PR without it, I'm fine with that. However, we should still encourage writing a proposal before writing code because we can avoid unnecessary effort. I think this kind of issue usually happens for first time contributors and they will be better once they get used to Druid development. And I believe someday even first contributors would follow this policy once it gets settled down well in the community as Kafka community does. Jihoon On Tue, Jan 15, 2019 at 4:31 AM Roman Leventov <leventov...@gmail.com> wrote: > In such small PRs, authors likely won't be aware that they need to create a > proposal in the first place. The first reviewer just adds the "Design > Review" tag. It's also absolutely not about considering designs and gauging > the proposal, it's just verifying that a configuration / parameter / HTTP > endpoint name is reasonable and aligned with the rest of Druid. So I think > that a separate proposal issue for such PRs is unnecessary bureaucracy. > > On Tue, 15 Jan 2019 at 07:45, Jihoon Son <ghoon...@gmail.com> wrote: > > > Roman, > > > > > Jihoon in > > > > > https://lists.apache.org/thread.html/e007fbf362c2a870a2d88d04431789289807e00fd91d087559a01d1f@%3Cdev.druid.apache.org%3E > > and later Gian in this thread suggested that _every_ piece of work that > > should be labelled as "Design Review" according to the current rules > should > > be accompanied by an issue. I don't agree with this, there are some PRs > as > > small as a few dozens of lines of code, that add some configuration > > parameter and therefore should be labelled "Design Review". I don't > thing a > > separate proposal issue is needed for them, and even for a little larger > > PRs too. > > > > What I'm concerned with is how people feel if their design is not > accepted > > even though they wrote code. Of course, as Clint said, sometimes code > helps > > better understanding of the proposal. But, I believe this is the case > when > > the proposal is quite complicated and not easy to understand without > code. > > Also the authors should be aware of that they might rewrite the entire > code > > if the design should be changed. > > > > If writing code is simple, I don't see why the authors don't wait until > the > > review for their proposal is finished. > > > > Jihoon > > > > On Fri, Jan 11, 2019 at 9:51 AM Fangjin Yang <fang...@imply.io> wrote: > > > > > I agree with Gian, as an Apache committer, your responsibility is for > the > > > betterment of the project. I agree it is in the best interest of the > > > project to stop thinking about what orgs people belong to. We are all a > > > part of the Apache software foundation, regardless of what our roles > and > > > titles are outside of it. > > > > > > On Fri, Jan 11, 2019 at 2:22 AM Roman Leventov <leventov...@gmail.com> > > > wrote: > > > > > > > It's not that people from one org could abuse the project and push > some > > > > change, but that they have similar perspective (bubble effect) and > some > > > > important aspects of a large feature could escape their attention. > > > > > > > > I suggest it to be not a rigid rule, but a recommendation for authors > > of > > > > large proposals to try to attract reviewers from other orgs. > > > > > > > > On Fri, 11 Jan 2019 at 02:51, Julian Hyde <jh...@apache.org> wrote: > > > > > > > > > I agree with Gian. > > > > > > > > > > As an Apache committer, you only have one affiliation: you are > > working > > > in > > > > > the best interests of the project. > > > > > > > > > > Obviously, in the real world there are other pressures. But we do > our > > > > best > > > > > to compensate for them. > > > > > > > > > > Also, as a a community we try to design our process so as to avoid > > > undue > > > > > influences. For instance, when I advocate for logging cases early, > I > > am > > > > > trying to mitigate the effect of product managers and VPs of > > > engineering, > > > > > who like to have their say in meeting rooms rather than on public > > > mailing > > > > > lists. That’s just one example; if we see other influences at play, > > > let’s > > > > > evolve our process to try to level the playing field. > > > > > > > > > > Julian > > > > > > > > > > > > > > > > On Jan 10, 2019, at 10:40 AM, Gian Merlino <g...@apache.org> > > wrote: > > > > > > > > > > > >>> What do community members think about also making a requirement > > > that > > > > > >>> "Design Review" PRs and proposals are reviewed by at least two > > > people > > > > > > with > > > > > >>> different affiliation? > > > > > >> This seems pretty reasonable to me. I haven't found anything in > > > Apache > > > > > >> voting procedure docs ( > > > https://www.apache.org/foundation/voting.html) > > > > > that > > > > > >> seems to explicitly forbid something like this yet at least. > > > > > > > > > > > > On the other hand, my understanding of the Apache Way would mean > > that > > > > > this > > > > > > kind of rule doesn't make sense. In particular from > > > > > > https://www.apache.org/foundation/how-it-works.html: "We firmly > > > > believe > > > > > in > > > > > > hats. Your role at the ASF is one assigned to you personally, and > > is > > > > > > bestowed on you by your peers. It is not tied to your job or > > current > > > > > > employer or company." That sentiment seems incompatible with > making > > > > > > explicit rules about organizational diversity in voting. I have > > also > > > > > heard > > > > > > a few people say things like: people are supposed to represent > > > > > themselves, > > > > > > not their employers. > > > > > > > > > > > > Obviously, though, people's actions and opinions are influenced > by > > > > their > > > > > > employer. IMO, a better way to approach a potential problem there > > is > > > > that > > > > > > if people from a particular organization end up behaving > abusively, > > > > then > > > > > > PMC members from other organizations (or in extreme situations > the > > > > Apache > > > > > > board itself) should tell them to knock it off. And then do more > > > > serious > > > > > > things if the inappropriate behavior continues. This kind > > > > > > > > > > > > The other thing you brought up, promoting more shared ownership > and > > > > > > awareness, I am hopeful that separating proposals from PRs will > > help > > > > with > > > > > > that. One reason is that it takes much less time to understand a > > well > > > > > > written proposal than it takes to understand a PR. Code is very > > clear > > > > but > > > > > > it is also very verbose and takes a while to read and understand. > > So > > > in > > > > > > theory changing how we operate, in this way, should promote more > > > > > > understanding of more people across the code base. > > > > > > > > > > > > On Thu, Jan 10, 2019 at 1:34 AM Clint Wylie < > clint.wy...@imply.io> > > > > > wrote: > > > > > > > > > > > >>> > > > > > >>> What do community members think about also making a requirement > > > that > > > > > >>> "Design Review" PRs and proposals are reviewed by at least two > > > people > > > > > >> with > > > > > >>> different affiliation? > > > > > >> > > > > > >> > > > > > >> This seems pretty reasonable to me. I haven't found anything in > > > Apache > > > > > >> voting procedure docs ( > > > https://www.apache.org/foundation/voting.html) > > > > > that > > > > > >> seems to explicitly forbid something like this yet at least. > > > > > >> > > > > > >> On Wed, Jan 9, 2019 at 8:52 PM Roman Leventov < > > > leventov...@gmail.com> > > > > > >> wrote: > > > > > >> > > > > > >>> What do community members think about also making a requirement > > > that > > > > > >>> "Design Review" PRs and proposals are reviewed by at least two > > > people > > > > > >> with > > > > > >>> different affiliation? IMO it's a good idea, because it ensures > > > that > > > > > >>> different interests are taken into account. Also it pushes > people > > > to > > > > > >> engage > > > > > >>> with work done in other parts of Druid, improving shared code > > > owning > > > > > and > > > > > >>> awareness. > > > > > >>> > > > > > >>> Or it's against the Apache Way because it's assumed that there > > are > > > no > > > > > >>> company boundaries within the community? > > > > > >>> > > > > > >>> On Thu, 10 Jan 2019 at 11:45, Roman Leventov < > > > leventov...@gmail.com> > > > > > >>> wrote: > > > > > >>> > > > > > >>>> I see two important reasons why it makes sense to file an > issue > > > and > > > > > >>>> probably announce it in the mailing list, before writing a lot > > of > > > > > code, > > > > > >>>> despite not having a clear picture of what it will be and any > > > > > >> performance > > > > > >>>> data: > > > > > >>>> 1) somebody could already work on this problem privately in > > > > parallel, > > > > > >> it > > > > > >>>> allows to avoid clash in people's works > > > > > >>>> 2) some people could quickly think about the problem field and > > > share > > > > > >>>> high-level ideas that could wildly change the direction in > which > > > the > > > > > >>> author > > > > > >>>> (the person who is going to write code) will move in his work > > from > > > > > >> early > > > > > >>> on. > > > > > >>>> > > > > > >>>> Jihoon in > > > > > >>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/e007fbf362c2a870a2d88d04431789289807e00fd91d087559a01d1f@%3Cdev.druid.apache.org%3E > > > > > >>> and > > > > > >>>> later Gian in this thread suggested that _every_ piece of work > > > that > > > > > >>> should > > > > > >>>> be labelled as "Design Review" according to the current rules > > > should > > > > > be > > > > > >>>> accompanied by an issue. I don't agree with this, there are > some > > > PRs > > > > > as > > > > > >>>> small as a few dozens of lines of code, that add some > > > configuration > > > > > >>>> parameter and therefore should be labelled "Design Review". I > > > don't > > > > > >>> thing a > > > > > >>>> separate proposal issue is needed for them, and even for a > > little > > > > > >> larger > > > > > >>>> PRs too. > > > > > >>>> > > > > > >>>> For the same reason, I also don't see the point of renaming > > > "Design > > > > > >>>> Review" into "Proposal", as well as separating "Design Review" > > > into > > > > > >>>> "Proposal" and something like "API Review". I think a single > > > "Design > > > > > >>>> Review" tag handles it well. > > > > > >>>> > > > > > >>>> Gian mentioned an idea that PRs that follow a "Design Review" > > > > proposal > > > > > >>>> issue shouldn't be "Design Review" themselves. I don't agree > > with > > > > > >> this, I > > > > > >>>> think that actual code and performance data are important > inputs > > > > that > > > > > >>>> should be re-evaluated at least by two people. I even think > that > > > > it's > > > > > >>> very > > > > > >>>> desirable that at least two people read _each line of > production > > > > code_ > > > > > >> in > > > > > >>>> large PRs, although it's not what was done historically in > > Druid, > > > > > >> because > > > > > >>>> large bodies of newly added code, with whole new classes and > > > > > subsystems > > > > > >>>> added, are also coincidentally tested worse than already > > existing > > > > > >> classes > > > > > >>>> and subsystems, including in production. It seems to me that > > those > > > > > huge > > > > > >>>> code influxes is a major source of bugs, that could later take > > > years > > > > > to > > > > > >>>> squeeze out from the codebase. > > > > > >>>> > > > > > >>>> On Wed, 9 Jan 2019 at 08:24, Clint Wylie < > clint.wy...@imply.io> > > > > > wrote: > > > > > >>>> > > > > > >>>>> Apologies for the delayed response. > > > > > >>>>> > > > > > >>>>> On Thu, Jan 3, 2019 at 12:48 PM Slim Bouguerra < > > > > > >>> slim.bougue...@gmail.com> > > > > > >>>>> wrote: > > > > > >>>>> > > > > > >>>>>> I am wondering here what is the case where code first is > > better? > > > > > >>>>>> > > > > > >>>>> > > > > > >>>>> I don't think it's wrong to share ideas as early as possible, > > and > > > > > >> after > > > > > >>>>> this discussion I think I am in favor of it too. I just meant > > > that > > > > I > > > > > >>> don't > > > > > >>>>> think it's always necessarily the most productive discussion > > > until > > > > > >> code > > > > > >>>>> exists sometimes, with the types of thing I am thinking of > are > > > > almost > > > > > >>>>> entirely limited to cases where things might sound good to > > anyone > > > > on > > > > > >>> paper > > > > > >>>>> but in reality need a large amount of experiments conducted > and > > > > > >>>>> observations collected to determine that something is > actually > > > > worth > > > > > >>>>> doing, > > > > > >>>>> which I imagine is mainly things like reworking internals for > > > > > >>> performance > > > > > >>>>> improvements. > > > > > >>>>> > > > > > >>>>> In the case of my combined proposal PR, I needed to prove > that > > > the > > > > > >>> thing I > > > > > >>>>> was working on was a good idea... and it wasn't directly. > But I > > > > came > > > > > >> up > > > > > >>>>> with another idea during the course of experiment turned into > > > > > >> something > > > > > >>>>> compelling, so an initial proposal would have looked quite a > > lot > > > > > >>> different > > > > > >>>>> than what I ended up with. Once I had proven to myself that > it > > > was > > > > a > > > > > >>> good > > > > > >>>>> idea, then I was comfortable sharing with the wider > community. > > > I'm > > > > > not > > > > > >>>>> certain how this would play out in an always proposal first > > > model, > > > > > >> maybe > > > > > >>>>> the first proposal exists, I personally reject it after > > updating > > > > with > > > > > >>>>> experiment results show it's a bad idea, continue > experimenting > > > and > > > > > >>> raise > > > > > >>>>> a > > > > > >>>>> new one after the experiments start looking promising? > > > > > >>>>> > > > > > >>>>> > > > > > >>>>>> Let's not be naive this is very rare that a contributor will > > > > accept > > > > > >>> that > > > > > >>>>>> his work is to be thrown, usually devs takes coding as > > personal > > > > > >>> creation > > > > > >>>>>> and they get attached to it. > > > > > >>>>>> > > > > > >>>>> > > > > > >>>>> I agree, just because a handful of the committers have this > > > > attitude, > > > > > >> it > > > > > >>>>> isn't fair to expect the wider community to also, that's why > I > > am > > > > in > > > > > >>> favor > > > > > >>>>> of formalizing the process. > > > > > >>>>> > > > > > >>>>> Can you please explain what is overbearing ? what can be > > changed > > > to > > > > > >> make > > > > > >>>>> it > > > > > >>>>>> easy ? > > > > > >>>>>> Most of the points are kind of the actual questions that you > > > want > > > > to > > > > > >>>>>> address before hand anyway isn't ? > > > > > >>>>>> > > > > > >>>>> > > > > > >>>>> Sorry for the confusion, I said it's "not overbearing", I > think > > > > it's > > > > > >>> fine. > > > > > >>>>> > > > > > >>>>> What are the complaints ? > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> Is this and other previous threads not a complaint about > > opening > > > a > > > > > >> large > > > > > >>>>> PR > > > > > >>>>> without a proposal? :) I just mean that formalizing the > > process, > > > > even > > > > > >>> if a > > > > > >>>>> proposal has a reference PR opened with it near concurrently, > > > could > > > > > >>>>> prevent > > > > > >>>>> these discussions from happening in the future because ground > > > rules > > > > > >> have > > > > > >>>>> been set and we are all on the same page I guess. > > > > > >>>>> > > > > > >>>>> I think we should also probably consider renaming the "design > > > > review" > > > > > >>>>> label > > > > > >>>>> to "proposal" or something to make it more clear that a PR is > > > > > >> associated > > > > > >>>>> with a proposal. It might also be worth considering using > > github > > > > > >>> projects > > > > > >>>>> for particularly large things that involve multiple follow up > > > PRs, > > > > > >> but I > > > > > >>>>> haven't used them in much depth to know if they add anything. > > > > > >>>>> > > > > > >>>>> It seems like we are all converging on agreement to do a > github > > > > issue > > > > > >>>>> proposal for larger changes (I would vote for announcing them > > on > > > > the > > > > > >> dev > > > > > >>>>> list too for more visibility), so that design review is > > separated > > > > > from > > > > > >>>>> code > > > > > >>>>> review. I guess my main concern was not wanting to discourage > > > > > >>>>> experimentation by walling it off behind mandatory > discussions, > > > but > > > > > >> the > > > > > >>>>> more I think about it, it doesn't technically change much > > about > > > > this > > > > > >>>>> process, it just requires a more formal proposal to accompany > > any > > > > > >>>>> experiments that are shared as a PR. > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> On Tue, Jan 8, 2019 at 12:28 PM Gian Merlino < > g...@apache.org> > > > > > wrote: > > > > > >>>>> > > > > > >>>>>> I think for us, choosing to use GitHub issues as discussion > > > > threads > > > > > >>> for > > > > > >>>>>> potential 'major' contributions would be a good idea, > > especially > > > > if > > > > > >> we > > > > > >>>>>> encourage people to start them before PRs show up. > Definitely > > > > agree > > > > > >>> that > > > > > >>>>>> all contributors should go through the same process -- I > > > couldn't > > > > > >> see > > > > > >>> it > > > > > >>>>>> working well any other way. > > > > > >>>>>> > > > > > >>>>>> On Mon, Jan 7, 2019 at 12:23 PM Julian Hyde < > jh...@apache.org > > > > > > > > >> wrote: > > > > > >>>>>> > > > > > >>>>>>> Statically, yes, GitHub PRs are the same as GitHub cases. > But > > > > > >>>>>> dynamically, > > > > > >>>>>>> they are different, because you can only log a PR when you > > have > > > > > >>>>> finished > > > > > >>>>>>> work. > > > > > >>>>>>> > > > > > >>>>>>> A lot of other Apache projects use JIRA, so there is a > clear > > > > > >>>>> distinction > > > > > >>>>>>> between cases and contributions. JIRA cases, especially > when > > > > > >> logged > > > > > >>>>> early > > > > > >>>>>>> in the lifecycle of a contribution, become long-running > > > > > >> conversation > > > > > >>>>>>> threads with a lot of community participation. If the Druid > > > chose > > > > > >> to > > > > > >>>>> do > > > > > >>>>>> so, > > > > > >>>>>>> GitHub cases could be the same. > > > > > >>>>>>> > > > > > >>>>>>> Be careful that you do not treat “potential contributors” > (by > > > > > >> which > > > > > >>> I > > > > > >>>>>>> presume you mean non-committers) differently from > committers > > > and > > > > > >> PMC > > > > > >>>>>>> members. Anyone starting a major piece of work should > follow > > > the > > > > > >>> same > > > > > >>>>>>> process. (Experienced committers probably have a somewhat > > > better > > > > > >>> idea > > > > > >>>>>> what > > > > > >>>>>>> work will turn out to be “major”, so they get a little more > > > > > >> leeway.) > > > > > >>>>>>> > > > > > >>>>>>> Julian > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>>> On Jan 7, 2019, at 12:10 PM, Gian Merlino < > g...@apache.org> > > > > > >>> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>> I don't think there's a need to raise issues for every > > > change: a > > > > > >>>>> small > > > > > >>>>>>> bug > > > > > >>>>>>>> fix or doc fix should just go straight to PR. (GitHub PRs > > show > > > > > >> up > > > > > >>> as > > > > > >>>>>>> issues > > > > > >>>>>>>> in the issue-search UI/API, so it's not like this means > the > > > > > >> patch > > > > > >>>>> has > > > > > >>>>>> no > > > > > >>>>>>>> corresponding issue -- in a sense the PR _is_ the issue.) > > > > > >>>>>>>> > > > > > >>>>>>>> I do think it makes sense to encourage potential > > contributors > > > to > > > > > >>>>> write > > > > > >>>>>> to > > > > > >>>>>>>> the dev list or raise an issue if they aren't sure if > > > something > > > > > >>>>> would > > > > > >>>>>>> need > > > > > >>>>>>>> to go through a more heavy weight process. > > > > > >>>>>>>> > > > > > >>>>>>>> Fwiw we do have a set of 'design review' criteria already > > (we > > > > > >> had > > > > > >>> a > > > > > >>>>>>>> discussion about this a couple years ago) at: > > > > > >>>>>>>> http://druid.io/community/#getting-your-changes-accepted. > > So > > > we > > > > > >>>>>>> wouldn't be > > > > > >>>>>>>> starting from zero on defining that. We set it up back > when > > we > > > > > >>> were > > > > > >>>>>>> trying > > > > > >>>>>>>> to _streamline_ our process -- we used to require two > > > non-author > > > > > >>> +1s > > > > > >>>>>> for > > > > > >>>>>>>> _every_ change, even minor ones. The introduction of > design > > > > > >> review > > > > > >>>>>>> criteria > > > > > >>>>>>>> was meant to classify which PRs need that level of review > > and > > > > > >>> which > > > > > >>>>>> ones > > > > > >>>>>>>> are minor and can be merged with less review. I do think > it > > > > > >> helped > > > > > >>>>> with > > > > > >>>>>>>> getting minor PRs merged more quickly. The list of > criteria > > > is, > > > > > >>>>>>>> > > > > > >>>>>>>> - Major architectural changes or API changes > > > > > >>>>>>>> - HTTP requests and responses (e. g. a new HTTP endpoint) > > > > > >>>>>>>> - Interfaces for extensions > > > > > >>>>>>>> - Server configuration (e. g. altering the behavior of a > > > config > > > > > >>>>>> property) > > > > > >>>>>>>> - Emitted metrics > > > > > >>>>>>>> - Other major changes, judged by the discretion of Druid > > > > > >>> committers > > > > > >>>>>>>> > > > > > >>>>>>>> Some of it is subjective, but it has been in place for a > > > while, > > > > > >> so > > > > > >>>>> it's > > > > > >>>>>>> at > > > > > >>>>>>>> least something we are relatively familiar with. > > > > > >>>>>>>> > > > > > >>>>>>>> On Mon, Jan 7, 2019 at 11:32 AM Julian Hyde < > > jh...@apache.org > > > > > > > > > >>>>> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>>> Small contributions don’t need any design review, whereas > > > large > > > > > >>>>>>>>> contributions need significant review. I don’t think we > > > should > > > > > >>>>> require > > > > > >>>>>>> an > > > > > >>>>>>>>> additional step for those (many) small contributions. But > > who > > > > > >>>>> decides > > > > > >>>>>>>>> whether a contribution fits into the small or large > > category? > > > > > >>>>>>>>> > > > > > >>>>>>>>> I think the solution is for authors to log a case (or > send > > an > > > > > >>>>> email to > > > > > >>>>>>>>> dev) before they start work on any contribution. Then > > > > > >> committers > > > > > >>>>> can > > > > > >>>>>>>>> request a more heavy-weight process if they think it is > > > needed. > > > > > >>>>>>>>> > > > > > >>>>>>>>> Julian > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>>> On Jan 7, 2019, at 11:24 AM, Gian Merlino < > > g...@apache.org> > > > > > >>>>> wrote: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> It sounds like splitting design from code review is a > > common > > > > > >>> theme > > > > > >>>>>> in a > > > > > >>>>>>>>> few > > > > > >>>>>>>>>> of the posts here. How does everyone feel about making a > > > point > > > > > >>> of > > > > > >>>>>>>>>> encouraging design reviews to be done as issues, > separate > > > from > > > > > >>> the > > > > > >>>>>> pull > > > > > >>>>>>>>>> request, with the expectations that (1) the design > review > > > > > >> issue > > > > > >>>>>>>>>> ("proposal") should generally appear somewhat _before_ > the > > > > > >> pull > > > > > >>>>>>> request; > > > > > >>>>>>>>>> (2) pull requests should _not_ have design review happen > > on > > > > > >>> them, > > > > > >>>>>>> meaning > > > > > >>>>>>>>>> there should no longer be PRs with design review tags, > and > > > we > > > > > >>>>> should > > > > > >>>>>>> move > > > > > >>>>>>>>>> the design review approval process to the issue rather > > than > > > > > >> the > > > > > >>>>> PR. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> For (1), even if we encourage design review discussions > to > > > > > >> start > > > > > >>>>>>> before a > > > > > >>>>>>>>>> pull request appears, I don't see an issue with them > > running > > > > > >>>>>>> concurrently > > > > > >>>>>>>>>> for a while at some point. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> On Thu, Jan 3, 2019 at 5:35 PM Jonathan Wei < > > > > > >> jon...@apache.org> > > > > > >>>>>> wrote: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>> Thanks for raising these concerns! > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> My initial thoughts: > > > > > >>>>>>>>>>> - I agree that separation of design review and > code-level > > > > > >>> review > > > > > >>>>> for > > > > > >>>>>>>>> major > > > > > >>>>>>>>>>> changes would be more efficient > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> - I agree that a clear, more formalized process for > > > handling > > > > > >>>>> major > > > > > >>>>>>>>> changes > > > > > >>>>>>>>>>> would be helpful for contributors: > > > > > >>>>>>>>>>> - Define what is considered a major change > > > > > >>>>>>>>>>> - Define a standard proposal structure, KIP-style > > proposal > > > > > >>> format > > > > > >>>>>>>>> sounds > > > > > >>>>>>>>>>> good to me > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> - I think it's too rigid to have a policy of "no code > at > > > all > > > > > >>> with > > > > > >>>>>> the > > > > > >>>>>>>>>>> initial proposal" > > > > > >>>>>>>>>>> - Code samples can be useful references for > understanding > > > > > >>> aspects > > > > > >>>>>> of a > > > > > >>>>>>>>>>> design > > > > > >>>>>>>>>>> - In some cases it's necessary to run experiments to > > fully > > > > > >>>>>> understand > > > > > >>>>>>> a > > > > > >>>>>>>>>>> problem and determine an appropriate design, or to > > > determine > > > > > >>>>> whether > > > > > >>>>>>>>>>> something is even worth doing before committing to the > > work > > > > > >> of > > > > > >>>>>>> fleshing > > > > > >>>>>>>>> out > > > > > >>>>>>>>>>> a proposal, prototype code is a natural outcome of that > > and > > > > > >> I'm > > > > > >>>>> not > > > > > >>>>>>>>> against > > > > > >>>>>>>>>>> someone providing such code for reference > > > > > >>>>>>>>>>> - I tend to view design/code as things that are often > > > > > >> developed > > > > > >>>>>>>>>>> simultaneously in an intertwined way > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> Let's not be naive this is very rare that a > contributor > > > will > > > > > >>>>> accept > > > > > >>>>>>>>> that > > > > > >>>>>>>>>>> his work is to be thrown, usually devs takes coding as > > > > > >> personal > > > > > >>>>>>> creation > > > > > >>>>>>>>>>> and they get attached to it. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> If we have a clear review process that emphasizes the > > need > > > > > >> for > > > > > >>>>> early > > > > > >>>>>>>>>>> consensus building, with separate design and code > review, > > > > > >> then > > > > > >>> I > > > > > >>>>>> feel > > > > > >>>>>>>>> we've > > > > > >>>>>>>>>>> done enough and don't need a hard rule against having > > some > > > > > >> code > > > > > >>>>>> linked > > > > > >>>>>>>>> with > > > > > >>>>>>>>>>> the initial proposal. If a potential contributor then > > still > > > > > >>>>> wants to > > > > > >>>>>>> go > > > > > >>>>>>>>>>> ahead and write a lot of code that may be rejected or > > > change > > > > > >>>>>>>>> significantly, > > > > > >>>>>>>>>>> the risks were made clear. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> Once code is written hard to think abstract. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> I can see the validity of the concern, but I personally > > > don't > > > > > >>>>> see it > > > > > >>>>>>> as > > > > > >>>>>>>>> a > > > > > >>>>>>>>>>> huge risk. My impression from the Druid PR reviews I've > > > seen > > > > > >> is > > > > > >>>>> that > > > > > >>>>>>> our > > > > > >>>>>>>>>>> reviewers are able to keep abstract design vs. > > > implementation > > > > > >>>>>> details > > > > > >>>>>>>>>>> separate and consider alternate designs when reviewing. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> To summarize I think it's probably enough to have a > > policy > > > > > >>> along > > > > > >>>>> the > > > > > >>>>>>>>> lines > > > > > >>>>>>>>>>> of: > > > > > >>>>>>>>>>> - Create more formalized guidelines for proposals and > > what > > > > > >>>>> changes > > > > > >>>>>>>>> require > > > > > >>>>>>>>>>> proposals > > > > > >>>>>>>>>>> - Separate design and code review for major changes, > with > > > > > >>> design > > > > > >>>>>>> review > > > > > >>>>>>>>>>> first, code-level review after reaching consensus on > the > > > > > >>> design. > > > > > >>>>>>>>>>> - Code before the design review is completed is just > for > > > > > >>>>> reference, > > > > > >>>>>>> not > > > > > >>>>>>>>>>> regarded as a candidate for review/merging. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> - Jon > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> On Thu, Jan 3, 2019 at 12:48 PM Slim Bouguerra < > > > > > >>>>>>>>> slim.bougue...@gmail.com> > > > > > >>>>>>>>>>> wrote: > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> On Thu, Jan 3, 2019 at 12:16 PM Clint Wylie < > > > > > >>>>> clint.wy...@imply.io> > > > > > >>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> I am definitely biased in this matter as an owner of > > > > > >> another > > > > > >>>>> large > > > > > >>>>>>> PR > > > > > >>>>>>>>>>>> that > > > > > >>>>>>>>>>>>> wasn't preceded by a direct proposal or dev list > > > > > >> discussion, > > > > > >>>>> and > > > > > >>>>>> in > > > > > >>>>>>>>>>>> general > > > > > >>>>>>>>>>>>> I agree that proposal first is usually better, but I > > > think > > > > > >> in > > > > > >>>>> some > > > > > >>>>>>>>>>> rarer > > > > > >>>>>>>>>>>>> cases approaching a problem code first *is* the most > > > > > >>>>> appropriate > > > > > >>>>>> way > > > > > >>>>>>>>> to > > > > > >>>>>>>>>>>>> have a discussion. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> I am wondering here what is the case where code first > is > > > > > >>> better? > > > > > >>>>>>>>>>>> In general when you are writing code you have an idea > > > about > > > > > >>> what > > > > > >>>>>> you > > > > > >>>>>>>>> want > > > > > >>>>>>>>>>>> to change, why you want to change and why you want to > > > change > > > > > >>> it. > > > > > >>>>>>>>>>>> I do not see what is wrong with sharing this primitive > > > ideas > > > > > >>> and > > > > > >>>>>>>>> thoughts > > > > > >>>>>>>>>>>> as an abstract proposal (at least to avoid > overlapping) > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> I see nothing wrong with it so long as the author > > > > > >>>>>>>>>>>>> accepts that the PR is treated as a combined proposal > > and > > > > > >>>>> proof of > > > > > >>>>>>>>>>>> concept, > > > > > >>>>>>>>>>>>> and fair game to be radically changed via discussion > or > > > > > >> even > > > > > >>>>>>> rejected, > > > > > >>>>>>>>>>>>> which sounds like Gian's attitude on the matter and > is > > > mine > > > > > >>> as > > > > > >>>>>> well > > > > > >>>>>>>>>>> with > > > > > >>>>>>>>>>>> my > > > > > >>>>>>>>>>>>> compression stuff. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Let's not be naive this is very rare that a > contributor > > > will > > > > > >>>>> accept > > > > > >>>>>>>>> that > > > > > >>>>>>>>>>>> his work is to be thrown, usually devs takes coding as > > > > > >>> personal > > > > > >>>>>>>>> creation > > > > > >>>>>>>>>>>> and they get attached to it. > > > > > >>>>>>>>>>>> To my point you can take a look on some old issue in > the > > > > > >> Druid > > > > > >>>>>> forum > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > https://github.com/apache/incubator-druid/pull/3755#issuecomment-265667690 > > > > > >>>>>>>>>>>> and am sure other communities have similar problems. > > > > > >>>>>>>>>>>> So leaving the door open to some side cases is not a > > good > > > > > >> idea > > > > > >>>>> in > > > > > >>>>>> my > > > > > >>>>>>>>>>>> opinion and will lead to similar issue in the future. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> This seems to me especially likely to happen in cases > > > > > >>>>>>>>>>>>> where an approach still needs proven to be a viable > > idea > > > > > >> *to > > > > > >>>>> the > > > > > >>>>>>>>>>> author*, > > > > > >>>>>>>>>>>>> so that a much more productive discussion can be had > in > > > the > > > > > >>>>> first > > > > > >>>>>>>>>>> place. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> I think there is a trade off, I don't think we want > to > > > > > >>>>> discourage > > > > > >>>>>>>>>>>>> experimentation by walling it off behind mandatory > > > > > >>> discussions > > > > > >>>>>>> before > > > > > >>>>>>>>>>> it > > > > > >>>>>>>>>>>>> can even start, but I do think formalizing the > process > > > for > > > > > >>>>> large > > > > > >>>>>>>>>>> changes > > > > > >>>>>>>>>>>> is > > > > > >>>>>>>>>>>>> a good thing, especially since we probably can't > expect > > > the > > > > > >>>>> wider > > > > > >>>>>>>>>>>> community > > > > > >>>>>>>>>>>>> to have the same attitude towards a large PR getting > > > > > >>> discarded > > > > > >>>>> as > > > > > >>>>>> a > > > > > >>>>>>>>>>>>> committer might. I think the Kafka approach is > > > reasonable, > > > > > >> a > > > > > >>>>> bit > > > > > >>>>>>> more > > > > > >>>>>>>>>>>>> formal than our design review process but not > > > overbearing. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Can you please explain what is overbearing ? what can > be > > > > > >>>>> changed to > > > > > >>>>>>>>> make > > > > > >>>>>>>>>>> it > > > > > >>>>>>>>>>>> easy ? > > > > > >>>>>>>>>>>> Most of the points are kind of the actual questions > that > > > you > > > > > >>>>> want > > > > > >>>>>> to > > > > > >>>>>>>>>>>> address before hand anyway isn't ? > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Going code first > > > > > >>>>>>>>>>>>> should be in general discouraged, but when it does > > > happen, > > > > > >> it > > > > > >>>>>> seems > > > > > >>>>>>>>>>> like > > > > > >>>>>>>>>>>>> opening DIP/an issue/starting a mailing list thread > or > > > > > >>>>> whatever we > > > > > >>>>>>> go > > > > > >>>>>>>>>>>> with > > > > > >>>>>>>>>>>>> to have a more high level design discussion alongside > > the > > > > > >>>>>> reference > > > > > >>>>>>> PR > > > > > >>>>>>>>>>>>> could alleviate some of these complaints? > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> What are the complaints ? > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> +1 for "DIP" heh, I think making > > > > > >>>>>>>>>>>>> them in the form of github issues is probably > > > appropriate, > > > > > >>>>> with a > > > > > >>>>>>> dev > > > > > >>>>>>>>>>>> list > > > > > >>>>>>>>>>>>> thread to announce them perhaps? > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> I think github issue with [Proposal] header like > > > > > >>>>>>>>>>>> https://github.com/apache/incubator-druid/issues/4349 > > is > > > > > >> good > > > > > >>>>> to > > > > > >>>>>> me, > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Thanks! > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> On Thu, Jan 3, 2019 at 10:28 AM Slim Bouguerra < > > > > > >>>>> bs...@apache.org> > > > > > >>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Thanks everyone for interacting with this thread. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> The fact that i, Roman, Jihoon and others in the > past > > > (FJ > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > https://groups.google.com/forum/#!msg/druid-user/gkUEsAYIfBA/6B2GJdLkAgAJ > > > > > >>>>> ) > > > > > >>>>>>>>>>>>>> raised this point indicates that PRs without a > > proposal > > > > > >> are > > > > > >>>>>> indeed > > > > > >>>>>>> an > > > > > >>>>>>>>>>>>> issue > > > > > >>>>>>>>>>>>>> and we need to solve it. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Something Similar to KIP maybe called DIPs is fine > > with > > > > > >> me. > > > > > >>>>>>>>>>>>>> What i strive to see is the following: > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> [Step 1] formalize what is the kind of work that > > needs a > > > > > >>>>> formal > > > > > >>>>>>>>>>>>> Proposal, I > > > > > >>>>>>>>>>>>>> think Roman and Jihoon has already covered that > pretty > > > > > >> well. > > > > > >>>>> am > > > > > >>>>>> +1 > > > > > >>>>>>> on > > > > > >>>>>>>>>>>>> that. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/9b30d893bdb6bb633cf6a9a700183ffb5b98f115330531a55328ac77@%3Cdev.druid.apache.org%3E > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> I am strongly in favor of the separation of Proposal > > > > > >> Review > > > > > >>>>> and > > > > > >>>>>>>>>>> (later) > > > > > >>>>>>>>>>>>>> Code review PRs. My main reasons: > > > > > >>>>>>>>>>>>>> Most importantly code reviewing will introduce lot > of > > > > > >> noise > > > > > >>>>> and > > > > > >>>>>>> will > > > > > >>>>>>>>>>>>>> ultimately make the GitHub page unreadable. > > > > > >>>>>>>>>>>>>> Avoid overlapping of work. > > > > > >>>>>>>>>>>>>> Once code is written hard to think abstract. > > > > > >>>>>>>>>>>>>> Separate page for Design review later can always be > > used > > > > > >> it > > > > > >>>>> as a > > > > > >>>>>>>>>>> Design > > > > > >>>>>>>>>>>>>> document that is readable and code free-ish. > > > > > >>>>>>>>>>>>>> As i said the goal of this first round is to see if > > the > > > > > >>>>> community > > > > > >>>>>>>>>>> agree > > > > > >>>>>>>>>>>>>> about such change, then make the process of design > > more > > > > > >>>>> inclusive > > > > > >>>>>>>>>>> thus > > > > > >>>>>>>>>>>>>> other contributors can submit a counter proposals. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> [Step 2] IF everybody agree about that point Step 2 > is > > > to > > > > > >>>>> define > > > > > >>>>>>>>>>> which > > > > > >>>>>>>>>>>>>> medium is used to Publish a primitive form of a CODE > > > FREE > > > > > >>>>>> Abstract > > > > > >>>>>>>>>>>>> Proposal > > > > > >>>>>>>>>>>>>> containing at least the following bullet points. > > > > > >>>>>>>>>>>>>> - The problem description and motivation > > > > > >>>>>>>>>>>>>> - Overview of the proposed change > > > > > >>>>>>>>>>>>>> - Operational impact (compatibility/ plans to > > upgrades) > > > > > >>> public > > > > > >>>>>> API > > > > > >>>>>>>>>>>>> changes, > > > > > >>>>>>>>>>>>>> configuration changes, algorithm, and so on > > > > > >>>>>>>>>>>>>> - Expected benefits and drawbacks > > > > > >>>>>>>>>>>>>> - Rationale and alternatives > > > > > >>>>>>>>>>>>>> - Estimate Time to Deliver if possible. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> The way i think this can be is a Github issue where > > > member > > > > > >>> of > > > > > >>>>> the > > > > > >>>>>>>>>>>>> community > > > > > >>>>>>>>>>>>>> will interact via comments and the author will be > > > updating > > > > > >>> the > > > > > >>>>>>>>>>>>> description > > > > > >>>>>>>>>>>>>> in the light of comments provided by the community. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> During and near the end of the design discussions > the > > > > > >>> author/s > > > > > >>>>>> can > > > > > >>>>>>>>>>>> start > > > > > >>>>>>>>>>>>>> writing POCs to help guide the review process this > > > > > >> naturally > > > > > >>>>> will > > > > > >>>>>>> be > > > > > >>>>>>>>>>> a > > > > > >>>>>>>>>>>>> Pull > > > > > >>>>>>>>>>>>>> request with actual code. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> *Now the most important thing is that we need to > agree > > > > > >> that > > > > > >>>>> any > > > > > >>>>>>> work > > > > > >>>>>>>>>>>> that > > > > > >>>>>>>>>>>>>> does not align with this formal process will be > > ignored > > > > > >> and > > > > > >>>>> the > > > > > >>>>>>>>>>> author > > > > > >>>>>>>>>>>>> will > > > > > >>>>>>>>>>>>>> be asked to start with a DIP* > > > > > >>>>>>>>>>>>>> *That is what i meant with “If it didn’t happen on > > the > > > > > >>>>> mailing > > > > > >>>>>>> list, > > > > > >>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>> didn’t happen.”* > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Thanks and happy coding! > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> On Wed, Jan 2, 2019 at 6:47 PM Gian Merlino < > > > > > >>> g...@apache.org> > > > > > >>>>>>> wrote: > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> One of the advantages I see with a more formal > > process > > > is > > > > > >>>>> (like > > > > > >>>>>>>>>>> Kafka > > > > > >>>>>>>>>>>>>> KIPs) > > > > > >>>>>>>>>>>>>>> is that it levels the playing field a bit and sets > > some > > > > > >>>>> ground > > > > > >>>>>>>>>>> rules > > > > > >>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>> working together. In a way it can help encourage > > > > > >>>>> contributions > > > > > >>>>>> by > > > > > >>>>>>>>>>>>> making > > > > > >>>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>>> clear what is expected of potential contributors. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> We have a design review process today that is not > as > > > > > >> formal > > > > > >>>>> as > > > > > >>>>>>>>>>> KIPs, > > > > > >>>>>>>>>>>>> but > > > > > >>>>>>>>>>>>>> is > > > > > >>>>>>>>>>>>>>> somewhat heavier than the one you describe. Maybe > we > > > > > >> could > > > > > >>>>> tweak > > > > > >>>>>>>>>>> our > > > > > >>>>>>>>>>>>>>> current one by starting to do design reviews > > separately > > > > > >>> from > > > > > >>>>>> PRs. > > > > > >>>>>>>>>>>> i.e., > > > > > >>>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>> anything that meets our 'design review' criteria, > do > > > that > > > > > >>> on > > > > > >>>>> the > > > > > >>>>>>>>>>> dev > > > > > >>>>>>>>>>>>> list > > > > > >>>>>>>>>>>>>>> or in a separate issue, and keep the PR focused on > > > > > >>> code-level > > > > > >>>>>>>>>>> stuff. > > > > > >>>>>>>>>>>>> That > > > > > >>>>>>>>>>>>>>> way we don't end up trying to do both at once. And > it > > > > > >> makes > > > > > >>>>> it > > > > > >>>>>>>>>>> easier > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>> start talking about design before the code is > ready, > > > > > >> which > > > > > >>>>> would > > > > > >>>>>>> be > > > > > >>>>>>>>>>>>>> better. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> On Wed, Jan 2, 2019 at 6:10 PM Julian Hyde < > > > > > >>> jh...@apache.org > > > > > >>>>>> > > > > > >>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> It’s really hard to say no to a contribution when > > > > > >> someone > > > > > >>>>> has > > > > > >>>>>> put > > > > > >>>>>>>>>>>> in > > > > > >>>>>>>>>>>>> a > > > > > >>>>>>>>>>>>>>>> significant amount of work. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> The following approach is simple and works really > > > well: > > > > > >>>>> Before > > > > > >>>>>>>>>>> you > > > > > >>>>>>>>>>>>>> start > > > > > >>>>>>>>>>>>>>>> work, log a case, describing the problem. When you > > > have > > > > > >>> some > > > > > >>>>>>>>>>> ideas > > > > > >>>>>>>>>>>>>> about > > > > > >>>>>>>>>>>>>>>> design, add those to the case. When you have a > code > > > > > >>> branch, > > > > > >>>>> add > > > > > >>>>>>>>>>> its > > > > > >>>>>>>>>>>>> URL > > > > > >>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>> the case. And so forth. At any point in the > > > proceedings, > > > > > >>>>> people > > > > > >>>>>>>>>>> can > > > > > >>>>>>>>>>>>>> chime > > > > > >>>>>>>>>>>>>>>> in with their opinions. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> In my opinion, a formal “design review” process is > > not > > > > > >>>>>> necessary. > > > > > >>>>>>>>>>>>> Just > > > > > >>>>>>>>>>>>>>>> build consensus iteratively, by starting the > > > > > >> conversation > > > > > >>>>> early > > > > > >>>>>>>>>>> in > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>> process. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Julian > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> On Jan 2, 2019, at 12:37 PM, Gian Merlino < > > > > > >>> g...@apache.org > > > > > >>>>>> > > > > > >>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> In this particular case: please consider the PR > as > > a > > > > > >>>>> proposal. > > > > > >>>>>>>>>>>>> Don't > > > > > >>>>>>>>>>>>>>> feel > > > > > >>>>>>>>>>>>>>>>> like just because there is code there that takes > a > > > > > >>> certain > > > > > >>>>>>>>>>>>> approach, > > > > > >>>>>>>>>>>>>>> that > > > > > >>>>>>>>>>>>>>>>> the approach is somehow sacred. I had to > implement > > > > > >>>>> something > > > > > >>>>>> to > > > > > >>>>>>>>>>>>>>>> crystallize > > > > > >>>>>>>>>>>>>>>>> my own thinking about how the problem could be > > > > > >>> approached. > > > > > >>>>> I > > > > > >>>>>>>>>>>> won't > > > > > >>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>>>> disappointed if, as a community, we decide a > > > different > > > > > >>>>>>>>>>> direction > > > > > >>>>>>>>>>>> is > > > > > >>>>>>>>>>>>>>>> better > > > > > >>>>>>>>>>>>>>>>> and the code all gets thrown away. That's one of > > the > > > > > >>>>> reasons > > > > > >>>>>>>>>>>> that I > > > > > >>>>>>>>>>>>>>>> removed > > > > > >>>>>>>>>>>>>>>>> the 0.14.0 milestone that was added to the patch. > > (I > > > > > >>> don't > > > > > >>>>>> want > > > > > >>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>> rush > > > > > >>>>>>>>>>>>>>>> it, > > > > > >>>>>>>>>>>>>>>>> nor do I think that's a good idea.) > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> In general: Sounds like we could do with some > more > > > > > >>>>>>>>>>> formalization > > > > > >>>>>>>>>>>>>> around > > > > > >>>>>>>>>>>>>>>>> what a proposal looks like, which sorts of > changes > > > need > > > > > >>>>> one, > > > > > >>>>>>>>>>> and > > > > > >>>>>>>>>>>>> when > > > > > >>>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>> the dev cycle it is appropriate. FWIW I think > > Kafka's > > > > > >>>>> process > > > > > >>>>>>>>>>> is > > > > > >>>>>>>>>>>>> more > > > > > >>>>>>>>>>>>>>> or > > > > > >>>>>>>>>>>>>>>>> less fine, and would be okay with adopting it for > > > Druid > > > > > >>> if > > > > > >>>>>>>>>>> people > > > > > >>>>>>>>>>>>>> like > > > > > >>>>>>>>>>>>>>>> it. > > > > > >>>>>>>>>>>>>>>>> Right now our standards for what requires a > "design > > > > > >>> review" > > > > > >>>>>> are > > > > > >>>>>>>>>>>>> very > > > > > >>>>>>>>>>>>>>>>> similar to the Kafka community standards for what > > > > > >>> requires > > > > > >>>>> a > > > > > >>>>>>>>>>> KIP, > > > > > >>>>>>>>>>>>> so > > > > > >>>>>>>>>>>>>> we > > > > > >>>>>>>>>>>>>>>>> have some familiarity with those concepts. > However > > we > > > > > >>> don't > > > > > >>>>>>>>>>>>> separate > > > > > >>>>>>>>>>>>>> PR > > > > > >>>>>>>>>>>>>>>>> review and proposal discussion as strictly as > they > > > do, > > > > > >>>>> which > > > > > >>>>>>>>>>>> seems > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>>>> the foundation for the feeling of exclusion that > is > > > > > >> being > > > > > >>>>> felt > > > > > >>>>>>>>>>>>> here. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Separately: I just redid the description on > > > > > >>>>>>>>>>>>>>>>> > > https://github.com/apache/incubator-druid/pull/6794 > > > to > > > > > >>> be > > > > > >>>>>> more > > > > > >>>>>>>>>>>>>>>> proposal-y. > > > > > >>>>>>>>>>>>>>>>> I followed the KIP style: > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > > >>>>>>>>>>>>>>>> . > > > > > >>>>>>>>>>>>>>>>> Please refresh the page and see if it looks more > > > > > >> useful. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Gian > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> On Wed, Jan 2, 2019 at 10:52 AM Julian Hyde < > > > > > >>>>> jh...@apache.org > > > > > >>>>>>> > > > > > >>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Slim, > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> I agree with your points that offline > development > > is > > > > > >> bad > > > > > >>>>> for > > > > > >>>>>>>>>>>>>>> community. > > > > > >>>>>>>>>>>>>>>>>> But I don’t think you need much mentor help. You > > > have > > > > > >>>>> raised > > > > > >>>>>>>>>>>> valid > > > > > >>>>>>>>>>>>>>>> issues > > > > > >>>>>>>>>>>>>>>>>> and the Druid community needs to decide what its > > > > > >>>>> development > > > > > >>>>>>>>>>>>>> practices > > > > > >>>>>>>>>>>>>>>>>> should be. > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Julian > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> On Jan 2, 2019, at 10:29 AM, Slim Bouguerra < > > > > > >>>>>>>>>>> bs...@apache.org> > > > > > >>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> Hello everyone and hope you all have very good > > > > > >>> holidays. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> First, this email is not directed on the author > > or > > > > > >> the > > > > > >>> PR > > > > > >>>>>>>>>>>>>>>>>>> > > > https://github.com/apache/incubator-druid/pull/6794 > > > > > >>> it > > > > > >>>>>>>>>>> self, > > > > > >>>>>>>>>>>>> but > > > > > >>>>>>>>>>>>>> i > > > > > >>>>>>>>>>>>>>>> see > > > > > >>>>>>>>>>>>>>>>>>> this PR as a perfect example. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> One of the foundation of Apache Way or what i > > would > > > > > >>>>> simply > > > > > >>>>>>>>>>> call > > > > > >>>>>>>>>>>>>> open > > > > > >>>>>>>>>>>>>>>>>> source > > > > > >>>>>>>>>>>>>>>>>>> community driven development is that "Technical > > > > > >>> decisions > > > > > >>>>>> are > > > > > >>>>>>>>>>>>>>>> discussed, > > > > > >>>>>>>>>>>>>>>>>>> decided, and archived publicly. > > > > > >>>>>>>>>>>>>>>>>>> developpement" > > > > > >>>>>>>>>>>>>>>>>>> Which means that big technical changes such as > > the > > > > > >> one > > > > > >>>>>>>>>>> brought > > > > > >>>>>>>>>>>>> by > > > > > >>>>>>>>>>>>>>>> #/6794 > > > > > >>>>>>>>>>>>>>>>>>> should have started as a proposal and round of > > > > > >>>>> discussions > > > > > >>>>>>>>>>>> about > > > > > >>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>> major > > > > > >>>>>>>>>>>>>>>>>>> changes designs not as 11K line of code. > > > > > >>>>>>>>>>>>>>>>>>> I believe such openness will promote a lot of > > good > > > > > >>>>> benefits > > > > > >>>>>>>>>>>> such > > > > > >>>>>>>>>>>>>> as: > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> - ensures community health and growth. > > > > > >>>>>>>>>>>>>>>>>>> - ensures everyone can participate not only the > > > > > >> authors > > > > > >>>>> and > > > > > >>>>>>>>>>> his > > > > > >>>>>>>>>>>>>>>>>> co-workers. > > > > > >>>>>>>>>>>>>>>>>>> - ensures that the project is driven by the > > > community > > > > > >>> and > > > > > >>>>>>>>>>> not a > > > > > >>>>>>>>>>>>>> given > > > > > >>>>>>>>>>>>>>>>>>> company or an individual. > > > > > >>>>>>>>>>>>>>>>>>> - ensures that there is consensus (not saying > > 100% > > > > > >>>>>>>>>>> agreement;) > > > > > >>>>>>>>>>>>>>> however > > > > > >>>>>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>>>>>>> means that all individuals will accept the > > current > > > > > >>>>> progress > > > > > >>>>>>>>>>> on > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>> project > > > > > >>>>>>>>>>>>>>>>>>> until some better proposal is put forth. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> Personally such BIG offline PR makes me feel > > > excluded > > > > > >>> and > > > > > >>>>>>>>>>>> doesn't > > > > > >>>>>>>>>>>>>>> give > > > > > >>>>>>>>>>>>>>>>>> me a > > > > > >>>>>>>>>>>>>>>>>>> sense that i belong to a community at all. > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> To prevent such off list development i think > as a > > > > > >> Druid > > > > > >>>>>>>>>>>> Community > > > > > >>>>>>>>>>>>>> we > > > > > >>>>>>>>>>>>>>>> need > > > > > >>>>>>>>>>>>>>>>>>> to stick to the apache way “If it didn’t happen > > on > > > > > >> the > > > > > >>>>>>>>>>> mailing > > > > > >>>>>>>>>>>>>> list, > > > > > >>>>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>>>>>>> didn’t happen.” > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> I would appreciate if some of the Apache mentor > > > help > > > > > >>> with > > > > > >>>>>>>>>>> this. > > > > > >>>>>>>>>>>>>>>>>>> Thanks > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>> > > > > > >>> > > > --------------------------------------------------------------------- > > > > > >>>>>>>>>>>>>>>>>> To unsubscribe, e-mail: > > > > > >>> dev-unsubscr...@druid.apache.org > > > > > >>>>>>>>>>>>>>>>>> For additional commands, e-mail: > > > > > >>>>> dev-h...@druid.apache.org > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>> > > > > > >> > > > --------------------------------------------------------------------- > > > > > >>>>>>>>>>>>>>>> To unsubscribe, e-mail: > > > > > >> dev-unsubscr...@druid.apache.org > > > > > >>>>>>>>>>>>>>>> For additional commands, e-mail: > > > > > >>> dev-h...@druid.apache.org > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> -- > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> B-Slim > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>> > > > > > >>> > > > > > > > > > _______/\/\/\_______/\/\/\_______/\/\/\_______/\/\/\_______/\/\/\_______ > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>> > > > > --------------------------------------------------------------------- > > > > > >>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > > > > > >>>>>>>>> For additional commands, e-mail: > dev-h...@druid.apache.org > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>> > > > --------------------------------------------------------------------- > > > > > >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > > > > > >>>>>>> For additional commands, e-mail: dev-h...@druid.apache.org > > > > > >>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > > > > > For additional commands, e-mail: dev-h...@druid.apache.org > > > > > > > > > > > > > > > > > > > >
