> > What do community members think about also making a requirement that > > "Design Review" PRs and proposals are reviewed by at least two people with > > different affiliation? > This seems pretty reasonable to me. I haven't found anything in Apache > voting procedure docs (https://www.apache.org/foundation/voting.html) that > seems to explicitly forbid something like this yet at least.
On the other hand, my understanding of the Apache Way would mean that this kind of rule doesn't make sense. In particular from https://www.apache.org/foundation/how-it-works.html: "We firmly believe in hats. Your role at the ASF is one assigned to you personally, and is bestowed on you by your peers. It is not tied to your job or current employer or company." That sentiment seems incompatible with making explicit rules about organizational diversity in voting. I have also heard a few people say things like: people are supposed to represent themselves, not their employers. Obviously, though, people's actions and opinions are influenced by their employer. IMO, a better way to approach a potential problem there is that if people from a particular organization end up behaving abusively, then PMC members from other organizations (or in extreme situations the Apache board itself) should tell them to knock it off. And then do more serious things if the inappropriate behavior continues. This kind The other thing you brought up, promoting more shared ownership and awareness, I am hopeful that separating proposals from PRs will help with that. One reason is that it takes much less time to understand a well written proposal than it takes to understand a PR. Code is very clear but it is also very verbose and takes a while to read and understand. So in theory changing how we operate, in this way, should promote more understanding of more people across the code base. On Thu, Jan 10, 2019 at 1:34 AM Clint Wylie <clint.wy...@imply.io> wrote: > > > > What do community members think about also making a requirement that > > "Design Review" PRs and proposals are reviewed by at least two people > with > > different affiliation? > > > This seems pretty reasonable to me. I haven't found anything in Apache > voting procedure docs (https://www.apache.org/foundation/voting.html) that > seems to explicitly forbid something like this yet at least. > > On Wed, Jan 9, 2019 at 8:52 PM Roman Leventov <leventov...@gmail.com> > wrote: > > > What do community members think about also making a requirement that > > "Design Review" PRs and proposals are reviewed by at least two people > with > > different affiliation? IMO it's a good idea, because it ensures that > > different interests are taken into account. Also it pushes people to > engage > > with work done in other parts of Druid, improving shared code owning and > > awareness. > > > > Or it's against the Apache Way because it's assumed that there are no > > company boundaries within the community? > > > > On Thu, 10 Jan 2019 at 11:45, Roman Leventov <leventov...@gmail.com> > > wrote: > > > > > I see two important reasons why it makes sense to file an issue and > > > probably announce it in the mailing list, before writing a lot of code, > > > despite not having a clear picture of what it will be and any > performance > > > data: > > > 1) somebody could already work on this problem privately in parallel, > it > > > allows to avoid clash in people's works > > > 2) some people could quickly think about the problem field and share > > > high-level ideas that could wildly change the direction in which the > > author > > > (the person who is going to write code) will move in his work from > early > > on. > > > > > > Jihoon in > > > > > > https://lists.apache.org/thread.html/e007fbf362c2a870a2d88d04431789289807e00fd91d087559a01d1f@%3Cdev.druid.apache.org%3E > > and > > > later Gian in this thread suggested that _every_ piece of work that > > should > > > be labelled as "Design Review" according to the current rules should be > > > accompanied by an issue. I don't agree with this, there are some PRs as > > > small as a few dozens of lines of code, that add some configuration > > > parameter and therefore should be labelled "Design Review". I don't > > thing a > > > separate proposal issue is needed for them, and even for a little > larger > > > PRs too. > > > > > > For the same reason, I also don't see the point of renaming "Design > > > Review" into "Proposal", as well as separating "Design Review" into > > > "Proposal" and something like "API Review". I think a single "Design > > > Review" tag handles it well. > > > > > > Gian mentioned an idea that PRs that follow a "Design Review" proposal > > > issue shouldn't be "Design Review" themselves. I don't agree with > this, I > > > think that actual code and performance data are important inputs that > > > should be re-evaluated at least by two people. I even think that it's > > very > > > desirable that at least two people read _each line of production code_ > in > > > large PRs, although it's not what was done historically in Druid, > because > > > large bodies of newly added code, with whole new classes and subsystems > > > added, are also coincidentally tested worse than already existing > classes > > > and subsystems, including in production. It seems to me that those huge > > > code influxes is a major source of bugs, that could later take years to > > > squeeze out from the codebase. > > > > > > On Wed, 9 Jan 2019 at 08:24, Clint Wylie <clint.wy...@imply.io> wrote: > > > > > >> Apologies for the delayed response. > > >> > > >> On Thu, Jan 3, 2019 at 12:48 PM Slim Bouguerra < > > slim.bougue...@gmail.com> > > >> wrote: > > >> > > >> > I am wondering here what is the case where code first is better? > > >> > > > >> > > >> I don't think it's wrong to share ideas as early as possible, and > after > > >> this discussion I think I am in favor of it too. I just meant that I > > don't > > >> think it's always necessarily the most productive discussion until > code > > >> exists sometimes, with the types of thing I am thinking of are almost > > >> entirely limited to cases where things might sound good to anyone on > > paper > > >> but in reality need a large amount of experiments conducted and > > >> observations collected to determine that something is actually worth > > >> doing, > > >> which I imagine is mainly things like reworking internals for > > performance > > >> improvements. > > >> > > >> In the case of my combined proposal PR, I needed to prove that the > > thing I > > >> was working on was a good idea... and it wasn't directly. But I came > up > > >> with another idea during the course of experiment turned into > something > > >> compelling, so an initial proposal would have looked quite a lot > > different > > >> than what I ended up with. Once I had proven to myself that it was a > > good > > >> idea, then I was comfortable sharing with the wider community. I'm not > > >> certain how this would play out in an always proposal first model, > maybe > > >> the first proposal exists, I personally reject it after updating with > > >> experiment results show it's a bad idea, continue experimenting and > > raise > > >> a > > >> new one after the experiments start looking promising? > > >> > > >> > > >> > Let's not be naive this is very rare that a contributor will accept > > that > > >> > his work is to be thrown, usually devs takes coding as personal > > creation > > >> > and they get attached to it. > > >> > > > >> > > >> I agree, just because a handful of the committers have this attitude, > it > > >> isn't fair to expect the wider community to also, that's why I am in > > favor > > >> of formalizing the process. > > >> > > >> Can you please explain what is overbearing ? what can be changed to > make > > >> it > > >> > easy ? > > >> > Most of the points are kind of the actual questions that you want to > > >> > address before hand anyway isn't ? > > >> > > > >> > > >> Sorry for the confusion, I said it's "not overbearing", I think it's > > fine. > > >> > > >> What are the complaints ? > > >> > > >> > > >> Is this and other previous threads not a complaint about opening a > large > > >> PR > > >> without a proposal? :) I just mean that formalizing the process, even > > if a > > >> proposal has a reference PR opened with it near concurrently, could > > >> prevent > > >> these discussions from happening in the future because ground rules > have > > >> been set and we are all on the same page I guess. > > >> > > >> I think we should also probably consider renaming the "design review" > > >> label > > >> to "proposal" or something to make it more clear that a PR is > associated > > >> with a proposal. It might also be worth considering using github > > projects > > >> for particularly large things that involve multiple follow up PRs, > but I > > >> haven't used them in much depth to know if they add anything. > > >> > > >> It seems like we are all converging on agreement to do a github issue > > >> proposal for larger changes (I would vote for announcing them on the > dev > > >> list too for more visibility), so that design review is separated from > > >> code > > >> review. I guess my main concern was not wanting to discourage > > >> experimentation by walling it off behind mandatory discussions, but > the > > >> more I think about it, it doesn't technically change much about this > > >> process, it just requires a more formal proposal to accompany any > > >> experiments that are shared as a PR. > > >> > > >> > > >> On Tue, Jan 8, 2019 at 12:28 PM Gian Merlino <g...@apache.org> wrote: > > >> > > >> > I think for us, choosing to use GitHub issues as discussion threads > > for > > >> > potential 'major' contributions would be a good idea, especially if > we > > >> > encourage people to start them before PRs show up. Definitely agree > > that > > >> > all contributors should go through the same process -- I couldn't > see > > it > > >> > working well any other way. > > >> > > > >> > On Mon, Jan 7, 2019 at 12:23 PM Julian Hyde <jh...@apache.org> > wrote: > > >> > > > >> > > Statically, yes, GitHub PRs are the same as GitHub cases. But > > >> > dynamically, > > >> > > they are different, because you can only log a PR when you have > > >> finished > > >> > > work. > > >> > > > > >> > > A lot of other Apache projects use JIRA, so there is a clear > > >> distinction > > >> > > between cases and contributions. JIRA cases, especially when > logged > > >> early > > >> > > in the lifecycle of a contribution, become long-running > conversation > > >> > > threads with a lot of community participation. If the Druid chose > to > > >> do > > >> > so, > > >> > > GitHub cases could be the same. > > >> > > > > >> > > Be careful that you do not treat “potential contributors” (by > which > > I > > >> > > presume you mean non-committers) differently from committers and > PMC > > >> > > members. Anyone starting a major piece of work should follow the > > same > > >> > > process. (Experienced committers probably have a somewhat better > > idea > > >> > what > > >> > > work will turn out to be “major”, so they get a little more > leeway.) > > >> > > > > >> > > Julian > > >> > > > > >> > > > > >> > > > On Jan 7, 2019, at 12:10 PM, Gian Merlino <g...@apache.org> > > wrote: > > >> > > > > > >> > > > I don't think there's a need to raise issues for every change: a > > >> small > > >> > > bug > > >> > > > fix or doc fix should just go straight to PR. (GitHub PRs show > up > > as > > >> > > issues > > >> > > > in the issue-search UI/API, so it's not like this means the > patch > > >> has > > >> > no > > >> > > > corresponding issue -- in a sense the PR _is_ the issue.) > > >> > > > > > >> > > > I do think it makes sense to encourage potential contributors to > > >> write > > >> > to > > >> > > > the dev list or raise an issue if they aren't sure if something > > >> would > > >> > > need > > >> > > > to go through a more heavy weight process. > > >> > > > > > >> > > > Fwiw we do have a set of 'design review' criteria already (we > had > > a > > >> > > > discussion about this a couple years ago) at: > > >> > > > http://druid.io/community/#getting-your-changes-accepted. So we > > >> > > wouldn't be > > >> > > > starting from zero on defining that. We set it up back when we > > were > > >> > > trying > > >> > > > to _streamline_ our process -- we used to require two non-author > > +1s > > >> > for > > >> > > > _every_ change, even minor ones. The introduction of design > review > > >> > > criteria > > >> > > > was meant to classify which PRs need that level of review and > > which > > >> > ones > > >> > > > are minor and can be merged with less review. I do think it > helped > > >> with > > >> > > > getting minor PRs merged more quickly. The list of criteria is, > > >> > > > > > >> > > > - Major architectural changes or API changes > > >> > > > - HTTP requests and responses (e. g. a new HTTP endpoint) > > >> > > > - Interfaces for extensions > > >> > > > - Server configuration (e. g. altering the behavior of a config > > >> > property) > > >> > > > - Emitted metrics > > >> > > > - Other major changes, judged by the discretion of Druid > > committers > > >> > > > > > >> > > > Some of it is subjective, but it has been in place for a while, > so > > >> it's > > >> > > at > > >> > > > least something we are relatively familiar with. > > >> > > > > > >> > > > On Mon, Jan 7, 2019 at 11:32 AM Julian Hyde <jh...@apache.org> > > >> wrote: > > >> > > > > > >> > > >> Small contributions don’t need any design review, whereas large > > >> > > >> contributions need significant review. I don’t think we should > > >> require > > >> > > an > > >> > > >> additional step for those (many) small contributions. But who > > >> decides > > >> > > >> whether a contribution fits into the small or large category? > > >> > > >> > > >> > > >> I think the solution is for authors to log a case (or send an > > >> email to > > >> > > >> dev) before they start work on any contribution. Then > committers > > >> can > > >> > > >> request a more heavy-weight process if they think it is needed. > > >> > > >> > > >> > > >> Julian > > >> > > >> > > >> > > >> > > >> > > >>> On Jan 7, 2019, at 11:24 AM, Gian Merlino <g...@apache.org> > > >> wrote: > > >> > > >>> > > >> > > >>> It sounds like splitting design from code review is a common > > theme > > >> > in a > > >> > > >> few > > >> > > >>> of the posts here. How does everyone feel about making a point > > of > > >> > > >>> encouraging design reviews to be done as issues, separate from > > the > > >> > pull > > >> > > >>> request, with the expectations that (1) the design review > issue > > >> > > >>> ("proposal") should generally appear somewhat _before_ the > pull > > >> > > request; > > >> > > >>> (2) pull requests should _not_ have design review happen on > > them, > > >> > > meaning > > >> > > >>> there should no longer be PRs with design review tags, and we > > >> should > > >> > > move > > >> > > >>> the design review approval process to the issue rather than > the > > >> PR. > > >> > > >>> > > >> > > >>> For (1), even if we encourage design review discussions to > start > > >> > > before a > > >> > > >>> pull request appears, I don't see an issue with them running > > >> > > concurrently > > >> > > >>> for a while at some point. > > >> > > >>> > > >> > > >>> On Thu, Jan 3, 2019 at 5:35 PM Jonathan Wei < > jon...@apache.org> > > >> > wrote: > > >> > > >>> > > >> > > >>>> Thanks for raising these concerns! > > >> > > >>>> > > >> > > >>>> My initial thoughts: > > >> > > >>>> - I agree that separation of design review and code-level > > review > > >> for > > >> > > >> major > > >> > > >>>> changes would be more efficient > > >> > > >>>> > > >> > > >>>> - I agree that a clear, more formalized process for handling > > >> major > > >> > > >> changes > > >> > > >>>> would be helpful for contributors: > > >> > > >>>> - Define what is considered a major change > > >> > > >>>> - Define a standard proposal structure, KIP-style proposal > > format > > >> > > >> sounds > > >> > > >>>> good to me > > >> > > >>>> > > >> > > >>>> - I think it's too rigid to have a policy of "no code at all > > with > > >> > the > > >> > > >>>> initial proposal" > > >> > > >>>> - Code samples can be useful references for understanding > > aspects > > >> > of a > > >> > > >>>> design > > >> > > >>>> - In some cases it's necessary to run experiments to fully > > >> > understand > > >> > > a > > >> > > >>>> problem and determine an appropriate design, or to determine > > >> whether > > >> > > >>>> something is even worth doing before committing to the work > of > > >> > > fleshing > > >> > > >> out > > >> > > >>>> a proposal, prototype code is a natural outcome of that and > I'm > > >> not > > >> > > >> against > > >> > > >>>> someone providing such code for reference > > >> > > >>>> - I tend to view design/code as things that are often > developed > > >> > > >>>> simultaneously in an intertwined way > > >> > > >>>> > > >> > > >>>>> Let's not be naive this is very rare that a contributor will > > >> accept > > >> > > >> that > > >> > > >>>> his work is to be thrown, usually devs takes coding as > personal > > >> > > creation > > >> > > >>>> and they get attached to it. > > >> > > >>>> > > >> > > >>>> If we have a clear review process that emphasizes the need > for > > >> early > > >> > > >>>> consensus building, with separate design and code review, > then > > I > > >> > feel > > >> > > >> we've > > >> > > >>>> done enough and don't need a hard rule against having some > code > > >> > linked > > >> > > >> with > > >> > > >>>> the initial proposal. If a potential contributor then still > > >> wants to > > >> > > go > > >> > > >>>> ahead and write a lot of code that may be rejected or change > > >> > > >> significantly, > > >> > > >>>> the risks were made clear. > > >> > > >>>> > > >> > > >>>>> Once code is written hard to think abstract. > > >> > > >>>> > > >> > > >>>> I can see the validity of the concern, but I personally don't > > >> see it > > >> > > as > > >> > > >> a > > >> > > >>>> huge risk. My impression from the Druid PR reviews I've seen > is > > >> that > > >> > > our > > >> > > >>>> reviewers are able to keep abstract design vs. implementation > > >> > details > > >> > > >>>> separate and consider alternate designs when reviewing. > > >> > > >>>> > > >> > > >>>> To summarize I think it's probably enough to have a policy > > along > > >> the > > >> > > >> lines > > >> > > >>>> of: > > >> > > >>>> - Create more formalized guidelines for proposals and what > > >> changes > > >> > > >> require > > >> > > >>>> proposals > > >> > > >>>> - Separate design and code review for major changes, with > > design > > >> > > review > > >> > > >>>> first, code-level review after reaching consensus on the > > design. > > >> > > >>>> - Code before the design review is completed is just for > > >> reference, > > >> > > not > > >> > > >>>> regarded as a candidate for review/merging. > > >> > > >>>> > > >> > > >>>> - Jon > > >> > > >>>> > > >> > > >>>> > > >> > > >>>> On Thu, Jan 3, 2019 at 12:48 PM Slim Bouguerra < > > >> > > >> slim.bougue...@gmail.com> > > >> > > >>>> wrote: > > >> > > >>>> > > >> > > >>>>> On Thu, Jan 3, 2019 at 12:16 PM Clint Wylie < > > >> clint.wy...@imply.io> > > >> > > >>>> wrote: > > >> > > >>>>> > > >> > > >>>>>> I am definitely biased in this matter as an owner of > another > > >> large > > >> > > PR > > >> > > >>>>> that > > >> > > >>>>>> wasn't preceded by a direct proposal or dev list > discussion, > > >> and > > >> > in > > >> > > >>>>> general > > >> > > >>>>>> I agree that proposal first is usually better, but I think > in > > >> some > > >> > > >>>> rarer > > >> > > >>>>>> cases approaching a problem code first *is* the most > > >> appropriate > > >> > way > > >> > > >> to > > >> > > >>>>>> have a discussion. > > >> > > >>>>> > > >> > > >>>>> > > >> > > >>>>> I am wondering here what is the case where code first is > > better? > > >> > > >>>>> In general when you are writing code you have an idea about > > what > > >> > you > > >> > > >> want > > >> > > >>>>> to change, why you want to change and why you want to change > > it. > > >> > > >>>>> I do not see what is wrong with sharing this primitive ideas > > and > > >> > > >> thoughts > > >> > > >>>>> as an abstract proposal (at least to avoid overlapping) > > >> > > >>>>> > > >> > > >>>>> I see nothing wrong with it so long as the author > > >> > > >>>>>> accepts that the PR is treated as a combined proposal and > > >> proof of > > >> > > >>>>> concept, > > >> > > >>>>>> and fair game to be radically changed via discussion or > even > > >> > > rejected, > > >> > > >>>>>> which sounds like Gian's attitude on the matter and is mine > > as > > >> > well > > >> > > >>>> with > > >> > > >>>>> my > > >> > > >>>>>> compression stuff. > > >> > > >>>>> > > >> > > >>>>> > > >> > > >>>>> Let's not be naive this is very rare that a contributor will > > >> accept > > >> > > >> that > > >> > > >>>>> his work is to be thrown, usually devs takes coding as > > personal > > >> > > >> creation > > >> > > >>>>> and they get attached to it. > > >> > > >>>>> To my point you can take a look on some old issue in the > Druid > > >> > forum > > >> > > >>>>> > > >> > > >>>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-druid/pull/3755#issuecomment-265667690 > > >> > > >>>>> and am sure other communities have similar problems. > > >> > > >>>>> So leaving the door open to some side cases is not a good > idea > > >> in > > >> > my > > >> > > >>>>> opinion and will lead to similar issue in the future. > > >> > > >>>>> > > >> > > >>>>> This seems to me especially likely to happen in cases > > >> > > >>>>>> where an approach still needs proven to be a viable idea > *to > > >> the > > >> > > >>>> author*, > > >> > > >>>>>> so that a much more productive discussion can be had in the > > >> first > > >> > > >>>> place. > > >> > > >>>>>> > > >> > > >>>>>> I think there is a trade off, I don't think we want to > > >> discourage > > >> > > >>>>>> experimentation by walling it off behind mandatory > > discussions > > >> > > before > > >> > > >>>> it > > >> > > >>>>>> can even start, but I do think formalizing the process for > > >> large > > >> > > >>>> changes > > >> > > >>>>> is > > >> > > >>>>>> a good thing, especially since we probably can't expect the > > >> wider > > >> > > >>>>> community > > >> > > >>>>>> to have the same attitude towards a large PR getting > > discarded > > >> as > > >> > a > > >> > > >>>>>> committer might. I think the Kafka approach is reasonable, > a > > >> bit > > >> > > more > > >> > > >>>>>> formal than our design review process but not overbearing. > > >> > > >>>>> > > >> > > >>>>> > > >> > > >>>>> Can you please explain what is overbearing ? what can be > > >> changed to > > >> > > >> make > > >> > > >>>> it > > >> > > >>>>> easy ? > > >> > > >>>>> Most of the points are kind of the actual questions that you > > >> want > > >> > to > > >> > > >>>>> address before hand anyway isn't ? > > >> > > >>>>> > > >> > > >>>>> > > >> > > >>>>>> Going code first > > >> > > >>>>>> should be in general discouraged, but when it does happen, > it > > >> > seems > > >> > > >>>> like > > >> > > >>>>>> opening DIP/an issue/starting a mailing list thread or > > >> whatever we > > >> > > go > > >> > > >>>>> with > > >> > > >>>>>> to have a more high level design discussion alongside the > > >> > reference > > >> > > PR > > >> > > >>>>>> could alleviate some of these complaints? > > >> > > >>>>> > > >> > > >>>>> > > >> > > >>>>> What are the complaints ? > > >> > > >>>>> > > >> > > >>>>> > > >> > > >>>>>> +1 for "DIP" heh, I think making > > >> > > >>>>>> them in the form of github issues is probably appropriate, > > >> with a > > >> > > dev > > >> > > >>>>> list > > >> > > >>>>>> thread to announce them perhaps? > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>>> I think github issue with [Proposal] header like > > >> > > >>>>> https://github.com/apache/incubator-druid/issues/4349 is > good > > >> to > > >> > me, > > >> > > >>>>> > > >> > > >>>>> Thanks! > > >> > > >>>>> > > >> > > >>>>> > > >> > > >>>>>> On Thu, Jan 3, 2019 at 10:28 AM Slim Bouguerra < > > >> bs...@apache.org> > > >> > > >>>> wrote: > > >> > > >>>>>> > > >> > > >>>>>>> Thanks everyone for interacting with this thread. > > >> > > >>>>>>> > > >> > > >>>>>>> The fact that i, Roman, Jihoon and others in the past (FJ > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://groups.google.com/forum/#!msg/druid-user/gkUEsAYIfBA/6B2GJdLkAgAJ > > >> ) > > >> > > >>>>>>> raised this point indicates that PRs without a proposal > are > > >> > indeed > > >> > > an > > >> > > >>>>>> issue > > >> > > >>>>>>> and we need to solve it. > > >> > > >>>>>>> > > >> > > >>>>>>> Something Similar to KIP maybe called DIPs is fine with > me. > > >> > > >>>>>>> What i strive to see is the following: > > >> > > >>>>>>> > > >> > > >>>>>>> [Step 1] formalize what is the kind of work that needs a > > >> formal > > >> > > >>>>>> Proposal, I > > >> > > >>>>>>> think Roman and Jihoon has already covered that pretty > well. > > >> am > > >> > +1 > > >> > > on > > >> > > >>>>>> that. > > >> > > >>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://lists.apache.org/thread.html/9b30d893bdb6bb633cf6a9a700183ffb5b98f115330531a55328ac77@%3Cdev.druid.apache.org%3E > > >> > > >>>>>>> > > >> > > >>>>>>> I am strongly in favor of the separation of Proposal > Review > > >> and > > >> > > >>>> (later) > > >> > > >>>>>>> Code review PRs. My main reasons: > > >> > > >>>>>>> Most importantly code reviewing will introduce lot of > noise > > >> and > > >> > > will > > >> > > >>>>>>> ultimately make the GitHub page unreadable. > > >> > > >>>>>>> Avoid overlapping of work. > > >> > > >>>>>>> Once code is written hard to think abstract. > > >> > > >>>>>>> Separate page for Design review later can always be used > it > > >> as a > > >> > > >>>> Design > > >> > > >>>>>>> document that is readable and code free-ish. > > >> > > >>>>>>> As i said the goal of this first round is to see if the > > >> community > > >> > > >>>> agree > > >> > > >>>>>>> about such change, then make the process of design more > > >> inclusive > > >> > > >>>> thus > > >> > > >>>>>>> other contributors can submit a counter proposals. > > >> > > >>>>>>> > > >> > > >>>>>>> [Step 2] IF everybody agree about that point Step 2 is to > > >> define > > >> > > >>>> which > > >> > > >>>>>>> medium is used to Publish a primitive form of a CODE FREE > > >> > Abstract > > >> > > >>>>>> Proposal > > >> > > >>>>>>> containing at least the following bullet points. > > >> > > >>>>>>> - The problem description and motivation > > >> > > >>>>>>> - Overview of the proposed change > > >> > > >>>>>>> - Operational impact (compatibility/ plans to upgrades) > > public > > >> > API > > >> > > >>>>>> changes, > > >> > > >>>>>>> configuration changes, algorithm, and so on > > >> > > >>>>>>> - Expected benefits and drawbacks > > >> > > >>>>>>> - Rationale and alternatives > > >> > > >>>>>>> - Estimate Time to Deliver if possible. > > >> > > >>>>>>> > > >> > > >>>>>>> The way i think this can be is a Github issue where member > > of > > >> the > > >> > > >>>>>> community > > >> > > >>>>>>> will interact via comments and the author will be updating > > the > > >> > > >>>>>> description > > >> > > >>>>>>> in the light of comments provided by the community. > > >> > > >>>>>>> > > >> > > >>>>>>> During and near the end of the design discussions the > > author/s > > >> > can > > >> > > >>>>> start > > >> > > >>>>>>> writing POCs to help guide the review process this > naturally > > >> will > > >> > > be > > >> > > >>>> a > > >> > > >>>>>> Pull > > >> > > >>>>>>> request with actual code. > > >> > > >>>>>>> > > >> > > >>>>>>> *Now the most important thing is that we need to agree > that > > >> any > > >> > > work > > >> > > >>>>> that > > >> > > >>>>>>> does not align with this formal process will be ignored > and > > >> the > > >> > > >>>> author > > >> > > >>>>>> will > > >> > > >>>>>>> be asked to start with a DIP* > > >> > > >>>>>>> *That is what i meant with “If it didn’t happen on the > > >> mailing > > >> > > list, > > >> > > >>>>> it > > >> > > >>>>>>> didn’t happen.”* > > >> > > >>>>>>> > > >> > > >>>>>>> Thanks and happy coding! > > >> > > >>>>>>> > > >> > > >>>>>>> On Wed, Jan 2, 2019 at 6:47 PM Gian Merlino < > > g...@apache.org> > > >> > > wrote: > > >> > > >>>>>>> > > >> > > >>>>>>>> One of the advantages I see with a more formal process is > > >> (like > > >> > > >>>> Kafka > > >> > > >>>>>>> KIPs) > > >> > > >>>>>>>> is that it levels the playing field a bit and sets some > > >> ground > > >> > > >>>> rules > > >> > > >>>>>> for > > >> > > >>>>>>>> working together. In a way it can help encourage > > >> contributions > > >> > by > > >> > > >>>>>> making > > >> > > >>>>>>> it > > >> > > >>>>>>>> clear what is expected of potential contributors. > > >> > > >>>>>>>> > > >> > > >>>>>>>> We have a design review process today that is not as > formal > > >> as > > >> > > >>>> KIPs, > > >> > > >>>>>> but > > >> > > >>>>>>> is > > >> > > >>>>>>>> somewhat heavier than the one you describe. Maybe we > could > > >> tweak > > >> > > >>>> our > > >> > > >>>>>>>> current one by starting to do design reviews separately > > from > > >> > PRs. > > >> > > >>>>> i.e., > > >> > > >>>>>>> for > > >> > > >>>>>>>> anything that meets our 'design review' criteria, do that > > on > > >> the > > >> > > >>>> dev > > >> > > >>>>>> list > > >> > > >>>>>>>> or in a separate issue, and keep the PR focused on > > code-level > > >> > > >>>> stuff. > > >> > > >>>>>> That > > >> > > >>>>>>>> way we don't end up trying to do both at once. And it > makes > > >> it > > >> > > >>>> easier > > >> > > >>>>>> to > > >> > > >>>>>>>> start talking about design before the code is ready, > which > > >> would > > >> > > be > > >> > > >>>>>>> better. > > >> > > >>>>>>>> > > >> > > >>>>>>>> On Wed, Jan 2, 2019 at 6:10 PM Julian Hyde < > > jh...@apache.org > > >> > > > >> > > >>>> wrote: > > >> > > >>>>>>>> > > >> > > >>>>>>>>> It’s really hard to say no to a contribution when > someone > > >> has > > >> > put > > >> > > >>>>> in > > >> > > >>>>>> a > > >> > > >>>>>>>>> significant amount of work. > > >> > > >>>>>>>>> > > >> > > >>>>>>>>> The following approach is simple and works really well: > > >> Before > > >> > > >>>> you > > >> > > >>>>>>> start > > >> > > >>>>>>>>> work, log a case, describing the problem. When you have > > some > > >> > > >>>> ideas > > >> > > >>>>>>> about > > >> > > >>>>>>>>> design, add those to the case. When you have a code > > branch, > > >> add > > >> > > >>>> its > > >> > > >>>>>> URL > > >> > > >>>>>>>> to > > >> > > >>>>>>>>> the case. And so forth. At any point in the proceedings, > > >> people > > >> > > >>>> can > > >> > > >>>>>>> chime > > >> > > >>>>>>>>> in with their opinions. > > >> > > >>>>>>>>> > > >> > > >>>>>>>>> In my opinion, a formal “design review” process is not > > >> > necessary. > > >> > > >>>>>> Just > > >> > > >>>>>>>>> build consensus iteratively, by starting the > conversation > > >> early > > >> > > >>>> in > > >> > > >>>>>> the > > >> > > >>>>>>>>> process. > > >> > > >>>>>>>>> > > >> > > >>>>>>>>> Julian > > >> > > >>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>>>>> On Jan 2, 2019, at 12:37 PM, Gian Merlino < > > g...@apache.org > > >> > > > >> > > >>>>> wrote: > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>> In this particular case: please consider the PR as a > > >> proposal. > > >> > > >>>>>> Don't > > >> > > >>>>>>>> feel > > >> > > >>>>>>>>>> like just because there is code there that takes a > > certain > > >> > > >>>>>> approach, > > >> > > >>>>>>>> that > > >> > > >>>>>>>>>> the approach is somehow sacred. I had to implement > > >> something > > >> > to > > >> > > >>>>>>>>> crystallize > > >> > > >>>>>>>>>> my own thinking about how the problem could be > > approached. > > >> I > > >> > > >>>>> won't > > >> > > >>>>>> be > > >> > > >>>>>>>>>> disappointed if, as a community, we decide a different > > >> > > >>>> direction > > >> > > >>>>> is > > >> > > >>>>>>>>> better > > >> > > >>>>>>>>>> and the code all gets thrown away. That's one of the > > >> reasons > > >> > > >>>>> that I > > >> > > >>>>>>>>> removed > > >> > > >>>>>>>>>> the 0.14.0 milestone that was added to the patch. (I > > don't > > >> > want > > >> > > >>>>> to > > >> > > >>>>>>> rush > > >> > > >>>>>>>>> it, > > >> > > >>>>>>>>>> nor do I think that's a good idea.) > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>> In general: Sounds like we could do with some more > > >> > > >>>> formalization > > >> > > >>>>>>> around > > >> > > >>>>>>>>>> what a proposal looks like, which sorts of changes need > > >> one, > > >> > > >>>> and > > >> > > >>>>>> when > > >> > > >>>>>>>> in > > >> > > >>>>>>>>>> the dev cycle it is appropriate. FWIW I think Kafka's > > >> process > > >> > > >>>> is > > >> > > >>>>>> more > > >> > > >>>>>>>> or > > >> > > >>>>>>>>>> less fine, and would be okay with adopting it for Druid > > if > > >> > > >>>> people > > >> > > >>>>>>> like > > >> > > >>>>>>>>> it. > > >> > > >>>>>>>>>> Right now our standards for what requires a "design > > review" > > >> > are > > >> > > >>>>>> very > > >> > > >>>>>>>>>> similar to the Kafka community standards for what > > requires > > >> a > > >> > > >>>> KIP, > > >> > > >>>>>> so > > >> > > >>>>>>> we > > >> > > >>>>>>>>>> have some familiarity with those concepts. However we > > don't > > >> > > >>>>>> separate > > >> > > >>>>>>> PR > > >> > > >>>>>>>>>> review and proposal discussion as strictly as they do, > > >> which > > >> > > >>>>> seems > > >> > > >>>>>> to > > >> > > >>>>>>>> be > > >> > > >>>>>>>>>> the foundation for the feeling of exclusion that is > being > > >> felt > > >> > > >>>>>> here. > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>> Separately: I just redid the description on > > >> > > >>>>>>>>>> https://github.com/apache/incubator-druid/pull/6794 to > > be > > >> > more > > >> > > >>>>>>>>> proposal-y. > > >> > > >>>>>>>>>> I followed the KIP style: > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>> > > >> > > >> > > >> > > > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > >> > > >>>>>>>>> . > > >> > > >>>>>>>>>> Please refresh the page and see if it looks more > useful. > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>> Gian > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>> On Wed, Jan 2, 2019 at 10:52 AM Julian Hyde < > > >> jh...@apache.org > > >> > > > > >> > > >>>>>>> wrote: > > >> > > >>>>>>>>>> > > >> > > >>>>>>>>>>> Slim, > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>> I agree with your points that offline development is > bad > > >> for > > >> > > >>>>>>>> community. > > >> > > >>>>>>>>>>> But I don’t think you need much mentor help. You have > > >> raised > > >> > > >>>>> valid > > >> > > >>>>>>>>> issues > > >> > > >>>>>>>>>>> and the Druid community needs to decide what its > > >> development > > >> > > >>>>>>> practices > > >> > > >>>>>>>>>>> should be. > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>> Julian > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>>> On Jan 2, 2019, at 10:29 AM, Slim Bouguerra < > > >> > > >>>> bs...@apache.org> > > >> > > >>>>>>>> wrote: > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> Hello everyone and hope you all have very good > > holidays. > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> First, this email is not directed on the author or > the > > PR > > >> > > >>>>>>>>>>>> https://github.com/apache/incubator-druid/pull/6794 > > it > > >> > > >>>> self, > > >> > > >>>>>> but > > >> > > >>>>>>> i > > >> > > >>>>>>>>> see > > >> > > >>>>>>>>>>>> this PR as a perfect example. > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> One of the foundation of Apache Way or what i would > > >> simply > > >> > > >>>> call > > >> > > >>>>>>> open > > >> > > >>>>>>>>>>> source > > >> > > >>>>>>>>>>>> community driven development is that "Technical > > decisions > > >> > are > > >> > > >>>>>>>>> discussed, > > >> > > >>>>>>>>>>>> decided, and archived publicly. > > >> > > >>>>>>>>>>>> developpement" > > >> > > >>>>>>>>>>>> Which means that big technical changes such as the > one > > >> > > >>>> brought > > >> > > >>>>>> by > > >> > > >>>>>>>>> #/6794 > > >> > > >>>>>>>>>>>> should have started as a proposal and round of > > >> discussions > > >> > > >>>>> about > > >> > > >>>>>>> the > > >> > > >>>>>>>>>>> major > > >> > > >>>>>>>>>>>> changes designs not as 11K line of code. > > >> > > >>>>>>>>>>>> I believe such openness will promote a lot of good > > >> benefits > > >> > > >>>>> such > > >> > > >>>>>>> as: > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> - ensures community health and growth. > > >> > > >>>>>>>>>>>> - ensures everyone can participate not only the > authors > > >> and > > >> > > >>>> his > > >> > > >>>>>>>>>>> co-workers. > > >> > > >>>>>>>>>>>> - ensures that the project is driven by the community > > and > > >> > > >>>> not a > > >> > > >>>>>>> given > > >> > > >>>>>>>>>>>> company or an individual. > > >> > > >>>>>>>>>>>> - ensures that there is consensus (not saying 100% > > >> > > >>>> agreement;) > > >> > > >>>>>>>> however > > >> > > >>>>>>>>> it > > >> > > >>>>>>>>>>>> means that all individuals will accept the current > > >> progress > > >> > > >>>> on > > >> > > >>>>>> the > > >> > > >>>>>>>>>>> project > > >> > > >>>>>>>>>>>> until some better proposal is put forth. > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> Personally such BIG offline PR makes me feel excluded > > and > > >> > > >>>>> doesn't > > >> > > >>>>>>>> give > > >> > > >>>>>>>>>>> me a > > >> > > >>>>>>>>>>>> sense that i belong to a community at all. > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> To prevent such off list development i think as a > Druid > > >> > > >>>>> Community > > >> > > >>>>>>> we > > >> > > >>>>>>>>> need > > >> > > >>>>>>>>>>>> to stick to the apache way “If it didn’t happen on > the > > >> > > >>>> mailing > > >> > > >>>>>>> list, > > >> > > >>>>>>>> it > > >> > > >>>>>>>>>>>> didn’t happen.” > > >> > > >>>>>>>>>>>> > > >> > > >>>>>>>>>>>> I would appreciate if some of the Apache mentor help > > with > > >> > > >>>> this. > > >> > > >>>>>>>>>>>> Thanks > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>> > > >> > > > > --------------------------------------------------------------------- > > >> > > >>>>>>>>>>> To unsubscribe, e-mail: > > dev-unsubscr...@druid.apache.org > > >> > > >>>>>>>>>>> For additional commands, e-mail: > > >> dev-h...@druid.apache.org > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>> > > >> > > --------------------------------------------------------------------- > > >> > > >>>>>>>>> To unsubscribe, e-mail: > dev-unsubscr...@druid.apache.org > > >> > > >>>>>>>>> For additional commands, e-mail: > > dev-h...@druid.apache.org > > >> > > >>>>>>>>> > > >> > > >>>>>>>>> > > >> > > >>>>>>>> > > >> > > >>>>>>> > > >> > > >>>>>> > > >> > > >>>>> > > >> > > >>>>> > > >> > > >>>>> -- > > >> > > >>>>> > > >> > > >>>>> B-Slim > > >> > > >>>>> > > >> > > >> > > >> > > > _______/\/\/\_______/\/\/\_______/\/\/\_______/\/\/\_______/\/\/\_______ > > >> > > >>>>> > > >> > > >>>> > > >> > > >> > > >> > > >> > > >> > > >> > > >> --------------------------------------------------------------------- > > >> > > >> To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > > >> > > >> For additional commands, e-mail: dev-h...@druid.apache.org > > >> > > >> > > >> > > >> > > >> > > > > >> > > > > >> > > > > --------------------------------------------------------------------- > > >> > > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > > >> > > For additional commands, e-mail: dev-h...@druid.apache.org > > >> > > > > >> > > > > >> > > > >> > > > > > >
