Re: Impala commit policy
On Thu, Dec 3, 2015 at 7:41 PM, Ralph Goerswrote: > In my opinion the correct approach is to identify the parts of the code that > a) > seem to be most susceptible to bugs, b) are hard to understand well, or c) > where > simple changes can have huge impacts on performance and then use RTC for > those areas I tend to agree but I'd say *maybe, if really needed* use RTC... And re-evaluate that periodically. RTC can have a big cost community-wise, so it's important IMO to use it as sparingly as possible. -Bertrand - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Impala commit policy
On 2 December 2015 at 23:04, Greg Steinwrote: > On Wed, Dec 2, 2015 at 8:50 PM, Julian Hyde wrote: > > > Thanks, Roman. For the record, I don’t plan to contribute to Impala or > > Kudu, and I don’t like strict commit policies such as RTC. But I wanted > to > > stand up for “states' rights”, the right of podlings and projects to > > determine their own processes and cultures. > > > > LOL ... being a Texan, I can certainly get on board with the notion of > states' rights :-P > > But I caution: as I said else-thread, we use the Incubation process because > we believe the podling needs to *learn* how we like communities to operate. > Peer respect, inclusivity, open dialog, consensus, etc. By definition, the > podling is unable to make these decisions within the guides and desires of > the Foundation. If we trusted them to do so, then we'd just make them a TLP > and skip incubation. > > Josh puts it well: > > On Thu, Dec 3, 2015 at 12:26 AM, Josh Elser wrote: > >... > > > +1 I'm not entirely sold on saying they have no explicitly policy up > front > > (I'd be worried about that causing confusion -- the project will operate > > how they're comfortable operating), but I'd definitely want to see _real_ > > discussion is had after the podling gets on its feet and grows beyond the > > initial membership. > > > > I'd like to see podlings have enough diversity and independence from the > initial PPMC, to have such a discussion. My fear is that RTC holds back > growing the diversity of opinion, and that status quo will not allow for > moving away from Gerrit. > > ... > > I will also note that one of the primary reasons explained for RTC is "the > code is too complex to allow for unreviewed changes to be applied". Has > that basis been justified for Impala? Are we talking data loss? Nope. It's > a layer over the data substrate. Synchronization across a cluster? Nah. > Where is the complexity? > I'm happy to field technical questions about Impala. You seem to be conflating 'complexity' with 'severity of potential bugs' - I see the two as separate. Under the 'severity' heading, Impala both writes and reads data from a variety of data stores. So if there's a bug in Impala's write path, data can be lost. But because Impala also returns results to client applications, there's a significant risk of business impact if the *wrong* results are returned. I know, because I have dealt with situations where this has happened, and no-one is very happy about it. Our customers typically run business-critical analytic workloads through Impala; if it stops working correctly that's usually a big problem. As far as 'complexity' goes, I make no comparative claims about Impala's complexity vs any other project. But to give some indication of the moving parts inside Impala: there's a component which compiles highly optimised versions of each query operator at run time, there's a query planner which parses and plans a large portion of the SQL standard, there is the added complexity of being a 'massively' (with many deployments in the high 100s of nodes) distributed system with the added coordination and consistency guarantees that brings to it, and there is also the added complexity of running highly concurrent workloads in a single process, with all the concurrency headaches etc. that can bring. That's not to mention implementations of 'standard' SQL operators like joins, sorts and so on that are still the subject of active research in academia and industry. All this is in the context of Impala's main differentiator, which is that it is amongst the very fastest SQL engine for data stored in HDFS and friends. That means that small changes can have large unexpected consequences, since efficiency is a subtle and capricious thing. It has always, therefore, helped us to have more than one set of eyes on every change in the past, to ensure that the probability of the introduction of subtle performance and functional regressions is reduced. Automated testing plays a huge role here as well, but for us it's been most effective in concert with code review. (There are other reasons I vastly prefer RTC as well, but I'm addressing your specific points here so as not to kick off another RTCvsCTR thread :)). > > In this case, the RTC seems to stem from the choice of Gerrit, rather than > some innate complexity. > > Gerrit does not mandate RTC, since you can just push to refs/heads/ and bypass the review creation step. Historically, the Impala team at Cloudera has used at least three different review tools (including Review Board, which is used elsewhere at the ASF). The choice of review tool stems completely from pragmatism - we really did not like Review Board, and briefly used Rietveld before moving to Gerrit which we have preferred. At every step, we used RTC. Henry > I *do* note that possibly committers could choose to commit directly, or > choose to use Gerrit when they are
Re: Impala commit policy
Virtually any project you look at is going to have portions that are fairly complex and portions that are pretty straightforward. In my opinion the correct approach is to identify the parts of the code that a) seem to be most susceptible to bugs, b) are hard to understand well, or c) where simple changes can have huge impacts on performance and then use RTC for those areas. In addition, you might want to require RTC for significant new feature additions. Although I expect every project to have code that falls into these categories the ratio of that code will vary from project to project. As an example, I can honestly say that with Flume the File Channel should always be RTC. But almost everything else could be done more effectively with CTR. Ralph > On Dec 3, 2015, at 11:20 AM, Henry Robinsonwrote: > > I'm happy to field technical questions about Impala. You seem to be > conflating 'complexity' with 'severity of potential bugs' - I see the two > as separate. > > Under the 'severity' heading, Impala both writes and reads data from a > variety of data stores. So if there's a bug in Impala's write path, data > can be lost. But because Impala also returns results to client > applications, there's a significant risk of business impact if the *wrong* > results are returned. I know, because I have dealt with situations where > this has happened, and no-one is very happy about it. Our customers > typically run business-critical analytic workloads through Impala; if it > stops working correctly that's usually a big problem. > > As far as 'complexity' goes, I make no comparative claims about Impala's > complexity vs any other project. But to give some indication of the moving > parts inside Impala: there's a component which compiles highly optimised > versions of each query operator at run time, there's a query planner which > parses and plans a large portion of the SQL standard, there is the added > complexity of being a 'massively' (with many deployments in the high 100s > of nodes) distributed system with the added coordination and consistency > guarantees that brings to it, and there is also the added complexity of > running highly concurrent workloads in a single process, with all the > concurrency headaches etc. that can bring. That's not to mention > implementations of 'standard' SQL operators like joins, sorts and so on > that are still the subject of active research in academia and industry. > > All this is in the context of Impala's main differentiator, which is that > it is amongst the very fastest SQL engine for data stored in HDFS and > friends. That means that small changes can have large unexpected > consequences, since efficiency is a subtle and capricious thing. It has > always, therefore, helped us to have more than one set of eyes on every > change in the past, to ensure that the probability of the introduction of > subtle performance and functional regressions is reduced. Automated testing > plays a huge role here as well, but for us it's been most effective in > concert with code review. > > (There are other reasons I vastly prefer RTC as well, but I'm addressing > your specific points here so as not to kick off another RTCvsCTR thread :)). > > > >> >> In this case, the RTC seems to stem from the choice of Gerrit, rather than >> some innate complexity. >> >> > > Gerrit does not mandate RTC, since you can just push to refs/heads/ > and bypass the review creation step. > > Historically, the Impala team at Cloudera has used at least three different > review tools (including Review Board, which is used elsewhere at the ASF). > The choice of review tool stems completely from pragmatism - we really did > not like Review Board, and briefly used Rietveld before moving to Gerrit > which we have preferred. At every step, we used RTC. > > Henry > > > >> I *do* note that possibly committers could choose to commit directly, or >> choose to use Gerrit when they are unsure. Will the (P)PMC allow those >> direct commits? Or mandate Gerrit for every commit? >> >> Cheers, >> -g
Re: Impala commit policy
Nice +1 =) On Wed, Dec 2, 2015 at 2:01 AM, Tom Whitewrote: > The vote to accept Impala into the incubator has passed > (http://s.apache.org/u6r), however there are still some concerns about > CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not a > binary choice, and that it's entirely reasonable that different > communities have different commit policies at the ASF. > > I think Julian Hyde's suggestion that the Impala podling start with no > explicit commit policy is a good one. Incubation should be used as a > time to work out what works best for a project. The initial Impala > community should discuss the commit policy as they go through the > process of setting up ASF infra and start growing the podling. In > particular this will include how Gerrit can be used as a tool to > facilitate reviews, and how that fits with ASF culture, which is > something that other projects are looking at too. > > Cheers, > Tom > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Impala commit policy
I agree that this is something the Impala community will want to discuss fairly early on in incubation - along with a lot of other project procedural stuff as we adjust or rethink our workflows to be Apache-Way compatible. Until we have that discussion, I'd expect Impala will continue along RTC lines simply because this is what the existing community is used to (and I believe, prefers), and the workflows are well established and work smoothly. That is, I personally am ok with no explicit commit protocol, but in practice it is likely to appear as if there is no change, until the community discussion about policies happens during incubation. On 2 December 2015 at 10:08, Henry Saputrawrote: > Nice +1 =) > > On Wed, Dec 2, 2015 at 2:01 AM, Tom White wrote: > > The vote to accept Impala into the incubator has passed > > (http://s.apache.org/u6r), however there are still some concerns about > > CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not a > > binary choice, and that it's entirely reasonable that different > > communities have different commit policies at the ASF. > > > > I think Julian Hyde's suggestion that the Impala podling start with no > > explicit commit policy is a good one. Incubation should be used as a > > time to work out what works best for a project. The initial Impala > > community should discuss the commit policy as they go through the > > process of setting up ASF infra and start growing the podling. In > > particular this will include how Gerrit can be used as a tool to > > facilitate reviews, and how that fits with ASF culture, which is > > something that other projects are looking at too. > > > > Cheers, > > Tom > > > > - > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > > -- Henry Robinson Software Engineer Cloudera 415-994-6679
Re: Impala commit policy
+1 to this as well. Whether the community changes its mind or not is irrelevant in my opinion. What is important is it gets to choose for itself and possibly revisits regularly as it sees fit. This discussion should be encouraged and people who want to promote the merits of one approach or another can do so. On Wed, Dec 2, 2015 at 11:19 AM, Henry Robinsonwrote: > What might happen, however, is that the discussion is revisited with a > particular focus on the concerns that you've raised. So although it might > be unlikely that the community performs a volte-face and elects for CTR, we > might say "what can we do to limit the risk that RTC inhibits community > growth, without abandoning RTC completely?". At the very least, the > community becomes aware of the concerns, and more sensitive to their > potential impact. > > > On 2 December 2015 at 11:09, Greg Stein wrote: > > > Yeah, this is what I meant earlier. Leaving out a commit policy changes > > nothing. The same people who put together the proposal will be the same > set > > as those discussing it as a podling, and they will reach the same > > conclusion. > > > > If the PPMC doubles in size, with fresh faces, then a real discussion can > > happen. Tho I doubt that will be possible -- I'm unaware of any podling > > pulling off such growth. > > On Dec 2, 2015 12:45 PM, "Henry Robinson" wrote: > > > > > I agree that this is something the Impala community will want to > discuss > > > fairly early on in incubation - along with a lot of other project > > > procedural stuff as we adjust or rethink our workflows to be Apache-Way > > > compatible. > > > > > > Until we have that discussion, I'd expect Impala will continue along > RTC > > > lines simply because this is what the existing community is used to > (and > > I > > > believe, prefers), and the workflows are well established and work > > > smoothly. > > > > > > That is, I personally am ok with no explicit commit protocol, but in > > > practice it is likely to appear as if there is no change, until the > > > community discussion about policies happens during incubation. > > > > > > On 2 December 2015 at 10:08, Henry Saputra > > > wrote: > > > > > > > Nice +1 =) > > > > > > > > On Wed, Dec 2, 2015 at 2:01 AM, Tom White > wrote: > > > > > The vote to accept Impala into the incubator has passed > > > > > (http://s.apache.org/u6r), however there are still some concerns > > about > > > > > CTR/RTC. My main takeaways from the CTR/RTC thread are that it's > not > > a > > > > > binary choice, and that it's entirely reasonable that different > > > > > communities have different commit policies at the ASF. > > > > > > > > > > I think Julian Hyde's suggestion that the Impala podling start with > > no > > > > > explicit commit policy is a good one. Incubation should be used as > a > > > > > time to work out what works best for a project. The initial Impala > > > > > community should discuss the commit policy as they go through the > > > > > process of setting up ASF infra and start growing the podling. In > > > > > particular this will include how Gerrit can be used as a tool to > > > > > facilitate reviews, and how that fits with ASF culture, which is > > > > > something that other projects are looking at too. > > > > > > > > > > Cheers, > > > > > Tom > > > > > > > > > > > - > > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > > > > - > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > -- > > > Henry Robinson > > > Software Engineer > > > Cloudera > > > 415-994-6679 > > > > > > > > > -- > Henry Robinson > Software Engineer > Cloudera > 415-994-6679 > -- Julien
Re: Impala commit policy
I am not sure what "start with no explicit commit policy" even means. Will there be no commits, until the discussion on the subject happens? How code changes will be going into the source base? Cos On Wed, Dec 02, 2015 at 10:01AM, Tom White wrote: > The vote to accept Impala into the incubator has passed > (http://s.apache.org/u6r), however there are still some concerns about > CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not a > binary choice, and that it's entirely reasonable that different > communities have different commit policies at the ASF. > > I think Julian Hyde's suggestion that the Impala podling start with no > explicit commit policy is a good one. Incubation should be used as a > time to work out what works best for a project. The initial Impala > community should discuss the commit policy as they go through the > process of setting up ASF infra and start growing the podling. In > particular this will include how Gerrit can be used as a tool to > facilitate reviews, and how that fits with ASF culture, which is > something that other projects are looking at too. > > Cheers, > Tom > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > signature.asc Description: Digital signature
Re: Impala commit policy
What might happen, however, is that the discussion is revisited with a particular focus on the concerns that you've raised. So although it might be unlikely that the community performs a volte-face and elects for CTR, we might say "what can we do to limit the risk that RTC inhibits community growth, without abandoning RTC completely?". At the very least, the community becomes aware of the concerns, and more sensitive to their potential impact. On 2 December 2015 at 11:09, Greg Steinwrote: > Yeah, this is what I meant earlier. Leaving out a commit policy changes > nothing. The same people who put together the proposal will be the same set > as those discussing it as a podling, and they will reach the same > conclusion. > > If the PPMC doubles in size, with fresh faces, then a real discussion can > happen. Tho I doubt that will be possible -- I'm unaware of any podling > pulling off such growth. > On Dec 2, 2015 12:45 PM, "Henry Robinson" wrote: > > > I agree that this is something the Impala community will want to discuss > > fairly early on in incubation - along with a lot of other project > > procedural stuff as we adjust or rethink our workflows to be Apache-Way > > compatible. > > > > Until we have that discussion, I'd expect Impala will continue along RTC > > lines simply because this is what the existing community is used to (and > I > > believe, prefers), and the workflows are well established and work > > smoothly. > > > > That is, I personally am ok with no explicit commit protocol, but in > > practice it is likely to appear as if there is no change, until the > > community discussion about policies happens during incubation. > > > > On 2 December 2015 at 10:08, Henry Saputra > > wrote: > > > > > Nice +1 =) > > > > > > On Wed, Dec 2, 2015 at 2:01 AM, Tom White wrote: > > > > The vote to accept Impala into the incubator has passed > > > > (http://s.apache.org/u6r), however there are still some concerns > about > > > > CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not > a > > > > binary choice, and that it's entirely reasonable that different > > > > communities have different commit policies at the ASF. > > > > > > > > I think Julian Hyde's suggestion that the Impala podling start with > no > > > > explicit commit policy is a good one. Incubation should be used as a > > > > time to work out what works best for a project. The initial Impala > > > > community should discuss the commit policy as they go through the > > > > process of setting up ASF infra and start growing the podling. In > > > > particular this will include how Gerrit can be used as a tool to > > > > facilitate reviews, and how that fits with ASF culture, which is > > > > something that other projects are looking at too. > > > > > > > > Cheers, > > > > Tom > > > > > > > > - > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > - > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > > > -- > > Henry Robinson > > Software Engineer > > Cloudera > > 415-994-6679 > > > -- Henry Robinson Software Engineer Cloudera 415-994-6679
Re: Impala commit policy
Yeah, this is what I meant earlier. Leaving out a commit policy changes nothing. The same people who put together the proposal will be the same set as those discussing it as a podling, and they will reach the same conclusion. If the PPMC doubles in size, with fresh faces, then a real discussion can happen. Tho I doubt that will be possible -- I'm unaware of any podling pulling off such growth. On Dec 2, 2015 12:45 PM, "Henry Robinson"wrote: > I agree that this is something the Impala community will want to discuss > fairly early on in incubation - along with a lot of other project > procedural stuff as we adjust or rethink our workflows to be Apache-Way > compatible. > > Until we have that discussion, I'd expect Impala will continue along RTC > lines simply because this is what the existing community is used to (and I > believe, prefers), and the workflows are well established and work > smoothly. > > That is, I personally am ok with no explicit commit protocol, but in > practice it is likely to appear as if there is no change, until the > community discussion about policies happens during incubation. > > On 2 December 2015 at 10:08, Henry Saputra > wrote: > > > Nice +1 =) > > > > On Wed, Dec 2, 2015 at 2:01 AM, Tom White wrote: > > > The vote to accept Impala into the incubator has passed > > > (http://s.apache.org/u6r), however there are still some concerns about > > > CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not a > > > binary choice, and that it's entirely reasonable that different > > > communities have different commit policies at the ASF. > > > > > > I think Julian Hyde's suggestion that the Impala podling start with no > > > explicit commit policy is a good one. Incubation should be used as a > > > time to work out what works best for a project. The initial Impala > > > community should discuss the commit policy as they go through the > > > process of setting up ASF infra and start growing the podling. In > > > particular this will include how Gerrit can be used as a tool to > > > facilitate reviews, and how that fits with ASF culture, which is > > > something that other projects are looking at too. > > > > > > Cheers, > > > Tom > > > > > > - > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > - > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > -- > Henry Robinson > Software Engineer > Cloudera > 415-994-6679 >
Re: Impala commit policy
Yeup! On Wed, Dec 2, 2015 at 1:19 PM, Henry Robinsonwrote: > What might happen, however, is that the discussion is revisited with a > particular focus on the concerns that you've raised. So although it might > be unlikely that the community performs a volte-face and elects for CTR, we > might say "what can we do to limit the risk that RTC inhibits community > growth, without abandoning RTC completely?". At the very least, the > community becomes aware of the concerns, and more sensitive to their > potential impact. > > > On 2 December 2015 at 11:09, Greg Stein wrote: > > > Yeah, this is what I meant earlier. Leaving out a commit policy changes > > nothing. The same people who put together the proposal will be the same > set > > as those discussing it as a podling, and they will reach the same > > conclusion. > > > > If the PPMC doubles in size, with fresh faces, then a real discussion can > > happen. Tho I doubt that will be possible -- I'm unaware of any podling > > pulling off such growth. > > On Dec 2, 2015 12:45 PM, "Henry Robinson" wrote: > > > > > I agree that this is something the Impala community will want to > discuss > > > fairly early on in incubation - along with a lot of other project > > > procedural stuff as we adjust or rethink our workflows to be Apache-Way > > > compatible. > > > > > > Until we have that discussion, I'd expect Impala will continue along > RTC > > > lines simply because this is what the existing community is used to > (and > > I > > > believe, prefers), and the workflows are well established and work > > > smoothly. > > > > > > That is, I personally am ok with no explicit commit protocol, but in > > > practice it is likely to appear as if there is no change, until the > > > community discussion about policies happens during incubation. > > > > > > On 2 December 2015 at 10:08, Henry Saputra > > > wrote: > > > > > > > Nice +1 =) > > > > > > > > On Wed, Dec 2, 2015 at 2:01 AM, Tom White > wrote: > > > > > The vote to accept Impala into the incubator has passed > > > > > (http://s.apache.org/u6r), however there are still some concerns > > about > > > > > CTR/RTC. My main takeaways from the CTR/RTC thread are that it's > not > > a > > > > > binary choice, and that it's entirely reasonable that different > > > > > communities have different commit policies at the ASF. > > > > > > > > > > I think Julian Hyde's suggestion that the Impala podling start with > > no > > > > > explicit commit policy is a good one. Incubation should be used as > a > > > > > time to work out what works best for a project. The initial Impala > > > > > community should discuss the commit policy as they go through the > > > > > process of setting up ASF infra and start growing the podling. In > > > > > particular this will include how Gerrit can be used as a tool to > > > > > facilitate reviews, and how that fits with ASF culture, which is > > > > > something that other projects are looking at too. > > > > > > > > > > Cheers, > > > > > Tom > > > > > > > > > > > - > > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > > > > - > > > > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > > > > For additional commands, e-mail: general-h...@incubator.apache.org > > > > > > > > > > > > > > > > > -- > > > Henry Robinson > > > Software Engineer > > > Cloudera > > > 415-994-6679 > > > > > > > > > -- > Henry Robinson > Software Engineer > Cloudera > 415-994-6679 >
Re: Impala commit policy
“No explicit commit policy” means that only committers can commit. It is each committer’s discretion whether they ask for others to review the change before they commit it, whether they check in code that doesn’t build, whether they run the test suite before committing. This policy is the bare constitutional minimum. We would all hope and expect that the community would quickly agree on some policies, but that is up the community. They are not stupid, they want to produce high-quality software, and they want to grow their community, and they will figure out a policy that achieves these goals. Julian > On Dec 2, 2015, at 12:40 PM, Konstantin Boudnikwrote: > > I am not sure what "start with no explicit commit policy" even means. Will > there be no commits, until the discussion on the subject happens? > > How code changes will be going into the source base? > Cos > > On Wed, Dec 02, 2015 at 10:01AM, Tom White wrote: >> The vote to accept Impala into the incubator has passed >> (http://s.apache.org/u6r), however there are still some concerns about >> CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not a >> binary choice, and that it's entirely reasonable that different >> communities have different commit policies at the ASF. >> >> I think Julian Hyde's suggestion that the Impala podling start with no >> explicit commit policy is a good one. Incubation should be used as a >> time to work out what works best for a project. The initial Impala >> community should discuss the commit policy as they go through the >> process of setting up ASF infra and start growing the podling. In >> particular this will include how Gerrit can be used as a tool to >> facilitate reviews, and how that fits with ASF culture, which is >> something that other projects are looking at too. >> >> Cheers, >> Tom >> >> - >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> For additional commands, e-mail: general-h...@incubator.apache.org >> - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Impala commit policy
Tom White wrote: The vote to accept Impala into the incubator has passed (http://s.apache.org/u6r), however there are still some concerns about CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not a binary choice, and that it's entirely reasonable that different communities have different commit policies at the ASF. I think Julian Hyde's suggestion that the Impala podling start with no explicit commit policy is a good one. Incubation should be used as a time to work out what works best for a project. The initial Impala community should discuss the commit policy as they go through the process of setting up ASF infra and start growing the podling. In particular this will include how Gerrit can be used as a tool to facilitate reviews, and how that fits with ASF culture, which is something that other projects are looking at too. Cheers, Tom (Ugh, and then I catch up on the rest of my mail - sorry for the spam on the RESULT thread) +1 I'm not entirely sold on saying they have no explicitly policy up front (I'd be worried about that causing confusion -- the project will operate how they're comfortable operating), but I'd definitely want to see _real_ discussion is had after the podling gets on its feet and grows beyond the initial membership. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Impala commit policy
On Wed, Dec 2, 2015 at 4:24 PM, Julian Hydewrote: > “No explicit commit policy” means that only committers can commit. > It is each committer’s discretion whether they ask for others to review > the change before they commit it, whether they check in code that doesn’t > build, whether they run the test suite before committing. > > This policy is the bare constitutional minimum. We would all hope and expect > that the community would quickly agree on some policies, but that is up the > community. > They are not stupid, they want to produce high-quality software, and they > want to grow > their community, and they will figure out a policy that achieves these goals. If that's the expectation that is internalized by the podling community than I guess my concerns are somewhat taken care of to a point where I'd be comfortable giving the proposal a +0. Let me explain why it is a +0 instead of +1. This will also be an opportunity for me to clarify my seemingly inconsistent position with Kudu proposal (which I deliberately +1ed). It all comes back to trust and inclusiveness. I thought that Greg was super convincing at articulating that CTR policy is an indication, a proxy if you will for those qualities. If CTR is there -- I know that the community is trusting and inclusive. My -1 for Impala was based on their strong resistance to CTR as a proxy measure of how ready they are to accept the "Apache Way". Them fighting the idea of CTR gave me a strong indication that they are resisting to being trusting and inclusive. Now, of course, as any astute student of logic will know, necessity and sufficiency is not the same thing. While a presence of CTR policy is a strong indication of the community getting the "Apache Way" the lack of it is not necessarily an indication of them not getting it. Impala community, however has one extra strike against it that wasn't allowing me to give it the same benefit of the doubt that Kudu community got (hence +1 for Kudu despite their instance on RTC). Impala and its existing community are *not* new to Open Source. They've been out on GitHub since late 2012 and they have operated under a very explicit BDFL model. Based on their past attitude towards external contributions (personalized via statement of the project lead) I have strong reasons to believe that they are going to have really tough time adjusting to the Apache governance model AND when I saw that strong resistance to CTR that was it for me. That said, this thread and the expectations articulated by Tom and others make me more comfortable. However, the lack of *actionable* suggestion for how they are going to integrate with Apache governance model makes it +0 and not +1. Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Impala commit policy
On Wed, Dec 2, 2015 at 8:50 PM, Julian Hydewrote: > Thanks, Roman. For the record, I don’t plan to contribute to Impala or > Kudu, and I don’t like strict commit policies such as RTC. But I wanted to > stand up for “states' rights”, the right of podlings and projects to > determine their own processes and cultures. > LOL ... being a Texan, I can certainly get on board with the notion of states' rights :-P But I caution: as I said else-thread, we use the Incubation process because we believe the podling needs to *learn* how we like communities to operate. Peer respect, inclusivity, open dialog, consensus, etc. By definition, the podling is unable to make these decisions within the guides and desires of the Foundation. If we trusted them to do so, then we'd just make them a TLP and skip incubation. Josh puts it well: On Thu, Dec 3, 2015 at 12:26 AM, Josh Elser wrote: >... > +1 I'm not entirely sold on saying they have no explicitly policy up front > (I'd be worried about that causing confusion -- the project will operate > how they're comfortable operating), but I'd definitely want to see _real_ > discussion is had after the podling gets on its feet and grows beyond the > initial membership. > I'd like to see podlings have enough diversity and independence from the initial PPMC, to have such a discussion. My fear is that RTC holds back growing the diversity of opinion, and that status quo will not allow for moving away from Gerrit. ... I will also note that one of the primary reasons explained for RTC is "the code is too complex to allow for unreviewed changes to be applied". Has that basis been justified for Impala? Are we talking data loss? Nope. It's a layer over the data substrate. Synchronization across a cluster? Nah. Where is the complexity? In this case, the RTC seems to stem from the choice of Gerrit, rather than some innate complexity. I *do* note that possibly committers could choose to commit directly, or choose to use Gerrit when they are unsure. Will the (P)PMC allow those direct commits? Or mandate Gerrit for every commit? Cheers, -g
Re: Impala commit policy
Thanks, Roman. For the record, I don’t plan to contribute to Impala or Kudu, and I don’t like strict commit policies such as RTC. But I wanted to stand up for “states' rights”, the right of podlings and projects to determine their own processes and cultures. Julian > On Dec 2, 2015, at 6:42 PM, Roman Shaposhnikwrote: > > On Wed, Dec 2, 2015 at 4:24 PM, Julian Hyde wrote: >> “No explicit commit policy” means that only committers can commit. >> It is each committer’s discretion whether they ask for others to review >> the change before they commit it, whether they check in code that doesn’t >> build, whether they run the test suite before committing. >> >> This policy is the bare constitutional minimum. We would all hope and expect >> that the community would quickly agree on some policies, but that is up the >> community. >> They are not stupid, they want to produce high-quality software, and they >> want to grow >> their community, and they will figure out a policy that achieves these goals. > > If that's the expectation that is internalized by the podling community than > I guess my concerns are somewhat taken care of to a point where I'd > be comfortable giving the proposal a +0. > > Let me explain why it is a +0 instead of +1. This will also be an opportunity > for me to clarify my seemingly inconsistent position with Kudu proposal (which > I deliberately +1ed). > > It all comes back to trust and inclusiveness. I thought that Greg was super > convincing at articulating that CTR policy is an indication, a proxy if you > will > for those qualities. If CTR is there -- I know that the community is > trusting and > inclusive. My -1 for Impala was based on their strong resistance to CTR as > a proxy measure of how ready they are to accept the "Apache Way". Them > fighting the idea of CTR gave me a strong indication that they are resisting > to being trusting and inclusive. > > Now, of course, as any astute student of logic will know, necessity > and sufficiency > is not the same thing. While a presence of CTR policy is a strong indication > of > the community getting the "Apache Way" the lack of it is not necessarily an > indication of them not getting it. Impala community, however has one extra > strike against it that wasn't allowing me to give it the same benefit > of the doubt > that Kudu community got (hence +1 for Kudu despite their instance on RTC). > > Impala and its existing community are *not* new to Open Source. They've been > out on GitHub since late 2012 and they have operated under a very explicit > BDFL model. Based on their past attitude towards external contributions > (personalized via statement of the project lead) I have strong > reasons to believe > that they are going to have really tough time adjusting to the Apache > governance > model AND when I saw that strong resistance to CTR that was it for me. > > That said, this thread and the expectations articulated by Tom and others > make me more comfortable. However, the lack of *actionable* suggestion > for how they are going to integrate with Apache governance model > makes it +0 and not +1. > > Thanks, > Roman. > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Impala commit policy
The vote to accept Impala into the incubator has passed (http://s.apache.org/u6r), however there are still some concerns about CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not a binary choice, and that it's entirely reasonable that different communities have different commit policies at the ASF. I think Julian Hyde's suggestion that the Impala podling start with no explicit commit policy is a good one. Incubation should be used as a time to work out what works best for a project. The initial Impala community should discuss the commit policy as they go through the process of setting up ASF infra and start growing the podling. In particular this will include how Gerrit can be used as a tool to facilitate reviews, and how that fits with ASF culture, which is something that other projects are looking at too. Cheers, Tom - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Impala commit policy
On Wed, Dec 2, 2015 at 11:01 AM, Tom Whitewrote: > ...I think Julian Hyde's suggestion that the Impala podling start with no > explicit commit policy is a good one. Incubation should be used as a > time to work out what works best for a project Big +1 -Bertrand - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Impala commit policy
On 2 Dec 2015, at 10:01, Tom White> wrote: The vote to accept Impala into the incubator has passed (http://s.apache.org/u6r), however there are still some concerns about CTR/RTC. My main takeaways from the CTR/RTC thread are that it's not a binary choice, and that it's entirely reasonable that different communities have different commit policies at the ASF. I think Julian Hyde's suggestion that the Impala podling start with no explicit commit policy is a good one. Incubation should be used as a time to work out what works best for a project. The initial Impala community should discuss the commit policy as they go through the process of setting up ASF infra and start growing the podling. In particular this will include how Gerrit can be used as a tool to facilitate reviews, and how that fits with ASF culture, which is something that other projects are looking at too. +1 what we might want to think about —for future proposals— is what makes a good recommended process during incubation, to help build that community, for dev & commit that is: what is best to not only ship something usable, but to pull in outside contributors. This is more than commit policy, it's how to organise discussions, what automated patch checking mechanisms to set up, etc, etc. That means more than just cargo-cult duplication of existing projects, but allowing incubating projects to innovate in this area, and perhaps having some more rigorous review; successors to [Rigby08] (*) -Steve [Rigby08], Open Source Software Peer Review Practices: A Case Study of the Apache Server, http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.226.2868=rep1=pdf, related work, https://scholar.google.co.uk/scholar?q=related:Rf0V3sIgPL6vvM:scholar.google.com/