Re: Github integration for Hadoop

; wrote: > > > https://issues.apache.org/jira/browse/INFRA-11597 has been filed for this. > > > > -Vinay > > > > -Original Message- > > From: Colin McCabe [mailto:co...@cmccabe.xyz] > > Sent: 05 April 2016 08:07 > > To: common-dev@hadoop.apache.org &g

Re: Github integration for Hadoop

- > From: Colin McCabe [mailto:co...@cmccabe.xyz] > Sent: 05 April 2016 08:07 > To: common-dev@hadoop.apache.org > Subject: Re: Github integration for Hadoop > > Yes, please. Let's disable these mails. > > C. > > On Mon, Apr 4, 2016, at 06:21, Vinayakumar B wrote: &

RE: Github integration for Hadoop

https://issues.apache.org/jira/browse/INFRA-11597 has been filed for this. -Vinay -Original Message- From: Colin McCabe [mailto:co...@cmccabe.xyz] Sent: 05 April 2016 08:07 To: common-dev@hadoop.apache.org Subject: Re: Github integration for Hadoop Yes, please. Let's disable

Re: Github integration for Hadoop

Yes, please. Let's disable these mails. C. On Mon, Apr 4, 2016, at 06:21, Vinayakumar B wrote: > bq. We don't spam common-dev about every time a new patch attachment > gets posted > to an existing JIRA. We shouldn't do that for github either. > > Is there any update on this. ? > Any INFRA tick

Re: Github integration for Hadoop

bq. We don't spam common-dev about every time a new patch attachment gets posted to an existing JIRA. We shouldn't do that for github either. Is there any update on this. ? Any INFRA ticket filed to disable these mails? Because, I am sure, people would have got frustrated by seeing mails generat

Re: Github integration for Hadoop

Hi Eric, My comment wrt GH permissions is available in context earlier in the thread, pasting again here: === Great, glad to hear it. That wasn't mentioned in the email thread or on the INFRA ticket, and the INFRA ticket mentions these integrations: Commit Comment, Create, Issue Comment, Is

Re: Github integration for Hadoop

We don't spam common-dev about every time a new patch attachment gets posted to an existing JIRA. We shouldn't do that for github either. +1 to Andrew and Steve's suggestion of disabling these emails (or at least sending them to a separate list that people can opt in to). best, Colin On Sat, No

Re: Github integration for Hadoop

Hi Andrew, Many Apache projects already have Github integration. Chukwa also has Github integration. Pull request can be integrated into JIRA, i.e. https://issues.apache.org/jira/browse/CHUKWA-783 This allows code review process to happen at per line level on Github, or comment on JIRA for summ

Re: Github integration for Hadoop

Has there been any progress on locking down the Github permissions like I asked above? It's been about 3 weeks. Steve also just asked on another thread to turn off the emails to common-dev. Since we're sticking with JIRA as the source-of-truth, the common-dev emails aren't useful. On Fri, Nov 13,

Re: Github integration for Hadoop

Hi Colin! If Yetus is working on an issue and can't tell what the intended branch is it points folks to project specific contribution guides. For Hadoop, the patch naming for specific branches should be covered in this section of Hadoop's contribution guide: http://wiki.apache.org/hadoop/HowToCo

Re: Github integration for Hadoop

Thanks, Allen, I wasn't aware that Yetus now supported testing for other branches. Is there documentation about how to name the branch so it gets tested? best, Colin On Fri, Nov 13, 2015 at 7:52 AM, Allen Wittenauer wrote: > >> On Nov 12, 2015, at 10:55 AM, Colin P. McCabe wrote: >> >> gerrit

Re: Github integration for Hadoop

> On Nov 12, 2015, at 10:55 AM, Colin P. McCabe wrote: > > gerrit has a button on the UI to cherry-pick to different branches. > The button creates separate "gerrit changes" which you can then > commit. Eventually we could hook those up to Jenkins-- something > which we've never been able to do

Re: Github integration for Hadoop

gerrit has a button on the UI to cherry-pick to different branches. The button creates separate "gerrit changes" which you can then commit. Eventually we could hook those up to Jenkins-- something which we've never been able to do for different branches with the patch-file-based workflow. best, C

Re: Github integration for Hadoop

> On 6 Nov 2015, at 15:21, Owen O'Malley wrote: > > On Sun, Nov 1, 2015 at 2:07 PM, Chris Douglas wrote: > >> Wow, this happened quickly. >> >> Owen, could you please create a Wiki describing the proposal and >> cataloging infra references so others can understand the >> implementation in det

Re: Github integration for Hadoop

I have to some scripts that are tailored for the current git workflow, which is available at https://github.com/haohui/hdcli It's relatively straightforward to make it support github. ~Haohui On Tue, Nov 10, 2015 at 9:31 AM, Karthik Kambatla wrote: > Owen: Thanks for putting the documentation

Re: Github integration for Hadoop

Owen: Thanks for putting the documentation together. I think I understand the proposal better now. I agree that reviewing on github is easier than having to download the patch, apply locally, and for reviews copy-paste code to the JIRA. This, we get from RB or any other review tool as well. The c

Re: Github integration for Hadoop

On Sun, Nov 1, 2015 at 2:07 PM, Chris Douglas wrote: > Wow, this happened quickly. > > Owen, could you please create a Wiki describing the proposal and > cataloging infra references so others can understand the > implementation in detail? Even after reading this thread, I'm still > confused what

Re: Github integration for Hadoop

On Sun, Nov 1, 2015 at 12:52 PM, Allen Wittenauer wrote: > >> On Nov 1, 2015, at 6:05 AM, Tsuyoshi Ozawa wrote: >> >> Thank you for starting this discussion. It's good for us to rethink >> our workflow to grow community. >> However, at the moment, my concern is that we can put more pressure on >>

Re: Github integration for Hadoop

+1 for setting up a gerrit instance to try it out. cheers, Colin On Mon, Nov 2, 2015 at 7:13 AM, Zhe Zhang wrote: >> >> So I think Gerrit/Crucible/whatever are on the table, with some work. >> Does anyone want to take the token for asking other projects and >> assembling a proposal for an altern

Re: Github integration for Hadoop

> > So I think Gerrit/Crucible/whatever are on the table, with some work. > Does anyone want to take the token for asking other projects and > assembling a proposal for an alternative they're excited about? Thanks for the update and the suggestion Chris, I think it's a great idea. I'd like to vol

Re: Github integration for Hadoop

Wow, this happened quickly. Owen, could you please create a Wiki describing the proposal and cataloging infra references so others can understand the implementation in detail? Even after reading this thread, I'm still confused what changes this proposes and how the integration works. A document pa

Re: Github integration for Hadoop

> On Nov 1, 2015, at 6:05 AM, Tsuyoshi Ozawa wrote: > > Thank you for starting this discussion. It's good for us to rethink > our workflow to grow community. > However, at the moment, my concern is that we can put more pressure on > Yetus community if we move the main workflows into github. > >

Re: Github integration for Hadoop

Thank you for starting this discussion. It's good for us to rethink our workflow to grow community. However, at the moment, my concern is that we can put more pressure on Yetus community if we move the main workflows into github. Allen and Sean, what do you think? Is it good timing for Yetus commu

Re: Github integration for Hadoop

> On 31 Oct 2015, at 17:58, Colin P. McCabe wrote: > > Thanks for your responses here. It sounds like the proposal here is > for doing code reviews on GH, but still doing commits in our existing > way. Since it wasn't spelled out in the initial proposal, I > interpreted it as doing both review

Re: Github integration for Hadoop

Thanks for your responses here. It sounds like the proposal here is for doing code reviews on GH, but still doing commits in our existing way. Since it wasn't spelled out in the initial proposal, I interpreted it as doing both reviews and commits on GH, like Spark does-- which I think is problema

Re: Github integration for Hadoop

> On 30 Oct 2015, at 17:15, Colin P. McCabe wrote: > > I think the Spark guys eventually built some kind of UI on top of > github to help them search through pull requests. We would probably > also need something like this. https://spark-prs.appspot.com/users They do have to impose naming sc

Re: Github integration for Hadoop

Thanks Owen. I'll volunteer myself and Colin to review the docs when you put them together. Regarding open q's, from my earlier email: = Great, glad to hear it. That wasn't mentioned in the email thread or on the INFRA ticket, and the INFRA ticket mentions these integrations: Commit Comment

Re: Github integration for Hadoop

On Fri, Oct 30, 2015 at 2:37 PM, Andrew Wang wrote: > > Strongly agree that we need documentation for all this. Agreed. > Some of my open questions are also still pending. > Which open questions? Owen, are you chasing all of this down? Do we need some JIRAs to track? > Yeah, I'll take a fi

Re: Github integration for Hadoop

On Fri, Oct 30, 2015 at 1:59 PM, Colin P. McCabe wrote: > I am -1 on the current GH stuff, just because I feel like there wasn't > enough discussion, testing, and documentation of it. The initial > proposal had no details and was implemented before any of the people > who had had misgivings on t

Re: Github integration for Hadoop

I am -1 on the current GH stuff, just because I feel like there wasn't enough discussion, testing, and documentation of it. The initial proposal had no details and was implemented before any of the people who had had misgivings on the previous email thread had a chance to even see it. It's a big

Re: Github integration for Hadoop

On Fri, Oct 30, 2015 at 1:22 PM, Allen Wittenauer wrote: > >> * Have we tried our precommit on PRs yet? Does it work for multiple >> branches? Is there a way to enforce rebase+squash vs. merge on the PR, >> since, per Allen, Yetus requires one commit to work? > > > I don’t know about the J

Re: Github integration for Hadoop

> > > > 2) Better attribution of contributors with their GH id (Arpit) > > > > This doesn't happen very naturally other than the pull requests are > typically shown in their fork of the apache repo. > > It happens if we used PRs for integration, but yes I agree that it does not if GH is just used a

Re: Github integration for Hadoop

> * Have we tried our precommit on PRs yet? Does it work for multiple > branches? Is there a way to enforce rebase+squash vs. merge on the PR, > since, per Allen, Yetus requires one commit to work? I don’t know about the Jenkins-side of things (e.g., how does Jenkins trigger a build?).

Re: Github integration for Hadoop

> > > >> > >> > If that's the case, we should also look at review alternatives > >> > like RB and Crucible. > >> > >> Okay by me if the community consensus supports one of them. The fact > that > >> they exist but no one uses them is not a ringing endorsement. > >> > >> HBase uses reviewboard, as d

Re: Github integration for Hadoop

On Fri, Oct 30, 2015 at 10:20 AM, Andrew Wang wrote: > I'm trying to slow this discussion down so we can: > > a) Determine the problem(s) we're trying to solve > The problem is that many people like the github code review tools. Enabling the github integration does not mean that everyone has to

Re: Github integration for Hadoop

I'm trying to slow this discussion down so we can: a) Determine the problem(s) we're trying to solve b) See how the proposed solutions meet this problem The flood of +1's were made before a full proposal of what "enabling github integration" means, which has only been really specified in Owen's m

Re: Github integration for Hadoop

I think we should take more than 24 hours of discussion to make a big, fundamental change to our code review process. I used github while working on the Spark project, and frankly I didn't like it. I didn't like the way it split the discussion between JIRA and github. Also, often people would fi

Re: Github integration for Hadoop

It seems like there is overwhelming support for enabling the github integration, so I went ahead and filed the infra ticket . This is explicitly not changing the way that we commit on Hadoop and commits should be squashed an

Re: Github integration for Hadoop

+1 if the current review process is kept. Pros: * Using PRs is good for increasing developers because PRs are very familiar for many developers. * Better UI. (no need to copy the diff manually for reviewing patches!) Cons: * Discussions can be split. However, according to the below blog post

Re: Github integration for Hadoop

Andrew, look at Spark github. They use PRs and I do not see extra merge commits. Committer can do fast-farward merge using git command line. PRs are used to leave inline feedbacks for the fix On Oct 29, 2015 1:34 PM, "Andrew Wang" wrote: > Has anything changed regarding the github integration sin

Re: Github integration for Hadoop

On 10/29/15, 5:14 PM, "Andrew Wang" wrote: >On Thu, Oct 29, 2015 at 4:58 PM, Arpit Agarwal >wrote: > >> On 10/29/15, 3:48 PM, "Andrew Wang" wrote: >> >> >> > If we pushed for it, I think it could happen. >> >> Gerrit support is a complete unknown. The community response to date >> supports Gi

Re: Github integration for Hadoop

> On Oct 29, 2015, at 5:14 PM, Andrew Wang wrote: > > However, if we were to use GH for integration, it would be with the > auto-squash to avoid the merge commit. Would this preserve the correct > attribution? FWIW, Yetus *really really really* wants a single commit when it comes to di

Re: Github integration for Hadoop

On Thu, Oct 29, 2015 at 4:58 PM, Arpit Agarwal wrote: > On 10/29/15, 3:48 PM, "Andrew Wang" wrote: > > > > If we pushed for it, I think it could happen. > > Gerrit support is a complete unknown. The community response to date > supports Github integration. Github will appeal to new contributors

Re: Github integration for Hadoop

On 10/29/15, 3:48 PM, "Andrew Wang" wrote: > If we pushed for it, I think it could happen. Gerrit support is a complete unknown. The community response to date supports Github integration. Github will appeal to new contributors as their Github profiles will reflect their work. I'd be interes

Re: Github integration for Hadoop

On Thu, Oct 29, 2015 at 3:29 PM, Owen O'Malley wrote: > On Thu, Oct 29, 2015 at 2:42 PM, Andrew Wang > wrote: > > > > > If I had my druthers, we'd focus on Gerrit rather than Github as a > > successor review system. Better review, better fits our dev workflow. The > > only question is JIRA integ

Re: Github integration for Hadoop

On Thu, Oct 29, 2015 at 3:12 PM, Owen O'Malley wrote: > On Thu, Oct 29, 2015 at 1:34 PM, Andrew Wang > wrote: > > > Has anything changed regarding the github integration since the last time > > we discussed this? > > > There is a lot more experience with it now and opinions change over time. > C

Re: Github integration for Hadoop

On Thu, Oct 29, 2015 at 2:42 PM, Andrew Wang wrote: > > If I had my druthers, we'd focus on Gerrit rather than Github as a > successor review system. Better review, better fits our dev workflow. The > only question is JIRA integration. > Last time I asked, Apache Infra wasn't willing to support

Re: Github integration for Hadoop

On Thu, Oct 29, 2015 at 1:34 PM, Andrew Wang wrote: > Has anything changed regarding the github integration since the last time > we discussed this? There is a lot more experience with it now and opinions change over time. Clearly, there is overwhelming support for this idea now. > There are

Re: Github integration for Hadoop

On Thu, Oct 29, 2015 at 2:29 PM, Steve Loughran wrote: > > > On 29 Oct 2015, at 20:34, Andrew Wang wrote: > > > > Has anything changed regarding the github integration since the last time > > we discussed this? That blog post is from 2014, and we discussed > > alternative review systems earlier

Re: Github integration for Hadoop

> On 29 Oct 2015, at 20:34, Andrew Wang wrote: > > Has anything changed regarding the github integration since the last time > we discussed this? That blog post is from 2014, and we discussed > alternative review systems earlier in 2015. > > Colin specifically was concerned about forking the di

Re: Github integration for Hadoop

+1 (non-binding) On Thursday, October 29, 2015 2:26 PM, Arpit Agarwal wrote: +1, thanks for proposing it. On 10/29/15, 10:47 AM, "Owen O'Malley" wrote: >All, >  For code & patch review, many of the newer projects are using the Github >pull request integration. You can read ab

Re: Github integration for Hadoop

Has anything changed regarding the github integration since the last time we discussed this? That blog post is from 2014, and we discussed alternative review systems earlier in 2015. Colin specifically was concerned about forking the discussion between JIRA and other places: http://search-hadoop.

Re: Github integration for Hadoop

+1, thanks for proposing it. On 10/29/15, 10:47 AM, "Owen O'Malley" wrote: >All, > For code & patch review, many of the newer projects are using the Github >pull request integration. You can read about it here: > >https://blogs.apache.org/infra/entry/improved_integration_between_apache_and

Re: Github integration for Hadoop

Don’t think we need a vote. If someone can demonstrate how this works end-to-end, and enough folks find it useful, we can start using it. There is no need for a mandate. +Vinod > On Oct 29, 2015, at 12:01 PM, Xiaoyu Yao wrote: > > +1, should we start a vote on this? > > > > > On 10/29/15,

Re: Github integration for Hadoop

It looks like there's pretty good consensus. Why do we need a VOTE thread? Perhaps better for someone to submit a patch with proposed text for hte contribution guide[1]? -Sean [1]: http://wiki.apache.org/hadoop/HowToContribute On Thu, Oct 29, 2015 at 2:01 PM, Xiaoyu Yao wrote: > +1, should we

Re: Github integration for Hadoop

+1, should we start a vote on this? On 10/29/15, 11:54 AM, "Ashish" wrote: >+1 > >On Thu, Oct 29, 2015 at 11:51 AM, Mingliang Liu wrote: >> +1 (non-binding) >> >> Mingliang Liu >> Member of Technical Staff - HDFS, >> Hortonworks Inc. >> m...@hortonworks.com >> >> >> >>> On Oct 29, 2015, at 1

Re: Github integration for Hadoop

+1 (non-binding) On Thu, Oct 29, 2015 at 1:54 PM, Ashish wrote: > +1 > > On Thu, Oct 29, 2015 at 11:51 AM, Mingliang Liu > wrote: > > +1 (non-binding) > > > > Mingliang Liu > > Member of Technical Staff - HDFS, > > Hortonworks Inc. > > m...@hortonworks.com > > > > > > > >> On Oct 29, 2015, at 1

Re: Github integration for Hadoop

+1 On Thu, Oct 29, 2015 at 11:51 AM, Mingliang Liu wrote: > +1 (non-binding) > > Mingliang Liu > Member of Technical Staff - HDFS, > Hortonworks Inc. > m...@hortonworks.com > > > >> On Oct 29, 2015, at 10:55 AM, Hitesh Shah wrote: >> >> +1 on supporting patch contributions through github pull re

Re: Github integration for Hadoop

+1 (non-binding) Mingliang Liu Member of Technical Staff - HDFS, Hortonworks Inc. m...@hortonworks.com > On Oct 29, 2015, at 10:55 AM, Hitesh Shah wrote: > > +1 on supporting patch contributions through github pull requests. > > — Hitesh > > On Oct 29, 2015, at 10:47 AM, Owen O'Malley wrot

Re: Github integration for Hadoop

+1 On Thu, Oct 29, 2015 at 10:55 AM, Hitesh Shah wrote: > +1 on supporting patch contributions through github pull requests. > > — Hitesh > > On Oct 29, 2015, at 10:47 AM, Owen O'Malley wrote: > >> All, >> For code & patch review, many of the newer projects are using the Github >> pull request

Re: Github integration for Hadoop

+1 on supporting patch contributions through github pull requests. — Hitesh On Oct 29, 2015, at 10:47 AM, Owen O'Malley wrote: > All, > For code & patch review, many of the newer projects are using the Github > pull request integration. You can read about it here: > > https://blogs.apache.or