Hey Zoltan, Thanks again for putting this together.
The original topic was dealing with a big pile of PRs. I'm starting to work through them and pair them down a bit before we kick in something automated (agree that we should eventually). However, I'm hampered by my ability to not be able to merge things myself. Thanks. On Thu, Jun 4, 2020 at 11:52 AM Zoltan Haindrich <k...@rxd.hu> wrote: > Hey David! > > > I'm trying to run the tests again and hopefully commit. Github has me > > listed as a "contributor" and lists Zoltan as a "Member of The Apache > > Software Foundation". Do you know how that list of members is managed? > > I don't know; maybe you should re-link your account? You could also try to > follow this wiki page: > > https://cwiki.apache.org/confluence/display/OPENWHISK/Accessing+Apache+GitHub+as+a+Committer > It was not my intention to move the patch approval process entirely to > github - it's just an option. > A balanced solution could be that we still upload the patch to the jira to > have it there as well - and continue commiting by pushing it manually. > I feel that this thread have deviated from it's original topic: I saw is > that the big pile of open PRs may backfire on the system if the jobs > "branch scanner" is triggered. > > >> Regarding auto-close in JIRA. Take a look at Apache Avro project. > I've > >> contributed there a little bit and I think they have this capability. > > Yes; it would make sense to auto-close jira tickets basaed on PR-s; but it > will pull up a lot of small questions(like what should it set as fix > version?) > Right now I don't see it as a big deal; I think that flaky tests are a > bigger problem - and there is also some kind of kubernetes issue from > time-to-time - which takes down > a full executor node with all its pods for no apparent reason. I'm putting > my efforts into fixing that... > > chers, > Zoltan > > > On 6/3/20 3:15 PM, David Mollitor wrote: > > Hmm, OK, working through this PR: > > > > https://github.com/apache/hive/pull/1052 > > > > I'm trying to run the tests again and hopefully commit. Github has me > > listed as a "contributor" and lists Zoltan as a "Member of The Apache > > Software Foundation". Do you know how that list of members is managed? > > > > https://github.com/orgs/apache/people?query= > > > > Thanks. > > > > On Wed, Jun 3, 2020 at 9:08 AM David Mollitor <dam6...@gmail.com> wrote: > > > >> Hello Zoltan, > >> > >> Regarding auto-close in JIRA. Take a look at Apache Avro project. I've > >> contributed there a little bit and I think they have this capability. > >> > >> Thanks. > >> > >> On Tue, Jun 2, 2020 at 8:00 PM Stamatis Zampetakis <zabe...@gmail.com> > >> wrote: > >> > >>> Hello, > >>> > >>> I am very happy working with the new system. Many thanks Zoltan! > >>> > >>> I find the bot a good idea and I think its worth trying it out. > >>> One thing to watch out is the case where contributors are willing to > push > >>> their work forward but there are no available reviewers to look to each > >>> case. > >>> I think people will reply to the bot once or twice but I don't think > they > >>> will do it much longer so we could take this into account for the > >>> configuration of the bot. > >>> > >>> Regarding merge squash option there might be a small caveat. I don't > know > >>> if it is possible to retain the information about the person who > performed > >>> the merge. > >>> According to the discussion in [1] it seems that the committer in this > >>> case > >>> will appear to be the GitHub account. > >>> This might not be a big problem for Hive since the reviewer's name is > part > >>> of the commit message so the credit and responsibility is not lost. > >>> > >>> Best, > >>> Stamatis > >>> > >>> [1] https://github.com/isaacs/github/issues/1303 > >>> > >>> > >>> > >>> On Tue, Jun 2, 2020 at 9:26 PM Zoltan Haindrich <k...@rxd.hu> wrote: > >>> > >>>> > >>>> > >>>> On 6/2/20 9:15 PM, David Mollitor wrote: > >>>>> I use a personal account for GitHub and it's not synced with my > >>> official > >>>>> Apache account. How do I go about registering my Apache account with > >>>>> GitHub so I can merge through their interface? > >>>> > >>>> IIRC I've linked my account by using this interface: > >>>> https://gitbox.apache.org/setup/ > >>>> > >>>>> > >>>>> In the meanwhile, can you assist with a merge here? :) > >>>>> > >>>> > >>>> sure; I think you should also add dmolli...@apache.org as a secondary > >>>> email to your github account > >>>> > >>>> About the open pr stuff: I still think our best approach of handling > >>> those > >>>> things would be to close most of that 400 or so PRs...easiest would be > >>> to > >>>> install the bot (at > >>>> least temporarily) > >>>> https://issues.apache.org/jira/browse/HIVE-23590 > >>>> what do you think? > >>>> > >>>> cheers, > >>>> Zoltan > >>>> > >>>> > >>>>> https://github.com/apache/hive/pull/1045 > >>>>> > >>>>> Thanks! > >>>>> > >>>>> On Tue, Jun 2, 2020 at 10:21 AM Zoltan Haindrich <k...@rxd.hu> > wrote: > >>>>> > >>>>>> > >>>>>> > >>>>>> On 6/2/20 3:10 PM, David Mollitor wrote: > >>>>>>> I think we might want to take one manual pass across the board. It > >>>> will > >>>>>>> most likely take more than 7 days to get through them all, so it > >>> may be > >>>>>>> closing things that are legitimate. > >>>>>> > >>>>>> yeah...a manual pass would be good; I went thru around 10 or so > >>> before > >>>>>> I've wrote the first mail in this thread... > >>>>>> and I definetly don't want to go thru 400 - so I would preffer the > >>> bot > >>>> :D > >>>>>> > >>>>>>> > >>>>>>> One low hanging fruit (that applied to one of my PRs). The JIRA it > >>> was > >>>>>>> associated with was already closed. Is there a way to target > those? > >>>>>> > >>>>>> yes; there might be certainly a lot of those...(that's why I've > >>> estimate > >>>>>> to 1/3 to be applicable) > >>>>>> but filtering out even this is an awful lot of work (or it might > >>> involve > >>>>>> writing a "bot")... > >>>>>> if it's important enough the contributor could reopen / rebase the > >>>> patch. > >>>>>> We could try to communicate the non-hostaile intention in the > message > >>>>>> placed by the bot. > >>>>>> The current message is the stale PRs would get is: > >>>>>> "This pull request has been automatically marked as stale because it > >>> has > >>>>>> not had recent activity. It will be closed if no further activity > >>>> occurs." > >>>>>> > >>>>>>> Also, I have submitted my first PR to test out the new system. It > >>>>>>> has passed tests. Ashutoshc has generously provided a +1. What's > >>> the > >>>>>>> next step to get it merged into the master? Do I download the > patch > >>>> from > >>>>>>> Github and apply manually using my Apache credentials? Is the > >>> "merge" > >>>>>>> feature setup in Github? As I understand it, GitHub is only > >>> mirroring > >>>>>> the > >>>>>>> Apache git system. Whatever the process we need an update in the > >>>>>>> HowToContribute docs. > >>>>>> > >>>>>> That's an interesting question; the github repo is linked to the > >>> apache > >>>>>> repo - so you may push/merge/whatever on the github interface; it > >>> will > >>>> work. > >>>>>> Github supports 3 modes to merge PRs: > >>>>>> * We should definetly disable the "merge" option as that will just > >>>> create > >>>>>> a internation railways station from our history :) > >>>>>> * rebase doesn't make it easier for reviewier to keep track new > >>>>>> changes...because the PR owner have to continuosly force push the > >>> branch > >>>>>> * squash merge work great - and I remembered that it changes the > >>> author > >>>> to > >>>>>> the user pushing the "squash" button; however right now it seems > >>> that it > >>>>>> changes the author to > >>>>>> the "user who opened the pr" which looks good-enough for me! > >>>>>> (I've added the neccessary .asf.yaml changes to the existing PR) > >>>>>> > >>>>>> cheers, > >>>>>> Zoltan > >>>>>> > >>>>>>> https://github.com/apache/hive/pull/1045 > >>>>>>> > >>>>>> > >>>> > >>> > https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-ApplyingaPatch > >>>>>>> > >>>>>>> > >>>>>>> Thanks! > >>>>>>> > >>>>>>> On Tue, Jun 2, 2020 at 4:58 AM Zoltan Haindrich <k...@rxd.hu> > >>> wrote: > >>>>>>> > >>>>>>>> I think to use "probot" we would need to ask infra to configure > the > >>>>>>>> "probot" github app. > >>>>>>>> It seems to me that the stale plugin from github actions provides > >>>> almost > >>>>>>>> the same functionaluty - as they seem to be more or less identical > >>> in > >>>>>>>> features I would go with the > >>>>>>>> latter. > >>>>>>>> > >>>>>>>> I've opened a pr to enable stale on the hive repo: > >>>>>>>> by default it will mark as stale afte 60 days; and close it if > >>> there > >>>> is > >>>>>> no > >>>>>>>> activity for another 7 days > >>>>>>>> https://github.com/apache/hive/pull/1049 > >>>>>>>> > >>>>>>>> cheers, > >>>>>>>> Zoltan > >>>>>>>> > >>>>>>>> > >>>>>>>> On 6/2/20 5:00 AM, Ashutosh Chauhan wrote: > >>>>>>>>> How about using stalebot : https://github.com/probot/stale ? > >>>>>>>>> > >>>>>>>>> On Mon, Jun 1, 2020 at 12:20 PM Zoltán Haindrich <k...@rxd.hu> > >>>> wrote: > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Hey David, > >>>>>>>>>> > >>>>>>>>>> On June 1, 2020 3:52:05 PM GMT+02:00, David Mollitor < > >>>>>> dam6...@gmail.com > >>>>>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>>> Any idea how long it will take to run precomit on all existing > >>> PRs? > >>>>>>>>>>> > >>>>>>>>>> I'm not entirely sure, but a rough estimate could be: > >>>>>>>>>> * not every pr is mergeable; there are many which was already > >>>>>>>>>> merged/outdated/etc...lets estimate that 1/3 is mergeable > >>>>>>>>>> * every pr runs for at least 1 hours > >>>>>>>>>> > >>>>>>>>>> this would mean 430/3*1/24 days of test execution which is at > >>> least > >>>> 6 > >>>>>>>> days. > >>>>>>>>>> > >>>>>>>>>> I see little to no value in running tests on archaic prs. > >>>>>>>>>> We could also configure an automatism to close prs after some > >>> time > >>>> of > >>>>>>>>>> inactivity > >>>>>>>>>> https://github.com/actions/stale/blob/master/README.md > >>>>>>>>>> The good side of this is that it will get rid of ancient prs; > >>>> however > >>>>>> it > >>>>>>>>>> might seem rude to a contributor in case he is waiting for > >>> feedback > >>>> or > >>>>>>>>>> something.... > >>>>>>>>>> Even with that argument I think we should configure it at least > >>> for > >>>> a > >>>>>>>> few > >>>>>>>>>> days to get rid of the dangling prs of almost a decade ! (pr#2 > is > >>>>>>>> opened in > >>>>>>>>>> 2011)... > >>>>>>>>>> What do you think? > >>>>>>>>>> > >>>>>>>>>> cheers, > >>>>>>>>>> Zoltan > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> On Mon, Jun 1, 2020 at 9:49 AM Panos Garefalakis < > >>>> panga...@gmail.com > >>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Same here, however, there are still ~ 430 PRs pending on > >>> master. > >>>>>>>>>>>> Thanks Zoltan for this great initiative! > >>>>>>>>>>>> > >>>>>>>>>>>> Cheers, > >>>>>>>>>>>> Panagiotis > >>>>>>>>>>>> > >>>>>>>>>>>> On Mon, Jun 1, 2020 at 2:33 PM David Mollitor < > >>> dam6...@gmail.com> > >>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Thanks so much for the work on this. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Just cleaned up mine. > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Sat, May 30, 2020 at 10:16 AM Zoltan Haindrich < > >>> k...@rxd.hu> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hey All, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> The new test executor will pick up any PR which doesn't yet > >>> have > >>>>>>>>>>> a test > >>>>>>>>>>>>>> result - now that the patch is on the master; every PR which > >>> is > >>>>>>>>>>>> mergeable > >>>>>>>>>>>>>> with the master branch is > >>>>>>>>>>>>>> a good candidate - so the right move would be to clean up > >>> our PR > >>>>>>>>>>>> backlog. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I would like to ask everyone to look at > >>>>>>>>>>>>>> https://github.com/apache/hive/pulls > >>>>>>>>>>>>>> and close some PRs which are already submitted or just > >>> leftovers > >>>>>>>>>> >from - > >>>>>>>>>>>>>> primarily I would ask you to look at PRs opened by > >>> yourself... > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> cheers, > >>>>>>>>>>>>>> Zoltan > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Zoltán Haindrich > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > > >
