Hey Zoltan, I think this stale bot is too aggressive. The Apache Druid program marks stale after 200+ days and closes 4 weeks after that. I've found a few interesting PRs that are old, and are being closed, but I just haven't gotten around to them and with such a big backlog, there needs to be a lot of allowance to start. just my 2 cents.
Thanks. On Fri, Jun 5, 2020 at 10:24 AM David Mollitor <dam6...@gmail.com> wrote: > No need to respond, just want to document somewhere: > > Part of the process involved enabling 2FA on my Github account. After > that, pushing to repositories failed with an authentication error. After > 2FA, it is required to use a access token instead of a password: > > > https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line > > > On Thu, Jun 4, 2020 at 5:49 PM David Mollitor <dam6...@gmail.com> wrote: > >> Hello Zoltan >> >> I just got access. Not sure if this was a manual process performed by >> someone on this thread or something automated, but I got an email >> acknowledging. >> >> I'm starting to close some old PRs. >> >> Thanks. >> >> On Thu, Jun 4, 2020 at 5:20 PM David Mollitor <dam6...@gmail.com> wrote: >> >>> Zoltan, >>> >>> >>> https://cwiki.apache.org/confluence/display/OPENWHISK/Accessing+Apache+GitHub+as+a+Committer >>> >>> >>> That did it for me. Thanks so much,... I'll try to remember to get this >>> in our Hive docs. >>> >>> I still however cannot close or merge anything. I get the warning: >>> >>> "Only those with write access to this repository can merge pull >>> requests." >>> >>> Any idea how to get my account write access? >>> >>> Thanks. >>> >>> On Thu, Jun 4, 2020 at 12:14 PM David Mollitor <dam6...@gmail.com> >>> wrote: >>> >>>> Hey Zoltan, >>>> >>>> Thanks again for putting this together. >>>> >>>> The original topic was dealing with a big pile of PRs. I'm starting to >>>> work through them and pair them down a bit before we kick in something >>>> automated (agree that we should eventually). However, I'm hampered by my >>>> ability to not be able to merge things myself. >>>> >>>> Thanks. >>>> >>>> On Thu, Jun 4, 2020 at 11:52 AM Zoltan Haindrich <k...@rxd.hu> wrote: >>>> >>>>> Hey David! >>>>> >>>>> > I'm trying to run the tests again and hopefully commit. Github has >>>>> me >>>>> > listed as a "contributor" and lists Zoltan as a "Member of The >>>>> Apache >>>>> > Software Foundation". Do you know how that list of members is >>>>> managed? >>>>> >>>>> I don't know; maybe you should re-link your account? You could also >>>>> try to follow this wiki page: >>>>> >>>>> https://cwiki.apache.org/confluence/display/OPENWHISK/Accessing+Apache+GitHub+as+a+Committer >>>>> It was not my intention to move the patch approval process entirely to >>>>> github - it's just an option. >>>>> A balanced solution could be that we still upload the patch to the >>>>> jira to have it there as well - and continue commiting by pushing it >>>>> manually. >>>>> I feel that this thread have deviated from it's original topic: I saw >>>>> is that the big pile of open PRs may backfire on the system if the jobs >>>>> "branch scanner" is triggered. >>>>> >>>>> >> Regarding auto-close in JIRA. Take a look at Apache Avro >>>>> project. I've >>>>> >> contributed there a little bit and I think they have this >>>>> capability. >>>>> >>>>> Yes; it would make sense to auto-close jira tickets basaed on PR-s; >>>>> but it will pull up a lot of small questions(like what should it set as >>>>> fix >>>>> version?) >>>>> Right now I don't see it as a big deal; I think that flaky tests are a >>>>> bigger problem - and there is also some kind of kubernetes issue from >>>>> time-to-time - which takes down >>>>> a full executor node with all its pods for no apparent reason. I'm >>>>> putting my efforts into fixing that... >>>>> >>>>> chers, >>>>> Zoltan >>>>> >>>>> >>>>> On 6/3/20 3:15 PM, David Mollitor wrote: >>>>> > Hmm, OK, working through this PR: >>>>> > >>>>> > https://github.com/apache/hive/pull/1052 >>>>> > >>>>> > I'm trying to run the tests again and hopefully commit. Github has >>>>> me >>>>> > listed as a "contributor" and lists Zoltan as a "Member of The Apache >>>>> > Software Foundation". Do you know how that list of members is >>>>> managed? >>>>> > >>>>> > https://github.com/orgs/apache/people?query= >>>>> > >>>>> > Thanks. >>>>> > >>>>> > On Wed, Jun 3, 2020 at 9:08 AM David Mollitor <dam6...@gmail.com> >>>>> wrote: >>>>> > >>>>> >> Hello Zoltan, >>>>> >> >>>>> >> Regarding auto-close in JIRA. Take a look at Apache Avro project. >>>>> I've >>>>> >> contributed there a little bit and I think they have this >>>>> capability. >>>>> >> >>>>> >> Thanks. >>>>> >> >>>>> >> On Tue, Jun 2, 2020 at 8:00 PM Stamatis Zampetakis < >>>>> zabe...@gmail.com> >>>>> >> wrote: >>>>> >> >>>>> >>> Hello, >>>>> >>> >>>>> >>> I am very happy working with the new system. Many thanks Zoltan! >>>>> >>> >>>>> >>> I find the bot a good idea and I think its worth trying it out. >>>>> >>> One thing to watch out is the case where contributors are willing >>>>> to push >>>>> >>> their work forward but there are no available reviewers to look to >>>>> each >>>>> >>> case. >>>>> >>> I think people will reply to the bot once or twice but I don't >>>>> think they >>>>> >>> will do it much longer so we could take this into account for the >>>>> >>> configuration of the bot. >>>>> >>> >>>>> >>> Regarding merge squash option there might be a small caveat. I >>>>> don't know >>>>> >>> if it is possible to retain the information about the person who >>>>> performed >>>>> >>> the merge. >>>>> >>> According to the discussion in [1] it seems that the committer in >>>>> this >>>>> >>> case >>>>> >>> will appear to be the GitHub account. >>>>> >>> This might not be a big problem for Hive since the reviewer's name >>>>> is part >>>>> >>> of the commit message so the credit and responsibility is not lost. >>>>> >>> >>>>> >>> Best, >>>>> >>> Stamatis >>>>> >>> >>>>> >>> [1] https://github.com/isaacs/github/issues/1303 >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> On Tue, Jun 2, 2020 at 9:26 PM Zoltan Haindrich <k...@rxd.hu> >>>>> wrote: >>>>> >>> >>>>> >>>> >>>>> >>>> >>>>> >>>> On 6/2/20 9:15 PM, David Mollitor wrote: >>>>> >>>>> I use a personal account for GitHub and it's not synced with my >>>>> >>> official >>>>> >>>>> Apache account. How do I go about registering my Apache account >>>>> with >>>>> >>>>> GitHub so I can merge through their interface? >>>>> >>>> >>>>> >>>> IIRC I've linked my account by using this interface: >>>>> >>>> https://gitbox.apache.org/setup/ >>>>> >>>> >>>>> >>>>> >>>>> >>>>> In the meanwhile, can you assist with a merge here? :) >>>>> >>>>> >>>>> >>>> >>>>> >>>> sure; I think you should also add dmolli...@apache.org as a >>>>> secondary >>>>> >>>> email to your github account >>>>> >>>> >>>>> >>>> About the open pr stuff: I still think our best approach of >>>>> handling >>>>> >>> those >>>>> >>>> things would be to close most of that 400 or so PRs...easiest >>>>> would be >>>>> >>> to >>>>> >>>> install the bot (at >>>>> >>>> least temporarily) >>>>> >>>> https://issues.apache.org/jira/browse/HIVE-23590 >>>>> >>>> what do you think? >>>>> >>>> >>>>> >>>> cheers, >>>>> >>>> Zoltan >>>>> >>>> >>>>> >>>> >>>>> >>>>> https://github.com/apache/hive/pull/1045 >>>>> >>>>> >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> >>>>> On Tue, Jun 2, 2020 at 10:21 AM Zoltan Haindrich <k...@rxd.hu> >>>>> wrote: >>>>> >>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> On 6/2/20 3:10 PM, David Mollitor wrote: >>>>> >>>>>>> I think we might want to take one manual pass across the >>>>> board. It >>>>> >>>> will >>>>> >>>>>>> most likely take more than 7 days to get through them all, so >>>>> it >>>>> >>> may be >>>>> >>>>>>> closing things that are legitimate. >>>>> >>>>>> >>>>> >>>>>> yeah...a manual pass would be good; I went thru around 10 or so >>>>> >>> before >>>>> >>>>>> I've wrote the first mail in this thread... >>>>> >>>>>> and I definetly don't want to go thru 400 - so I would preffer >>>>> the >>>>> >>> bot >>>>> >>>> :D >>>>> >>>>>> >>>>> >>>>>>> >>>>> >>>>>>> One low hanging fruit (that applied to one of my PRs). The >>>>> JIRA it >>>>> >>> was >>>>> >>>>>>> associated with was already closed. Is there a way to target >>>>> those? >>>>> >>>>>> >>>>> >>>>>> yes; there might be certainly a lot of those...(that's why I've >>>>> >>> estimate >>>>> >>>>>> to 1/3 to be applicable) >>>>> >>>>>> but filtering out even this is an awful lot of work (or it might >>>>> >>> involve >>>>> >>>>>> writing a "bot")... >>>>> >>>>>> if it's important enough the contributor could reopen / rebase >>>>> the >>>>> >>>> patch. >>>>> >>>>>> We could try to communicate the non-hostaile intention in the >>>>> message >>>>> >>>>>> placed by the bot. >>>>> >>>>>> The current message is the stale PRs would get is: >>>>> >>>>>> "This pull request has been automatically marked as stale >>>>> because it >>>>> >>> has >>>>> >>>>>> not had recent activity. It will be closed if no further >>>>> activity >>>>> >>>> occurs." >>>>> >>>>>> >>>>> >>>>>>> Also, I have submitted my first PR to test out the new >>>>> system. It >>>>> >>>>>>> has passed tests. Ashutoshc has generously provided a +1. >>>>> What's >>>>> >>> the >>>>> >>>>>>> next step to get it merged into the master? Do I download the >>>>> patch >>>>> >>>> from >>>>> >>>>>>> Github and apply manually using my Apache credentials? Is the >>>>> >>> "merge" >>>>> >>>>>>> feature setup in Github? As I understand it, GitHub is only >>>>> >>> mirroring >>>>> >>>>>> the >>>>> >>>>>>> Apache git system. Whatever the process we need an update in >>>>> the >>>>> >>>>>>> HowToContribute docs. >>>>> >>>>>> >>>>> >>>>>> That's an interesting question; the github repo is linked to the >>>>> >>> apache >>>>> >>>>>> repo - so you may push/merge/whatever on the github interface; >>>>> it >>>>> >>> will >>>>> >>>> work. >>>>> >>>>>> Github supports 3 modes to merge PRs: >>>>> >>>>>> * We should definetly disable the "merge" option as that will >>>>> just >>>>> >>>> create >>>>> >>>>>> a internation railways station from our history :) >>>>> >>>>>> * rebase doesn't make it easier for reviewier to keep track new >>>>> >>>>>> changes...because the PR owner have to continuosly force push >>>>> the >>>>> >>> branch >>>>> >>>>>> * squash merge work great - and I remembered that it changes the >>>>> >>> author >>>>> >>>> to >>>>> >>>>>> the user pushing the "squash" button; however right now it seems >>>>> >>> that it >>>>> >>>>>> changes the author to >>>>> >>>>>> the "user who opened the pr" which looks good-enough for me! >>>>> >>>>>> (I've added the neccessary .asf.yaml changes to the existing PR) >>>>> >>>>>> >>>>> >>>>>> cheers, >>>>> >>>>>> Zoltan >>>>> >>>>>> >>>>> >>>>>>> https://github.com/apache/hive/pull/1045 >>>>> >>>>>>> >>>>> >>>>>> >>>>> >>>> >>>>> >>> >>>>> https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-ApplyingaPatch >>>>> >>>>>>> >>>>> >>>>>>> >>>>> >>>>>>> Thanks! >>>>> >>>>>>> >>>>> >>>>>>> On Tue, Jun 2, 2020 at 4:58 AM Zoltan Haindrich <k...@rxd.hu> >>>>> >>> wrote: >>>>> >>>>>>> >>>>> >>>>>>>> I think to use "probot" we would need to ask infra to >>>>> configure the >>>>> >>>>>>>> "probot" github app. >>>>> >>>>>>>> It seems to me that the stale plugin from github actions >>>>> provides >>>>> >>>> almost >>>>> >>>>>>>> the same functionaluty - as they seem to be more or less >>>>> identical >>>>> >>> in >>>>> >>>>>>>> features I would go with the >>>>> >>>>>>>> latter. >>>>> >>>>>>>> >>>>> >>>>>>>> I've opened a pr to enable stale on the hive repo: >>>>> >>>>>>>> by default it will mark as stale afte 60 days; and close it if >>>>> >>> there >>>>> >>>> is >>>>> >>>>>> no >>>>> >>>>>>>> activity for another 7 days >>>>> >>>>>>>> https://github.com/apache/hive/pull/1049 >>>>> >>>>>>>> >>>>> >>>>>>>> cheers, >>>>> >>>>>>>> Zoltan >>>>> >>>>>>>> >>>>> >>>>>>>> >>>>> >>>>>>>> On 6/2/20 5:00 AM, Ashutosh Chauhan wrote: >>>>> >>>>>>>>> How about using stalebot : https://github.com/probot/stale ? >>>>> >>>>>>>>> >>>>> >>>>>>>>> On Mon, Jun 1, 2020 at 12:20 PM Zoltán Haindrich < >>>>> k...@rxd.hu> >>>>> >>>> wrote: >>>>> >>>>>>>>> >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> Hey David, >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> On June 1, 2020 3:52:05 PM GMT+02:00, David Mollitor < >>>>> >>>>>> dam6...@gmail.com >>>>> >>>>>>>>> >>>>> >>>>>>>>>> wrote: >>>>> >>>>>>>>>>> Any idea how long it will take to run precomit on all >>>>> existing >>>>> >>> PRs? >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>> I'm not entirely sure, but a rough estimate could be: >>>>> >>>>>>>>>> * not every pr is mergeable; there are many which was >>>>> already >>>>> >>>>>>>>>> merged/outdated/etc...lets estimate that 1/3 is mergeable >>>>> >>>>>>>>>> * every pr runs for at least 1 hours >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> this would mean 430/3*1/24 days of test execution which is >>>>> at >>>>> >>> least >>>>> >>>> 6 >>>>> >>>>>>>> days. >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> I see little to no value in running tests on archaic prs. >>>>> >>>>>>>>>> We could also configure an automatism to close prs after >>>>> some >>>>> >>> time >>>>> >>>> of >>>>> >>>>>>>>>> inactivity >>>>> >>>>>>>>>> https://github.com/actions/stale/blob/master/README.md >>>>> >>>>>>>>>> The good side of this is that it will get rid of ancient >>>>> prs; >>>>> >>>> however >>>>> >>>>>> it >>>>> >>>>>>>>>> might seem rude to a contributor in case he is waiting for >>>>> >>> feedback >>>>> >>>> or >>>>> >>>>>>>>>> something.... >>>>> >>>>>>>>>> Even with that argument I think we should configure it at >>>>> least >>>>> >>> for >>>>> >>>> a >>>>> >>>>>>>> few >>>>> >>>>>>>>>> days to get rid of the dangling prs of almost a decade ! >>>>> (pr#2 is >>>>> >>>>>>>> opened in >>>>> >>>>>>>>>> 2011)... >>>>> >>>>>>>>>> What do you think? >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> cheers, >>>>> >>>>>>>>>> Zoltan >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> >>>>> >>>>>>>>>>> On Mon, Jun 1, 2020 at 9:49 AM Panos Garefalakis < >>>>> >>>> panga...@gmail.com >>>>> >>>>>>> >>>>> >>>>>>>>>>> wrote: >>>>> >>>>>>>>>>> >>>>> >>>>>>>>>>>> Same here, however, there are still ~ 430 PRs pending on >>>>> >>> master. >>>>> >>>>>>>>>>>> Thanks Zoltan for this great initiative! >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>>>> Cheers, >>>>> >>>>>>>>>>>> Panagiotis >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>>>> On Mon, Jun 1, 2020 at 2:33 PM David Mollitor < >>>>> >>> dam6...@gmail.com> >>>>> >>>>>>>>>>> wrote: >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>>>>> Thanks so much for the work on this. >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> Just cleaned up mine. >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> On Sat, May 30, 2020 at 10:16 AM Zoltan Haindrich < >>>>> >>> k...@rxd.hu> >>>>> >>>>>>>>>>> wrote: >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> Hey All, >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> The new test executor will pick up any PR which doesn't >>>>> yet >>>>> >>> have >>>>> >>>>>>>>>>> a test >>>>> >>>>>>>>>>>>>> result - now that the patch is on the master; every PR >>>>> which >>>>> >>> is >>>>> >>>>>>>>>>>> mergeable >>>>> >>>>>>>>>>>>>> with the master branch is >>>>> >>>>>>>>>>>>>> a good candidate - so the right move would be to clean >>>>> up >>>>> >>> our PR >>>>> >>>>>>>>>>>> backlog. >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> I would like to ask everyone to look at >>>>> >>>>>>>>>>>>>> https://github.com/apache/hive/pulls >>>>> >>>>>>>>>>>>>> and close some PRs which are already submitted or just >>>>> >>> leftovers >>>>> >>>>>>>>>> >from - >>>>> >>>>>>>>>>>>>> primarily I would ask you to look at PRs opened by >>>>> >>> yourself... >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>>> cheers, >>>>> >>>>>>>>>>>>>> Zoltan >>>>> >>>>>>>>>>>>>> >>>>> >>>>>>>>>>>>> >>>>> >>>>>>>>>>>> >>>>> >>>>>>>>>> >>>>> >>>>>>>>>> -- >>>>> >>>>>>>>>> Zoltán Haindrich >>>>> >>>>>>>>>> >>>>> >>>>>>>>> >>>>> >>>>>>>> >>>>> >>>>>>> >>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>>>> >>> >>>>> >> >>>>> > >>>>> >>>>
