Hello, the ticket has been created: https://issues.apache.org/jira/browse/INFRA-20005
Best regards, Marco On Thu, Mar 19, 2020 at 11:49 PM Marco de Abreu <marco.g.ab...@gmail.com> wrote: > Sounds like a good plan! > > Please send me the URL (please make sure it's backed by DNS and not just > the gateway URL) of the webhook handler, GitHub events you're interested in > and the shared secret in a private email to my personal email address. I > will then create the ticket with Apache infra. > > -Marco > > Chaitanya Bapat <chai.ba...@gmail.com> schrieb am Do., 19. März 2020, > 23:07: > >> @Marco Alright, it makes total sense to test out the Bot feature alongside >> auto-trigger as a transition. >> >> Path Forward: >> 1. Setup MXNet Bot on apache/incubator-mxnet repo (GitHub WebHook and >> Infra) >> 2. We don't turn off automatic trigger of PR builds for now. >> 3. Hopefully, bot is used by developers to trigger specific jobs >> 4. Later on (say around April 20), let's discuss the possibility of >> switching off auto-trigger (with appropriate data) if it makes sense. >> Thanks Marco for volunteering to help enable the web hook on >> apache/incubator-mxnet. Let me know if we can sync up on Slack channel to >> get the ball rolling. >> >> Thanks once again for the entire community to step in and help try out >> this >> Bot. >> Chai >> >> On Wed, 18 Mar 2020 at 17:07, Marco de Abreu <marco.g.ab...@gmail.com> >> wrote: >> >> > Hi, that's correct. But as stated previously, it's not an option to >> remove >> > the hook. For now, I'd like to see how the system behaves while it's >> > optional. Later on, we can talk about revisiting this decision. But to >> me >> > it's not an option to deploy an entirely new system and approach without >> > having a transition or even a timeframe in which we are able to fall >> back. >> > >> > I'm happy to support the deployment of the bot and add an additional >> > webhook to enable it's functionality to support selective triggering by >> PR >> > authors and committers, but I will not support the disabling of >> automatic >> > triggering of branches or PRs. >> > >> > -Marco >> > >> > Chaitanya Bapat <chai.ba...@gmail.com> schrieb am Mi., 18. März 2020, >> > 21:00: >> > >> > > Hey Marco, >> > > >> > > I thought currently every commit on PR and master triggers CI >> > > because >> > > a. github webhook points to Jenkins Server >> > > b. GH Webhook events trigger builds on Jenkins for all commits to any >> > > branch in apache/incubator-mxnet >> > > may it be master/PR/non-PR >> > > Reason: >> > > Because all the 3 types of branches are discovered by Jenkins (non-PR >> > > (including master) and PR) >> > > >> > > Proposal: Remove GitHub WebHook to Jenkins and replace with GH >> Webhook to >> > > Lambda >> > > But after I remove the github webhook that points to Jenkins : *N**o >> > commit >> > > will trigger Jenkins build by default* (as Jenkins wont receive GH >> > events) >> > > Only those that Bot deems fit will be triggered (using Jenkins API >> > invoked >> > > by Lambda). >> > > Hence its needed to handle that case of master merge. >> > > Am I understanding this correctly? >> > > >> > > On Wed, 18 Mar 2020 at 04:23, Marco de Abreu <marco.g.ab...@gmail.com >> > >> > > wrote: >> > > >> > > > Thanks Chai, sounds good to me. Could you elaborate a bit on the >> point >> > > > about triggering a CI run after the PR has been merged? We already >> to >> > > that >> > > > automatically for the master, so what's the benefit to do it twice? >> > > > >> > > > -Marco >> > > > >> > > > Chaitanya Bapat <chai.ba...@gmail.com> schrieb am Mi., 18. März >> 2020, >> > > > 09:30: >> > > > >> > > > > Update: >> > > > > >> > > > > > we can ensure that all CI runs ran on the commit that will be >> > merged >> > > > > @Sam Skalicky <samskali...@gmail.com> Branch Protection is added >> to >> > > > public >> > > > > MXNet repo. It ensures that for every PR to be merged, the CI >> passes. >> > > All >> > > > > the jobs selected "required" jobs will have to be green for the >> PR to >> > > be >> > > > > merged. Ofcourse, users with "Adminstrator" access can merge >> without >> > it >> > > > but >> > > > > that's just a backdoor. It is the case now and will continue to be >> > the >> > > > case >> > > > > with the inclusion of Bot. >> > > > > >> > > > > > easily verify that the CI has executed all runs on the commit >> that >> > > will >> > > > > be merged >> > > > > GitHub UI shows all the jobs and the status corresponding to it on >> > > every >> > > > > commit. That should suffice. For the merged commits, Repo -> >> Commits >> > -> >> > > > > Commit ID (Status) can be tracked currently (only way that I know >> > of). >> > > > > Moreover, it is beyond the scope of this project (and possibly >> out of >> > > our >> > > > > control since this is purely GitHub UI specific use-case). >> > > > > >> > > > > Thanks @przemyslaw for supporting the opt-in. >> > > > > >> > > > > Thanks everyone in the community for sharing concerns, voicing >> your >> > > > opinion >> > > > > and participating in the discussion. >> > > > > Thanks to those who attended the demo last Friday. >> > > > > >> > > > > Action items from that discussion >> > > > > 1. Handle master merge builds [Done] >> > > > > Bot runs entire CI suite after the PR is merged and comments on >> the >> > PR >> > > > > about the same. >> > > > > Design decision : >> > > > > MXNet Bot comment about master merge build on the *merge commit vs >> > PR*. >> > > > > After the PR is merged, Bot runs entire CI and comments the >> result of >> > > CI >> > > > > trigger on the PR (because it is easy to track on a PR rather than >> > > > > commenting inside the merge commit) >> > > > > >> > > > > 2. Idempotent condition >> > > > > In case of already running build, if an attempt is made to >> retrigger >> > > the >> > > > > job then what should be the response >> > > > > a. Not to re-trigger, let the ongoing build continue till >> completion >> > > > > b. End the ongoing build and re-trigger >> > > > > c. Let the ongoing build continue, re-trigger new build >> > > > > >> > > > > From resource saving point of view, *c* looks costly and a can be >> > > > > avoided/optimized by B. >> > > > > In case when a re-trigger was started "erroneously" then killing >> > > ongoing >> > > > > build and re-trigger is a waste. >> > > > > In case when ongoing build failed in one sub-part, then >> re-triggering >> > > is >> > > > > justified. >> > > > > Erroneous re-triggers would be less often while conscious >> re-triggers >> > > to >> > > > > suppress failure is more common use-case. It looks like a safe >> > > assumption >> > > > > to make given the trade-off. >> > > > > [Open to debate] >> > > > > >> > > > > 3. Add security consideration [Use of secret manager, but without >> > > > > auto-rotation due to Jenkins manual config requirement] [Done] >> > > > > 4. New PR Instruction message by the Bot [Done] >> > > > > Thanks to the suggestion of Leonard, supported by others. I've now >> > > added >> > > > > the feature where the Bot comments a help message. [For reference >> - >> > > > > https://github.com/ChaiBapchya/incubator-mxnet/pull/52] >> > > > > >> > > > > Barring the opt-in vs opt-out debate & idempotency, consensus was >> > > quickly >> > > > > reached for the rest. >> > > > > >> > > > > In the coming days, I hope to roll-out this feature into Prod >> (public >> > > > > MXNet) for all devs to use. >> > > > > >> > > > > Thanks, >> > > > > Chai >> > > > > >> > > > > On Mon, 16 Mar 2020 at 11:57, Marco de Abreu < >> > marco.g.ab...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Well that's generally a problem with a deferred CI approach (CI >> is >> > > run >> > > > at >> > > > > > commit and not at merge time). This can either be solved through >> > the >> > > > > other >> > > > > > proposal that's currently on dev@, by having a bot which does >> > merges >> > > > by >> > > > > > having a global lock and a merge queue or by accepting the >> issue. >> > > > Reality >> > > > > > right now is that we're running that model where two PRs which >> are >> > > > merged >> > > > > > in parallel might break one another. One thing to consider >> though >> > is >> > > > that >> > > > > > this breakage would have to be introduced in two separate parts >> > since >> > > > > > otherwise there'd be merge conflicts. I think we had that >> situation >> > > > twice >> > > > > > so far and the result was a quick revert, so I'd say that it's a >> > > > problem >> > > > > > that can happily be accepted. All other solutions basically >> require >> > > > some >> > > > > > form of single-threaded and globally locked solution which >> limits >> > us >> > > in >> > > > > > scalability. I'd recommend to just accept that risk and revert >> a PR >> > > in >> > > > > case >> > > > > > it actually had a conflict. >> > > > > > >> > > > > > -Marco >> > > > > > >> > > > > > On Mon, Mar 16, 2020 at 6:29 PM Skalicky, Sam >> > > > <sska...@amazon.com.invalid >> > > > > > >> > > > > > wrote: >> > > > > > >> > > > > > > We probably need some way to track which CI runs ran for which >> > > commit >> > > > > > too, >> > > > > > > that way we can ensure that all CI runs ran on the commit that >> > will >> > > > be >> > > > > > > merged. Maybe the bot can comment with the commit hash when >> > users >> > > > > > command >> > > > > > > it to do something. Although since users can trigger >> individual >> > CI >> > > > runs >> > > > > > its >> > > > > > > possible to have some commits run some CI runs but not >> others. We >> > > > need >> > > > > > some >> > > > > > > way to easily verify that the CI has executed all runs on the >> > > commit >> > > > > that >> > > > > > > will be merged. >> > > > > > > >> > > > > > > Sam >> > > > > > > >> > > > > > > > On Mar 13, 2020, at 8:28 PM, Przemysław Trędak < >> > > ptre...@apache.org >> > > > > >> > > > > > > wrote: >> > > > > > > > >> > > > > > > > CAUTION: This email originated from outside of the >> > organization. >> > > Do >> > > > > not >> > > > > > > click links or open attachments unless you can confirm the >> sender >> > > and >> > > > > > know >> > > > > > > the content is safe. >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > I personally like the idea of opt-in more than opt-out: >> > > > > > > > - ultimately PR author wants the PR to be merged so they (or >> > > > > committer >> > > > > > > reviewing the PR) will trigger the CI >> > > > > > > > - if it is easy to trigger the PR via the bot command then >> the >> > > > amount >> > > > > > of >> > > > > > > work per PR should be less than with opt-out (since most of >> the >> > > > commits >> > > > > > > should then be marked as [skip ci] or something similar >> > > > > > > > >> > > > > > > > +1 to the bot making a comment on each new PR with its >> commands >> > > > (and >> > > > > > > also explaining, or at least giving links to the general PR >> > process >> > > > so >> > > > > > new >> > > > > > > PR authors are not lost). Maybe we could make the bot >> recognize >> > if >> > > > the >> > > > > PR >> > > > > > > author is new or existing contributor and offer advice based >> on >> > > that? >> > > > > > > > >> > > > > > > > Thanks >> > > > > > > > Przemek >> > > > > > > > >> > > > > > > > On 2020/03/13 22:06:58, Marco de Abreu < >> > marco.g.ab...@gmail.com> >> > > > > > wrote: >> > > > > > > >> Hi, >> > > > > > > >> >> > > > > > > >> since it's no longer necessary to push a new commit to >> trigger >> > > CI, >> > > > > it >> > > > > > > will >> > > > > > > >> already reduce the costs. But to me, requiring an action to >> > > enable >> > > > > CI >> > > > > > > after >> > > > > > > >> a PR has been created initially, is a no go. User can opt >> out >> > of >> > > > CI, >> > > > > > but >> > > > > > > >> the default has to be CI being triggered automatically for >> > every >> > > > > > commit >> > > > > > > >> unless specifically disabled by a participant. I'm also >> fine >> > > with >> > > > > > > >> triggering certain additional jobs (think about running a >> > > nightly >> > > > > job >> > > > > > > upon >> > > > > > > >> request for a PR) to require a manual step, but the PR >> > > validation >> > > > > > > pipelines >> > > > > > > >> have to run automatically. Every check that is marked as >> > > > "Required" >> > > > > in >> > > > > > > >> GitHub has to be automatically kicked off. >> > > > > > > >> >> > > > > > > >> -Marco >> > > > > > > >> >> > > > > > > >> On Fri, Mar 13, 2020 at 9:50 PM Chaitanya Bapat < >> > > > > chai.ba...@gmail.com >> > > > > > > >> > > > > > > >> wrote: >> > > > > > > >> >> > > > > > > >>> Firstly, >> > > > > > > >>> Sorry I missed out on attaching the mail thread that was >> sent >> > > on >> > > > > 12th >> > > > > > > >>> February for notifying the community of the upcoming >> changes >> > to >> > > > the >> > > > > > > MXNet >> > > > > > > >>> CI >> > > > > > > >>> For reference : >> > > > > > > >>> >> > > > > > > >>> >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://lists.apache.org/thread.html/r09a6ab2803a996fc80e00fe39ed312fa4865e8805e08df847f1addad%40%3Cdev.mxnet.apache.org%3E >> > > > > > > >>> >> > > > > > > >>> Now to the questions, >> > > > > > > >>>> Is it possible for re-triggering a single job to be >> abused? >> > > > > > > >>> @Tao In the case when a user re-triggers a single job >> > multiple >> > > > > times, >> > > > > > > that >> > > > > > > >>> will be visible in the PR conversation thread. A >> committer, >> > > even >> > > > > > after >> > > > > > > he >> > > > > > > >>> has approved the PR before, generally takes a look at the >> > final >> > > > > state >> > > > > > > of >> > > > > > > >>> the PR before merging. Would it be fair to assume the >> > committer >> > > > > could >> > > > > > > take >> > > > > > > >>> the multiple re-trigger of a single job into account >> before >> > > > > merging? >> > > > > > > The >> > > > > > > >>> committer then has the option to invoke `@mxnet-bot run ci >> > > [all] >> > > > ` >> > > > > to >> > > > > > > >>> trigger the entire build pipeline one last to counter the >> > > abuse. >> > > > > This >> > > > > > > is >> > > > > > > >>> aligned with what @Leonard said. >> > > > > > > >>> >> > > > > > > >>> @Sandeep Thanks a lot for collecting and sharing valuable >> > data. >> > > > I'd >> > > > > > > concur >> > > > > > > >>> with the opinion that given the existing things >> committers & >> > PR >> > > > > > Authors >> > > > > > > >>> take care of, invoking CI shouldn't be that big of an >> > > additional >> > > > > > > burden. >> > > > > > > >>> >> > > > > > > >>> @Marco With the opt-out, the onus remains on the PR >> Author. >> > It >> > > > > > doesn't >> > > > > > > help >> > > > > > > >>> reduce the resource usage. Hence, it was suggested to >> switch >> > to >> > > > > > > >>> opt-in. @Leo's suggestion for proactive commenting on the >> > part >> > > of >> > > > > bot >> > > > > > > makes >> > > > > > > >>> sense and is doable. >> > > > > > > >>> >> > > > > > > >>> Default : opt-out and User initiated opt-in (with >> addressing >> > > > Leo's >> > > > > > fix >> > > > > > > for >> > > > > > > >>> the usability issue you correctly pointed out ) >> > > > > > > >>> @Marco How does this sound to you? >> > > > > > > >>> >> > > > > > > >>> Again, thank you all for chiming in and voicing your >> opinion. >> > > > > > > Appreciate >> > > > > > > >>> it. >> > > > > > > >>> We can take ahead these discussions in today's demo >> meeting. >> > > > > [Design >> > > > > > > Doc >> > > > > > > >>> < >> > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+CI+Bot >> > > > >] >> > > > > > > [Demo >> > > > > > > >>> Video <https://www.youtube.com/watch?v=gfOGwZId8aU>] >> > > > > > > >>> >> > > > > > > >>> Thanks, >> > > > > > > >>> Chai >> > > > > > > >>> >> > > > > > > >>> On Fri, 13 Mar 2020 at 12:34, Marco de Abreu < >> > > > > > marco.g.ab...@gmail.com> >> > > > > > > >>> wrote: >> > > > > > > >>> >> > > > > > > >>>> I'd recommend that the bot makes an initial comment when >> a >> > PR >> > > > gets >> > > > > > > opened >> > > > > > > >>>> and informs the users of its commands. It then tells the >> > user >> > > > the >> > > > > > > commend >> > > > > > > >>>> to opt out of CI. >> > > > > > > >>>> >> > > > > > > >>>> -Marco >> > > > > > > >>>> >> > > > > > > >>>> Lausen, Leonard <lau...@amazon.com.invalid> schrieb am >> Fr., >> > > 13. >> > > > > > März >> > > > > > > >>> 2020, >> > > > > > > >>>> 20:27: >> > > > > > > >>>> >> > > > > > > >>>>> On opt-out: People may be unaware of opt-out would not >> use >> > > it. >> > > > > > There >> > > > > > > is >> > > > > > > >>>> no >> > > > > > > >>>>> incentive to use opt-out, as the PR author doesn't pay >> any >> > > > money >> > > > > > for >> > > > > > > CI >> > > > > > > >>>>> run. >> > > > > > > >>>>> >> > > > > > > >>>>> I agree with Marco though that opt-in alone may cause >> > > usability >> > > > > > > issues, >> > > > > > > >>>> as >> > > > > > > >>>>> contributors may not be aware of how to trigger the CI. >> > > > > > > >>>>> One solution is that the bot proactively comments on >> the PR >> > > and >> > > > > > > reminds >> > > > > > > >>>> the >> > > > > > > >>>>> author to trigger running CI once the author deems the >> PR >> > > > ready. >> > > > > > > >>>>> >> > > > > > > >>>>> But even if we choose opt-out, the bot will still add a >> lot >> > > of >> > > > > > value, >> > > > > > > >>> as >> > > > > > > >>>> PR >> > > > > > > >>>>> authors can retrigger single jobs that have failed due >> to >> > > > > > flakiness. >> > > > > > > >>>>> >> > > > > > > >>>>>> Is it possible for re-triggering a single job to be >> > abused? >> > > > For >> > > > > > > >>>> example, >> > > > > > > >>>>>> the author spends two days re-triggering a flaky job to >> > make >> > > > it >> > > > > > > pass. >> > > > > > > >>>>> >> > > > > > > >>>>> Yes, this is possible. I suggest the committer who >> likes to >> > > > > merge a >> > > > > > > PR >> > > > > > > >>>>> needs to >> > > > > > > >>>>> make a good judgement here if a PR is abusing the >> feature, >> > > and >> > > > if >> > > > > > so, >> > > > > > > >>>>> retrigger >> > > > > > > >>>>> all CI runs. >> > > > > > > >>>>> >> > > > > > > >>>>> Best regards >> > > > > > > >>>>> Leonard >> > > > > > > >>>>> >> > > > > > > >>>>> On Fri, 2020-03-13 at 08:07 +0100, Marco de Abreu wrote: >> > > > > > > >>>>>> Thanks for the data Sandeep. In these cases it sounds >> like >> > > it >> > > > > > would >> > > > > > > >>>> have >> > > > > > > >>>>>> rather been better when people explicitly turned off >> CI in >> > > > that >> > > > > > > case. >> > > > > > > >>>>>> What's the argument against an opt-out instead of an >> > opt-in? >> > > > > > > >>>>>> >> > > > > > > >>>>>> My intention is that I consider it quite cumbersome to >> > make >> > > > it a >> > > > > > > >>>>> *required* >> > > > > > > >>>>>> step to always trigger CI manually, even if just >> > submitting >> > > a >> > > > > > small >> > > > > > > >>> PR. >> > > > > > > >>>>> I'd >> > > > > > > >>>>>> rather see people explicitly turning off CI if they >> > wouldn't >> > > > > like >> > > > > > to >> > > > > > > >>>> use >> > > > > > > >>>>> it >> > > > > > > >>>>>> - and there's also the "draft" stage for a PR which >> some >> > > > > > > contributors >> > > > > > > >>>> are >> > > > > > > >>>>>> using. >> > > > > > > >>>>>> >> > > > > > > >>>>>> With regards to WIP and do not review: I think these >> are >> > use >> > > > > cases >> > > > > > > >>>> where >> > > > > > > >>>>>> you want CI feedback, as otherwise you wouldn't have >> > opened >> > > > the >> > > > > > PR. >> > > > > > > >>> If >> > > > > > > >>>>> you >> > > > > > > >>>>>> don't want human feedback and neither machine feedback, >> > why >> > > > open >> > > > > > the >> > > > > > > >>> PR >> > > > > > > >>>>> at >> > > > > > > >>>>>> all? >> > > > > > > >>>>>> >> > > > > > > >>>>>> -Marco >> > > > > > > >>>>>> >> > > > > > > >>>>>> >> > > > > > > >>>>>> sandeep krishnamurthy <sandeep.krishn...@gmail.com> >> > schrieb >> > > > am >> > > > > > Fr., >> > > > > > > >>>> 13. >> > > > > > > >>>>>> März 2020, 05:24: >> > > > > > > >>>>>> >> > > > > > > >>>>>>> I tried to gather some data for us to discuss this >> topic >> > in >> > > > > this >> > > > > > > >>>>> thread. I >> > > > > > > >>>>>>> tried to count number of un-necessary builds by >> looking >> > at >> > > > most >> > > > > > > >>>> recent >> > > > > > > >>>>> (as >> > > > > > > >>>>>>> of 12, March 9 PM PST) 50 PRs merged to master and 50 >> > PRs. >> > > > > > > >>>>>>> Identifying un-necessary builds is bit subjective. I >> > tried >> > > to >> > > > > be >> > > > > > > >>> more >> > > > > > > >>>>>>> conservative where I didn't count a build as >> un-necessary >> > > if >> > > > I >> > > > > > was >> > > > > > > >>> in >> > > > > > > >>>>>>> doubt. Hence, I was not able to automate, but I made >> an >> > > > effort >> > > > > to >> > > > > > > >>> go >> > > > > > > >>>>>>> through PRs manually and use below criteria to >> identify >> > > > > > > >>> un-necessary >> > > > > > > >>>>>>> commits triggering the builds. >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> 1. Explicitly marked as WIP / do not review PR >> > > > > > > >>>>>>> 2. Incremental WIP commit and finally commenting a >> > commit >> > > > > > > >>> “trigger >> > > > > > > >>>>> CI” >> > > > > > > >>>>>>> 3. Multiple commits to address all comments from >> single >> > > > > review. >> > > > > > > >>>>> This is >> > > > > > > >>>>>>> assuming we see a comment, address them, commit, >> next >> > the >> > > > > > > >>>> following >> > > > > > > >>>>>>> comment >> > > > > > > >>>>>>> 4. Sequence of documentation only changes >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> I found there were around 42 avoidable builds from >> most >> > > > recent >> > > > > 50 >> > > > > > > >>>>> merged >> > > > > > > >>>>>>> PRs and around 86 builds from recent 50 open PRs. >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> I synced up with other contributors (Joe Evans, Chai) >> > from >> > > > > Amazon >> > > > > > > >>> who >> > > > > > > >>>>> is >> > > > > > > >>>>>>> contributing to MXNet CI system. I was told that on an >> > > > average >> > > > > it >> > > > > > > >>>> costs >> > > > > > > >>>>>>> around $84 per build and on an average 6 commits per >> > merged >> > > > PR >> > > > > > (for >> > > > > > > >>>>> year >> > > > > > > >>>>>>> 2019). Going by that, it is approximately 1/6 builds >> are >> > > > > > avoidable. >> > > > > > > >>>>> [100 / >> > > > > > > >>>>>>> 300 + 300 ] >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> Usability should be top most priority. But, since >> either >> > a >> > > > > > reviewer >> > > > > > > >>>> or >> > > > > > > >>>>> pr >> > > > > > > >>>>>>> author can trigger the bot, is it really a hurdle for >> pr >> > > > author >> > > > > > or >> > > > > > > >>>>> reviewer >> > > > > > > >>>>>>> to call a bot to trigger CI? Given that PR author and >> > > > reviewer >> > > > > is >> > > > > > > >>>>> already >> > > > > > > >>>>>>> actively commenting various details such as - PR >> > > description, >> > > > > > > >>> review >> > > > > > > >>>>>>> comments and responses, adding labels etc. >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> Me too curious to know the behavior for Tao's above >> use >> > > case. >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> Best, >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> Sandeep >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> On Thu, Mar 12, 2020 at 7:18 PM Tao Lv < >> > mutou...@gmail.com >> > > > >> > > > > > wrote: >> > > > > > > >>>>>>> >> > > > > > > >>>>>>>> Is it possible for re-triggering a single job to be >> > > abused? >> > > > > For >> > > > > > > >>>>> example, >> > > > > > > >>>>>>>> the author spends two days re-triggering a flaky job >> to >> > > make >> > > > > it >> > > > > > > >>>>> pass. But >> > > > > > > >>>>>>>> other jobs which have passed the validation may be >> > broken >> > > by >> > > > > > > >>> other >> > > > > > > >>>>>>> commits >> > > > > > > >>>>>>>> during the two day without being noticed. And finally >> > the >> > > PR >> > > > > is >> > > > > > > >>>>> merged >> > > > > > > >>>>>>> with >> > > > > > > >>>>>>>> underlying problems. >> > > > > > > >>>>>>>> >> > > > > > > >>>>>>>> On Fri, Mar 13, 2020 at 6:19 AM Marco de Abreu < >> > > > > > > >>>>> marco.g.ab...@gmail.com> >> > > > > > > >>>>>>>> wrote: >> > > > > > > >>>>>>>> >> > > > > > > >>>>>>>>> In the end it only comes down to money, considering >> > that >> > > > the >> > > > > > > >>>>> system is >> > > > > > > >>>>>>>> auto >> > > > > > > >>>>>>>>> scaling, making the execution time constant. >> > > > > > > >>>>>>>>> >> > > > > > > >>>>>>>>> If we're trading money for usability, I certainly >> would >> > > > > prefer >> > > > > > > >>>>>>> usability. >> > > > > > > >>>>>>>>> I'd rather recommend to spend time on parallelizing >> > test >> > > > > > > >>>> execution >> > > > > > > >>>>> or >> > > > > > > >>>>>>>>> getting rid of integration tests in the PR stage >> > instead >> > > > > > > >>> reducing >> > > > > > > >>>>> the >> > > > > > > >>>>>>>> costs >> > > > > > > >>>>>>>>> by making people not use it. But taking a step back >> to >> > > > > > > >>> requiring >> > > > > > > >>>>> people >> > > > > > > >>>>>>>> to >> > > > > > > >>>>>>>>> manually trigger CI again doesn't feel right. >> > > > > > > >>>>>>>>> >> > > > > > > >>>>>>>>> I'm happy to see that bot deployed, but I do not >> agree >> > > with >> > > > > > > >>>>> removing >> > > > > > > >>>>>>> the >> > > > > > > >>>>>>>>> auto trigger functionality for new commits. >> > > > > > > >>>>>>>>> >> > > > > > > >>>>>>>>> -Marco >> > > > > > > >>>>>>>>> >> > > > > > > >>>>>>>>> Chaitanya Bapat <chai.ba...@gmail.com> schrieb am >> Do., >> > > 12. >> > > > > > > >>> März >> > > > > > > >>>>> 2020, >> > > > > > > >>>>>>>>> 22:47: >> > > > > > > >>>>>>>>> >> > > > > > > >>>>>>>>>> @Marco Thanks for pointing that out. >> > > > > > > >>>>>>>>>> Tomorrow i.e. Friday, March 13, 2020 at 3:00 PM - >> 3:30 >> > > PM >> > > > in >> > > > > > > >>>>>>>> (UTC-08:00) >> > > > > > > >>>>>>>>>> Pacific Time (US & Canada). >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>>> When do we expect this bot to be deployed? >> > > > > > > >>>>>>>>>> @Lin If all goes well in the next week I can >> deploy it >> > > to >> > > > > > > >>>> public >> > > > > > > >>>>>>> Apache >> > > > > > > >>>>>>>>>> (provided I get permissions from Apache Infra) >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>> @Marco Thanks for your feedback. >> > > > > > > >>>>>>>>>>> CI system has to support the community without >> > > requiring >> > > > > > > >>>>> people to >> > > > > > > >>>>>>>>>> constantly shepherd every single run >> > > > > > > >>>>>>>>>> We have data for the number of times CI was >> triggered >> > > > > > > >>>>> unnecessarily >> > > > > > > >>>>>>>> which >> > > > > > > >>>>>>>>>> includes >> > > > > > > >>>>>>>>>> - Entire build triggered instead of specific build >> > > > > > > >>>>>>>>>> - CI triggered when PR is still work in progress or >> > not >> > > > yet >> > > > > > > >>>> ready >> > > > > > > >>>>>>> (say >> > > > > > > >>>>>>>> - >> > > > > > > >>>>>>>>>> intermediate commits) >> > > > > > > >>>>>>>>>> At the end its a trade-off >> > > > > > > >>>>>>>>>> Money, Resources, Time to build for each and every >> > > commit >> > > > vs >> > > > > > > >>>>> Pain of >> > > > > > > >>>>>>>>>> triggering builds >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>>> Scan trigger plugin would poll SCM. Can we use >> plugin >> > > at >> > > > > > > >>>>> scale? >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>> 1. I haven't tested it on scale. But I think with >> the >> > > > > current >> > > > > > > >>>>> scale >> > > > > > > >>>>>>> of >> > > > > > > >>>>>>>>>> MXNet repo (191 open PRs i.e. checking for changes >> to >> > > 191 >> > > > > > > >>>>> branches - >> > > > > > > >>>>>>> It >> > > > > > > >>>>>>>>>> should be manageable) >> > > > > > > >>>>>>>>>> 2. What's the purpose of the plugin? tldr; Branch >> > > > discovery >> > > > > > > >>> or >> > > > > > > >>>>> branch >> > > > > > > >>>>>>>>>> indexing. >> > > > > > > >>>>>>>>>> Scan trigger plugin comes into the picture only >> once >> > per >> > > > PR >> > > > > > > >>> per >> > > > > > > >>>>> job >> > > > > > > >>>>>>>>> (i.e. 8 >> > > > > > > >>>>>>>>>> times per PR for 8 jobs). It is basically done >> when a >> > > new >> > > > PR >> > > > > > > >>> is >> > > > > > > >>>>> made >> > > > > > > >>>>>>>> and >> > > > > > > >>>>>>>>>> the job (say unix-cpu hasn't discovered the new PR >> > > branch >> > > > > > > >>> yet). >> > > > > > > >>>>>>> That's >> > > > > > > >>>>>>>>> it. >> > > > > > > >>>>>>>>>> So it shouldn't be a problem for public MXNet repo. >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>> Thanks, >> > > > > > > >>>>>>>>>> Chai >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>> On Thu, 12 Mar 2020 at 14:22, Marco de Abreu < >> > > > > > > >>>>>>> marco.g.ab...@gmail.com> >> > > > > > > >>>>>>>>>> wrote: >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>>> Btw you forgot to set a date and time for the >> metting >> > > > > > > >>>>>>>>>>> >> > > > > > > >>>>>>>>>>> >> > > > > > > >>>>>>>>>>> On Thu, Mar 12, 2020 at 10:18 PM Marco de Abreu < >> > > > > > > >>>>>>>>> marco.g.ab...@gmail.com >> > > > > > > >>>>>>>>>>> wrote: >> > > > > > > >>>>>>>>>>> >> > > > > > > >>>>>>>>>>>> Thanks Chai, I generally like the idea of the >> bot. >> > But >> > > > > > > >>> I'm >> > > > > > > >>>>> not a >> > > > > > > >>>>>>>>>>> supporter >> > > > > > > >>>>>>>>>>>> of the idea to disable any automatic triggering >> > > > > > > >>> (disabling >> > > > > > > >>>>> the >> > > > > > > >>>>>>>>> webhook >> > > > > > > >>>>>>>>>> is >> > > > > > > >>>>>>>>>>>> also not an option, considering that this will >> > disable >> > > > > > > >>>> master >> > > > > > > >>>>>>>>>> triggers). >> > > > > > > >>>>>>>>>>>> The CI system has to support the community >> without >> > > > > > > >>>> requiring >> > > > > > > >>>>>>> people >> > > > > > > >>>>>>>>> to >> > > > > > > >>>>>>>>>>>> constantly shepherd every single run. Disabling >> > > > automatic >> > > > > > > >>>>>>>> triggering >> > > > > > > >>>>>>>>>>> seems >> > > > > > > >>>>>>>>>>>> like a step back to me. >> > > > > > > >>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>> Instead, I'd recommend that CI gets triggered >> upon >> > > every >> > > > > > > >>>>> commit >> > > > > > > >>>>>>> as >> > > > > > > >>>>>>>>>> usual, >> > > > > > > >>>>>>>>>>>> but people have the possibility to call a >> "command" >> > > > (i.e. >> > > > > > > >>>>> make a >> > > > > > > >>>>>>>>>> message >> > > > > > > >>>>>>>>>>>> which results in the bot setting a label) to >> disable >> > > CI >> > > > > > > >>>> until >> > > > > > > >>>>>>> they >> > > > > > > >>>>>>>>>> revoke >> > > > > > > >>>>>>>>>>>> it. But the standard should still be that a new >> > commit >> > > > > > > >>>>> triggers a >> > > > > > > >>>>>>>> new >> > > > > > > >>>>>>>>>> CI >> > > > > > > >>>>>>>>>>>> run. >> > > > > > > >>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>> >> > > > > > > >>>> >> > https://plugins.jenkins.io/multibranch-scan-webhook-trigger/ >> > > > > > > >>>>>>>> seems >> > > > > > > >>>>>>>>>> like >> > > > > > > >>>>>>>>>>>> this would poll SCM. This will incur high quota >> > > > > > > >>>>> restrictions. Are >> > > > > > > >>>>>>>> you >> > > > > > > >>>>>>>>>>> sure >> > > > > > > >>>>>>>>>>>> that you can use that plugin at scale? >> > > > > > > >>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>> -Marco >> > > > > > > >>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:04 PM Lin Yuan < >> > > > > > > >>>>> apefor...@gmail.com> >> > > > > > > >>>>>>>>> wrote: >> > > > > > > >>>>>>>>>>>>> Chai, >> > > > > > > >>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>> Awesome work. When do we expect this bot to be >> > > > > > > >>> deployed? >> > > > > > > >>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>> Best, >> > > > > > > >>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>> Lin >> > > > > > > >>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>> On Thu, Mar 12, 2020 at 2:00 PM Chaitanya Bapat >> < >> > > > > > > >>>>>>>>> chai.ba...@gmail.com >> > > > > > > >>>>>>>>>>>>> wrote: >> > > > > > > >>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>>> Hello MXNet community, >> > > > > > > >>>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>>> I have built an MXNet Bot < >> > > > > > > >>>> https://github.com/mxnet-bot> >> > > > > > > >>>>> that >> > > > > > > >>>>>>>>>> allows >> > > > > > > >>>>>>>>>>> PR >> > > > > > > >>>>>>>>>>>>>> Authors, Committers and Jenkins Admins to >> trigger >> > CI >> > > > > > > >>>>> manually. >> > > > > > > >>>>>>>>>>>>>> It handles 2 problems >> > > > > > > >>>>>>>>>>>>>> 1. Manual CI trigger instead of existing >> automated >> > > CI >> > > > > > > >>>>> trigger >> > > > > > > >>>>>>>>>>>>>> 2. Gives permissions to PR Authors (in >> addition to >> > > > > > > >>>> MXNet >> > > > > > > >>>>>>>>> Committers >> > > > > > > >>>>>>>>>>> and >> > > > > > > >>>>>>>>>>>>>> Jenkins Admins) >> > > > > > > >>>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>>> Design Doc : >> > > > > > > >>>>>>>>>>>>>> >> > > > > > > >>>>>>> >> > > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+CI+Bot >> > > > > > > >>>>>>>>>>>>>> I urge you all to attend the demonstration >> meeting >> > > > > > > >>> and >> > > > > > > >>>>> lend >> > > > > > > >>>>>>> your >> > > > > > > >>>>>>>>>> views >> > > > > > > >>>>>>>>>>>>> on >> > > > > > > >>>>>>>>>>>>>> the same. >> > > > > > > >>>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>>> Thank you, >> > > > > > > >>>>>>>>>>>>>> Chai >> > > > > > > >>>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>>> *Meeting Details*: >> > > > > > > >>>>>>>>>>>>>> ==============Conference Bridge >> > > > > > > >>>> Information============== >> > > > > > > >>>>>>>>>>>>>> You have been invited to an online meeting, >> > powered >> > > > > > > >>> by >> > > > > > > >>>>> Amazon >> > > > > > > >>>>>>>>> Chime. >> > > > > > > >>>>>>>>>>>>>> *Chime meeting ID*: *9272158344* >> > > > > > > >>>>>>>>>>>>>> Join via Chime clients (manually): Select >> > 'Meetings >> > > > >> > > > > > > >>>>> Join a >> > > > > > > >>>>>>>>>> Meeting', >> > > > > > > >>>>>>>>>>>>> and >> > > > > > > >>>>>>>>>>>>>> enter 9272158344 >> > > > > > > >>>>>>>>>>>>>> Join via Chime clients (auto-call): If you >> invite >> > > > > > > >>>>> auto-call as >> > > > > > > >>>>>>>>>>> attendee, >> > > > > > > >>>>>>>>>>>>>> Chime will call you when the meeting starts, >> > select >> > > > > > > >>>>> 'Answer' >> > > > > > > >>>>>>>>>>>>>> *Join via browser screen share*: >> > > > > > > >>>>> https://chime.aws/9272158344 >> > > > > > > >>>>>>>>>>>>>> *Join via phone* (US): >> > +1-929-432-4463,,,9272158344# >> > > > > > > >>>>>>>>>>>>>> *Join via phone (US toll-free)*: >> > > > > > > >>>>> +1-855-552-4463,,,9272158344# >> > > > > > > >>>>>>>>>>>>>> International dial-in: >> > > > > > > >>>> https://chime.aws/dialinnumbers/ >> > > > > > > >>>>>>>>>>>>>> In-room video system: Ext: 62000, Meeting PIN: >> > > > > > > >>>>> 9272158344# >> > > > > > > >>>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>>> -- >> > > > > > > >>>>>>>>>>>>>> *Chaitanya Prakash Bapat* >> > > > > > > >>>>>>>>>>>>>> *+1 (973) 953-6299* >> > > > > > > >>>>>>>>>>>>>> >> > > > > > > >>>>>>>>>>>>>> [image: >> https://www.linkedin.com//in/chaibapat25] >> > > > > > > >>>>>>>>>>>>>> <https://github.com/ChaiBapchya>[image: >> > > > > > > >>>>>>>>>>>>> https://www.facebook.com/chaibapat >> > > > > > > >>>>>>>>>>>>>> ] >> > > > > > > >>>>>>>>>>>>>> <https://www.facebook.com/chaibapchya>[image: >> > > > > > > >>>>>>>>>>>>>> https://twitter.com/ChaiBapchya] < >> > > > > > > >>>>>>>> https://twitter.com/ChaiBapchya >> > > > > > > >>>>>>>>>>>>>> [image: >> > > > > > > >>>>>>>>>>>>>> https://www.linkedin.com//in/chaibapat25] >> > > > > > > >>>>>>>>>>>>>> <https://www.linkedin.com//in/chaibapchya/> >> > > > > > > >>>>>>>>>>>>>> >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>> -- >> > > > > > > >>>>>>>>>> *Chaitanya Prakash Bapat* >> > > > > > > >>>>>>>>>> *+1 (973) 953-6299* >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>>>>> [image: https://www.linkedin.com//in/chaibapat25] >> > > > > > > >>>>>>>>>> <https://github.com/ChaiBapchya>[image: >> > > > > > > >>>>>>>>> https://www.facebook.com/chaibapat >> > > > > > > >>>>>>>>>> ] >> > > > > > > >>>>>>>>>> <https://www.facebook.com/chaibapchya>[image: >> > > > > > > >>>>>>>>>> https://twitter.com/ChaiBapchya] < >> > > > > > > >>>>> https://twitter.com/ChaiBapchya >> > > > > > > >>>>>>>>>> [image: >> > > > > > > >>>>>>>>>> https://www.linkedin.com//in/chaibapat25] >> > > > > > > >>>>>>>>>> <https://www.linkedin.com//in/chaibapchya/> >> > > > > > > >>>>>>>>>> >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> -- >> > > > > > > >>>>>>> Sandeep Krishnamurthy >> > > > > > > >>>>>>> >> > > > > > > >>>>> >> > > > > > > >>>> >> > > > > > > >>> >> > > > > > > >>> >> > > > > > > >>> -- >> > > > > > > >>> *Chaitanya Prakash Bapat* >> > > > > > > >>> *+1 (973) 953-6299* >> > > > > > > >>> >> > > > > > > >>> [image: https://www.linkedin.com//in/chaibapat25] >> > > > > > > >>> <https://github.com/ChaiBapchya>[image: >> > > > > > > https://www.facebook.com/chaibapat >> > > > > > > >>> ] >> > > > > > > >>> <https://www.facebook.com/chaibapchya>[image: >> > > > > > > >>> https://twitter.com/ChaiBapchya] < >> > > > https://twitter.com/ChaiBapchya >> > > > > > > >[image: >> > > > > > > >>> https://www.linkedin.com//in/chaibapat25] >> > > > > > > >>> <https://www.linkedin.com//in/chaibapchya/> >> > > > > > > >>> >> > > > > > > >> >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > *Chaitanya Prakash Bapat* >> > > > > *+1 (973) 953-6299* >> > > > > >> > > > > [image: https://www.linkedin.com//in/chaibapat25] >> > > > > <https://github.com/ChaiBapchya>[image: >> > > > https://www.facebook.com/chaibapat >> > > > > ] >> > > > > <https://www.facebook.com/chaibapchya>[image: >> > > > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya >> > > > >[image: >> > > > > https://www.linkedin.com//in/chaibapat25] >> > > > > <https://www.linkedin.com//in/chaibapchya/> >> > > > > >> > > > >> > > >> > > >> > > -- >> > > *Chaitanya Prakash Bapat* >> > > *+1 (973) 953-6299* >> > > >> > > [image: https://www.linkedin.com//in/chaibapat25] >> > > <https://github.com/ChaiBapchya>[image: >> > https://www.facebook.com/chaibapat >> > > ] >> > > <https://www.facebook.com/chaibapchya>[image: >> > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya >> > >[image: >> > > https://www.linkedin.com//in/chaibapat25] >> > > <https://www.linkedin.com//in/chaibapchya/> >> > > >> > >> >> >> -- >> *Chaitanya Prakash Bapat* >> *+1 (973) 953-6299* >> >> [image: https://www.linkedin.com//in/chaibapat25] >> <https://github.com/ChaiBapchya>[image: >> https://www.facebook.com/chaibapat] >> <https://www.facebook.com/chaibapchya>[image: >> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image: >> https://www.linkedin.com//in/chaibapat25] >> <https://www.linkedin.com//in/chaibapchya/> >> >
