Yes Denisa, you can use it on existing PRs as well. Sorry for miswording. Bot will comment instructions on how-to-use for every new PR. You can invoke bot for all the PRs.
On Tue, 24 Mar 2020 at 18:00, Denisa Roberts <[email protected]> wrote: > Hi- Only for new PRs? Can I use it for an existing PR to retrigger only > specific tests? > > On Tue, Mar 24, 2020 at 8:27 PM Chaitanya Bapat <[email protected]> > wrote: > >> Hello MXNet community, >> >> Update: Bot has been deployed 🚀 on apache/incubator-mxnet. >> >> For every new PR, bot will comment with a help message (instructing what >> command to comment) >> It can trigger all jobs or specific jobs for users. >> >> Do use and if you find issues/suggestions do comment on this mail thread >> or >> the GitHub issue : https://github.com/apache/incubator-mxnet/issues/17801 >> >> Thanks to Denis, Sandeep, Joe, Pedro, Marco and the design doc reviewers >> for valuable feedback and guidance. >> >> Thank you, >> Chai >> >> On Mon, 23 Mar 2020 at 13:08, sandeep krishnamurthy < >> [email protected]> wrote: >> >> > Thank you Chaitanya and Marco for helping the MXNet community. >> > >> > On Mon, Mar 23, 2020 at 12:56 PM Marco de Abreu < >> [email protected]> >> > wrote: >> > >> > > Sure, already done. >> > > >> > > -Marco >> > > >> > > On Mon, Mar 23, 2020 at 8:53 PM Chaitanya Bapat <[email protected] >> > >> > > wrote: >> > > >> > > > Hello, >> > > > Update: Apache Infra Ticket for MXNet Bot >> > > > Thanks once again, Marco for opening the ticket. But turns out, >> Apache >> > > > Infra folks closed it stating: "Security concerns around allowing >> > unknown >> > > > person to submit PR and run our hardware". Furthermore, it goes onto >> > > state >> > > > that bot circumvents the dependence on Jenkins Admins which is like >> > > solving >> > > > a problem that doesn't exist. >> > > > >> > > > I sense there is some confusion in the communication (maybe on my >> > part). >> > > It >> > > > turns out the security concerns aren't actually correct. >> > > > >> > > > 1. Unknown person can submit a PR (before & after bot proposal), and >> > run >> > > > our hardware (pre as well as post bot). >> > > > 2. Code should be reviewed by somebody with an ICLA on file. This >> > doesn't >> > > > change either. Prior to merging a PR, code has to be approved by a >> > > > committer just like before. >> > > > Overall it looks like the job of the bot isn't clear to folks in >> Apache >> > > > Infra. Bot simply is a means for triggering CI (which could be done >> > > > manually by Log In to Jenkins -> PR -> Job -> Build) and doesn't >> quite >> > > > tweak with merging procedure. Yes, only addition is now unknown >> person >> > > (PR >> > > > Author) can trigger CI with a message (but that was possible anyway >> by >> > > > pushing a commit. Bot just prevents users from pushing empty commits >> > and >> > > > building entire suite). >> > > > >> > > > As can be seen from last 10 open PRs as of Monday 23rd March, 12pm >> PT >> > > most >> > > > PRs fail on 1/2 jobs. In such a scenario, the proposed MXNet bot >> would >> > > come >> > > > in handy for just invoking CI on that specific build (instead of a >> > > > non-committer PR Author to push empty commit : hurting on the >> resource, >> > > > time & cost considerations apart from undesirable dev experience) >> > > > >> > > > @Marco Since I am a non-committer, I guess these 2 clarifications >> need >> > to >> > > > be conveyed to the Apache Infra by someone with Committer access. >> > > > >> > > > What do you think? >> > > > >> > > > Thanks, >> > > > Chai >> > > > >> > > > On Sat, 21 Mar 2020 at 16:08, Marco de Abreu < >> [email protected]> >> > > > wrote: >> > > > >> > > > > Hello, >> > > > > >> > > > > the ticket has been created: >> > > > > https://issues.apache.org/jira/browse/INFRA-20005 >> > > > > >> > > > > Best regards, >> > > > > Marco >> > > > > >> > > > > On Thu, Mar 19, 2020 at 11:49 PM Marco de Abreu < >> > > [email protected] >> > > > > >> > > > > wrote: >> > > > > >> > > > > > Sounds like a good plan! >> > > > > > >> > > > > > Please send me the URL (please make sure it's backed by DNS and >> not >> > > > just >> > > > > > the gateway URL) of the webhook handler, GitHub events you're >> > > > interested >> > > > > in >> > > > > > and the shared secret in a private email to my personal email >> > > address. >> > > > I >> > > > > > will then create the ticket with Apache infra. >> > > > > > >> > > > > > -Marco >> > > > > > >> > > > > > Chaitanya Bapat <[email protected]> schrieb am Do., 19. März >> > > 2020, >> > > > > > 23:07: >> > > > > > >> > > > > >> @Marco Alright, it makes total sense to test out the Bot >> feature >> > > > > alongside >> > > > > >> auto-trigger as a transition. >> > > > > >> >> > > > > >> Path Forward: >> > > > > >> 1. Setup MXNet Bot on apache/incubator-mxnet repo (GitHub >> WebHook >> > > and >> > > > > >> Infra) >> > > > > >> 2. We don't turn off automatic trigger of PR builds for now. >> > > > > >> 3. Hopefully, bot is used by developers to trigger specific >> jobs >> > > > > >> 4. Later on (say around April 20), let's discuss the >> possibility >> > of >> > > > > >> switching off auto-trigger (with appropriate data) if it makes >> > > sense. >> > > > > >> Thanks Marco for volunteering to help enable the web hook on >> > > > > >> apache/incubator-mxnet. Let me know if we can sync up on Slack >> > > channel >> > > > > to >> > > > > >> get the ball rolling. >> > > > > >> >> > > > > >> Thanks once again for the entire community to step in and help >> try >> > > out >> > > > > >> this >> > > > > >> Bot. >> > > > > >> Chai >> > > > > >> >> > > > > >> On Wed, 18 Mar 2020 at 17:07, Marco de Abreu < >> > > [email protected] >> > > > > >> > > > > >> wrote: >> > > > > >> >> > > > > >> > Hi, that's correct. But as stated previously, it's not an >> option >> > > to >> > > > > >> remove >> > > > > >> > the hook. For now, I'd like to see how the system behaves >> while >> > > it's >> > > > > >> > optional. Later on, we can talk about revisiting this >> decision. >> > > But >> > > > to >> > > > > >> me >> > > > > >> > it's not an option to deploy an entirely new system and >> approach >> > > > > without >> > > > > >> > having a transition or even a timeframe in which we are able >> to >> > > fall >> > > > > >> back. >> > > > > >> > >> > > > > >> > I'm happy to support the deployment of the bot and add an >> > > additional >> > > > > >> > webhook to enable it's functionality to support selective >> > > triggering >> > > > > by >> > > > > >> PR >> > > > > >> > authors and committers, but I will not support the disabling >> of >> > > > > >> automatic >> > > > > >> > triggering of branches or PRs. >> > > > > >> > >> > > > > >> > -Marco >> > > > > >> > >> > > > > >> > Chaitanya Bapat <[email protected]> schrieb am Mi., 18. >> März >> > > > 2020, >> > > > > >> > 21:00: >> > > > > >> > >> > > > > >> > > Hey Marco, >> > > > > >> > > >> > > > > >> > > I thought currently every commit on PR and master triggers >> CI >> > > > > >> > > because >> > > > > >> > > a. github webhook points to Jenkins Server >> > > > > >> > > b. GH Webhook events trigger builds on Jenkins for all >> commits >> > > to >> > > > > any >> > > > > >> > > branch in apache/incubator-mxnet >> > > > > >> > > may it be master/PR/non-PR >> > > > > >> > > Reason: >> > > > > >> > > Because all the 3 types of branches are discovered by >> Jenkins >> > > > > (non-PR >> > > > > >> > > (including master) and PR) >> > > > > >> > > >> > > > > >> > > Proposal: Remove GitHub WebHook to Jenkins and replace >> with GH >> > > > > >> Webhook to >> > > > > >> > > Lambda >> > > > > >> > > But after I remove the github webhook that points to >> Jenkins : >> > > > *N**o >> > > > > >> > commit >> > > > > >> > > will trigger Jenkins build by default* (as Jenkins wont >> > receive >> > > GH >> > > > > >> > events) >> > > > > >> > > Only those that Bot deems fit will be triggered (using >> Jenkins >> > > API >> > > > > >> > invoked >> > > > > >> > > by Lambda). >> > > > > >> > > Hence its needed to handle that case of master merge. >> > > > > >> > > Am I understanding this correctly? >> > > > > >> > > >> > > > > >> > > On Wed, 18 Mar 2020 at 04:23, Marco de Abreu < >> > > > > [email protected] >> > > > > >> > >> > > > > >> > > wrote: >> > > > > >> > > >> > > > > >> > > > Thanks Chai, sounds good to me. Could you elaborate a >> bit on >> > > the >> > > > > >> point >> > > > > >> > > > about triggering a CI run after the PR has been merged? >> We >> > > > already >> > > > > >> to >> > > > > >> > > that >> > > > > >> > > > automatically for the master, so what's the benefit to >> do it >> > > > > twice? >> > > > > >> > > > >> > > > > >> > > > -Marco >> > > > > >> > > > >> > > > > >> > > > Chaitanya Bapat <[email protected]> schrieb am Mi., >> 18. >> > > März >> > > > > >> 2020, >> > > > > >> > > > 09:30: >> > > > > >> > > > >> > > > > >> > > > > Update: >> > > > > >> > > > > >> > > > > >> > > > > > we can ensure that all CI runs ran on the commit >> that >> > > will >> > > > be >> > > > > >> > merged >> > > > > >> > > > > @Sam Skalicky <[email protected]> Branch >> Protection >> > is >> > > > > added >> > > > > >> to >> > > > > >> > > > public >> > > > > >> > > > > MXNet repo. It ensures that for every PR to be merged, >> the >> > > CI >> > > > > >> passes. >> > > > > >> > > All >> > > > > >> > > > > the jobs selected "required" jobs will have to be green >> > for >> > > > the >> > > > > >> PR to >> > > > > >> > > be >> > > > > >> > > > > merged. Ofcourse, users with "Adminstrator" access can >> > merge >> > > > > >> without >> > > > > >> > it >> > > > > >> > > > but >> > > > > >> > > > > that's just a backdoor. It is the case now and will >> > continue >> > > > to >> > > > > be >> > > > > >> > the >> > > > > >> > > > case >> > > > > >> > > > > with the inclusion of Bot. >> > > > > >> > > > > >> > > > > >> > > > > > easily verify that the CI has executed all runs on >> the >> > > > commit >> > > > > >> that >> > > > > >> > > will >> > > > > >> > > > > be merged >> > > > > >> > > > > GitHub UI shows all the jobs and the status >> corresponding >> > to >> > > > it >> > > > > on >> > > > > >> > > every >> > > > > >> > > > > commit. That should suffice. For the merged commits, >> Repo >> > -> >> > > > > >> Commits >> > > > > >> > -> >> > > > > >> > > > > Commit ID (Status) can be tracked currently (only way >> > that I >> > > > > know >> > > > > >> > of). >> > > > > >> > > > > Moreover, it is beyond the scope of this project (and >> > > possibly >> > > > > >> out of >> > > > > >> > > our >> > > > > >> > > > > control since this is purely GitHub UI specific >> use-case). >> > > > > >> > > > > >> > > > > >> > > > > Thanks @przemyslaw for supporting the opt-in. >> > > > > >> > > > > >> > > > > >> > > > > Thanks everyone in the community for sharing concerns, >> > > voicing >> > > > > >> your >> > > > > >> > > > opinion >> > > > > >> > > > > and participating in the discussion. >> > > > > >> > > > > Thanks to those who attended the demo last Friday. >> > > > > >> > > > > >> > > > > >> > > > > Action items from that discussion >> > > > > >> > > > > 1. Handle master merge builds [Done] >> > > > > >> > > > > Bot runs entire CI suite after the PR is merged and >> > comments >> > > > on >> > > > > >> the >> > > > > >> > PR >> > > > > >> > > > > about the same. >> > > > > >> > > > > Design decision : >> > > > > >> > > > > MXNet Bot comment about master merge build on the >> *merge >> > > > commit >> > > > > vs >> > > > > >> > PR*. >> > > > > >> > > > > After the PR is merged, Bot runs entire CI and comments >> > the >> > > > > >> result of >> > > > > >> > > CI >> > > > > >> > > > > trigger on the PR (because it is easy to track on a PR >> > > rather >> > > > > than >> > > > > >> > > > > commenting inside the merge commit) >> > > > > >> > > > > >> > > > > >> > > > > 2. Idempotent condition >> > > > > >> > > > > In case of already running build, if an attempt is >> made to >> > > > > >> retrigger >> > > > > >> > > the >> > > > > >> > > > > job then what should be the response >> > > > > >> > > > > a. Not to re-trigger, let the ongoing build continue >> till >> > > > > >> completion >> > > > > >> > > > > b. End the ongoing build and re-trigger >> > > > > >> > > > > c. Let the ongoing build continue, re-trigger new build >> > > > > >> > > > > >> > > > > >> > > > > From resource saving point of view, *c* looks costly >> and a >> > > can >> > > > > be >> > > > > >> > > > > avoided/optimized by B. >> > > > > >> > > > > In case when a re-trigger was started "erroneously" >> then >> > > > killing >> > > > > >> > > ongoing >> > > > > >> > > > > build and re-trigger is a waste. >> > > > > >> > > > > In case when ongoing build failed in one sub-part, then >> > > > > >> re-triggering >> > > > > >> > > is >> > > > > >> > > > > justified. >> > > > > >> > > > > Erroneous re-triggers would be less often while >> conscious >> > > > > >> re-triggers >> > > > > >> > > to >> > > > > >> > > > > suppress failure is more common use-case. It looks >> like a >> > > safe >> > > > > >> > > assumption >> > > > > >> > > > > to make given the trade-off. >> > > > > >> > > > > [Open to debate] >> > > > > >> > > > > >> > > > > >> > > > > 3. Add security consideration [Use of secret manager, >> but >> > > > > without >> > > > > >> > > > > auto-rotation due to Jenkins manual config requirement] >> > > [Done] >> > > > > >> > > > > 4. New PR Instruction message by the Bot [Done] >> > > > > >> > > > > Thanks to the suggestion of Leonard, supported by >> others. >> > > I've >> > > > > now >> > > > > >> > > added >> > > > > >> > > > > the feature where the Bot comments a help message. [For >> > > > > reference >> > > > > >> - >> > > > > >> > > > > https://github.com/ChaiBapchya/incubator-mxnet/pull/52 >> ] >> > > > > >> > > > > >> > > > > >> > > > > Barring the opt-in vs opt-out debate & idempotency, >> > > consensus >> > > > > was >> > > > > >> > > quickly >> > > > > >> > > > > reached for the rest. >> > > > > >> > > > > >> > > > > >> > > > > In the coming days, I hope to roll-out this feature >> into >> > > Prod >> > > > > >> (public >> > > > > >> > > > > MXNet) for all devs to use. >> > > > > >> > > > > >> > > > > >> > > > > Thanks, >> > > > > >> > > > > Chai >> > > > > >> > > > > >> > > > > >> > > > > On Mon, 16 Mar 2020 at 11:57, Marco de Abreu < >> > > > > >> > [email protected]> >> > > > > >> > > > > wrote: >> > > > > >> > > > > >> > > > > >> > > > > > Well that's generally a problem with a deferred CI >> > > approach >> > > > > (CI >> > > > > >> is >> > > > > >> > > run >> > > > > >> > > > at >> > > > > >> > > > > > commit and not at merge time). This can either be >> solved >> > > > > through >> > > > > >> > the >> > > > > >> > > > > other >> > > > > >> > > > > > proposal that's currently on dev@, by having a bot >> > which >> > > > does >> > > > > >> > merges >> > > > > >> > > > by >> > > > > >> > > > > > having a global lock and a merge queue or by >> accepting >> > the >> > > > > >> issue. >> > > > > >> > > > Reality >> > > > > >> > > > > > right now is that we're running that model where two >> PRs >> > > > which >> > > > > >> are >> > > > > >> > > > merged >> > > > > >> > > > > > in parallel might break one another. One thing to >> > consider >> > > > > >> though >> > > > > >> > is >> > > > > >> > > > that >> > > > > >> > > > > > this breakage would have to be introduced in two >> > separate >> > > > > parts >> > > > > >> > since >> > > > > >> > > > > > otherwise there'd be merge conflicts. I think we had >> > that >> > > > > >> situation >> > > > > >> > > > twice >> > > > > >> > > > > > so far and the result was a quick revert, so I'd say >> > that >> > > > > it's a >> > > > > >> > > > problem >> > > > > >> > > > > > that can happily be accepted. All other solutions >> > > basically >> > > > > >> require >> > > > > >> > > > some >> > > > > >> > > > > > form of single-threaded and globally locked solution >> > which >> > > > > >> limits >> > > > > >> > us >> > > > > >> > > in >> > > > > >> > > > > > scalability. I'd recommend to just accept that risk >> and >> > > > revert >> > > > > >> a PR >> > > > > >> > > in >> > > > > >> > > > > case >> > > > > >> > > > > > it actually had a conflict. >> > > > > >> > > > > > >> > > > > >> > > > > > -Marco >> > > > > >> > > > > > >> > > > > >> > > > > > On Mon, Mar 16, 2020 at 6:29 PM Skalicky, Sam >> > > > > >> > > > <[email protected] >> > > > > >> > > > > > >> > > > > >> > > > > > wrote: >> > > > > >> > > > > > >> > > > > >> > > > > > > We probably need some way to track which CI runs >> ran >> > for >> > > > > which >> > > > > >> > > commit >> > > > > >> > > > > > too, >> > > > > >> > > > > > > that way we can ensure that all CI runs ran on the >> > > commit >> > > > > that >> > > > > >> > will >> > > > > >> > > > be >> > > > > >> > > > > > > merged. Maybe the bot can comment with the commit >> > hash >> > > > when >> > > > > >> > users >> > > > > >> > > > > > command >> > > > > >> > > > > > > it to do something. Although since users can >> trigger >> > > > > >> individual >> > > > > >> > CI >> > > > > >> > > > runs >> > > > > >> > > > > > its >> > > > > >> > > > > > > possible to have some commits run some CI runs but >> not >> > > > > >> others. We >> > > > > >> > > > need >> > > > > >> > > > > > some >> > > > > >> > > > > > > way to easily verify that the CI has executed all >> runs >> > > on >> > > > > the >> > > > > >> > > commit >> > > > > >> > > > > that >> > > > > >> > > > > > > will be merged. >> > > > > >> > > > > > > >> > > > > >> > > > > > > Sam >> > > > > >> > > > > > > >> > > > > >> > > > > > > > On Mar 13, 2020, at 8:28 PM, Przemysław Trędak < >> > > > > >> > > [email protected] >> > > > > >> > > > > >> > > > > >> > > > > > > wrote: >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > CAUTION: This email originated from outside of >> the >> > > > > >> > organization. >> > > > > >> > > Do >> > > > > >> > > > > not >> > > > > >> > > > > > > click links or open attachments unless you can >> confirm >> > > the >> > > > > >> sender >> > > > > >> > > and >> > > > > >> > > > > > know >> > > > > >> > > > > > > the content is safe. >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > I personally like the idea of opt-in more than >> > > opt-out: >> > > > > >> > > > > > > > - ultimately PR author wants the PR to be merged >> so >> > > they >> > > > > (or >> > > > > >> > > > > committer >> > > > > >> > > > > > > reviewing the PR) will trigger the CI >> > > > > >> > > > > > > > - if it is easy to trigger the PR via the bot >> > command >> > > > then >> > > > > >> the >> > > > > >> > > > amount >> > > > > >> > > > > > of >> > > > > >> > > > > > > work per PR should be less than with opt-out (since >> > most >> > > > of >> > > > > >> the >> > > > > >> > > > commits >> > > > > >> > > > > > > should then be marked as [skip ci] or something >> > similar >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > +1 to the bot making a comment on each new PR >> with >> > its >> > > > > >> commands >> > > > > >> > > > (and >> > > > > >> > > > > > > also explaining, or at least giving links to the >> > general >> > > > PR >> > > > > >> > process >> > > > > >> > > > so >> > > > > >> > > > > > new >> > > > > >> > > > > > > PR authors are not lost). Maybe we could make the >> bot >> > > > > >> recognize >> > > > > >> > if >> > > > > >> > > > the >> > > > > >> > > > > PR >> > > > > >> > > > > > > author is new or existing contributor and offer >> advice >> > > > based >> > > > > >> on >> > > > > >> > > that? >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > Thanks >> > > > > >> > > > > > > > Przemek >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > On 2020/03/13 22:06:58, Marco de Abreu < >> > > > > >> > [email protected]> >> > > > > >> > > > > > wrote: >> > > > > >> > > > > > > >> Hi, >> > > > > >> > > > > > > >> >> > > > > >> > > > > > > >> since it's no longer necessary to push a new >> commit >> > > to >> > > > > >> trigger >> > > > > >> > > CI, >> > > > > >> > > > > it >> > > > > >> > > > > > > will >> > > > > >> > > > > > > >> already reduce the costs. But to me, requiring >> an >> > > > action >> > > > > to >> > > > > >> > > enable >> > > > > >> > > > > CI >> > > > > >> > > > > > > after >> > > > > >> > > > > > > >> a PR has been created initially, is a no go. >> User >> > can >> > > > opt >> > > > > >> out >> > > > > >> > of >> > > > > >> > > > CI, >> > > > > >> > > > > > but >> > > > > >> > > > > > > >> the default has to be CI being triggered >> > > automatically >> > > > > for >> > > > > >> > every >> > > > > >> > > > > > commit >> > > > > >> > > > > > > >> unless specifically disabled by a participant. >> I'm >> > > also >> > > > > >> fine >> > > > > >> > > with >> > > > > >> > > > > > > >> triggering certain additional jobs (think about >> > > > running a >> > > > > >> > > nightly >> > > > > >> > > > > job >> > > > > >> > > > > > > upon >> > > > > >> > > > > > > >> request for a PR) to require a manual step, but >> the >> > > PR >> > > > > >> > > validation >> > > > > >> > > > > > > pipelines >> > > > > >> > > > > > > >> have to run automatically. Every check that is >> > marked >> > > > as >> > > > > >> > > > "Required" >> > > > > >> > > > > in >> > > > > >> > > > > > > >> GitHub has to be automatically kicked off. >> > > > > >> > > > > > > >> >> > > > > >> > > > > > > >> -Marco >> > > > > >> > > > > > > >> >> > > > > >> > > > > > > >> On Fri, Mar 13, 2020 at 9:50 PM Chaitanya Bapat >> < >> > > > > >> > > > > [email protected] >> > > > > >> > > > > > > >> > > > > >> > > > > > > >> wrote: >> > > > > >> > > > > > > >> >> > > > > >> > > > > > > >>> Firstly, >> > > > > >> > > > > > > >>> Sorry I missed out on attaching the mail thread >> > that >> > > > was >> > > > > >> sent >> > > > > >> > > on >> > > > > >> > > > > 12th >> > > > > >> > > > > > > >>> February for notifying the community of the >> > upcoming >> > > > > >> changes >> > > > > >> > to >> > > > > >> > > > the >> > > > > >> > > > > > > MXNet >> > > > > >> > > > > > > >>> CI >> > > > > >> > > > > > > >>> For reference : >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> >> > > > > >> > > > >> > > >> > >> https://lists.apache.org/thread.html/r09a6ab2803a996fc80e00fe39ed312fa4865e8805e08df847f1addad%40%3Cdev.mxnet.apache.org%3E >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> Now to the questions, >> > > > > >> > > > > > > >>>> Is it possible for re-triggering a single job >> to >> > be >> > > > > >> abused? >> > > > > >> > > > > > > >>> @Tao In the case when a user re-triggers a >> single >> > > job >> > > > > >> > multiple >> > > > > >> > > > > times, >> > > > > >> > > > > > > that >> > > > > >> > > > > > > >>> will be visible in the PR conversation thread. >> A >> > > > > >> committer, >> > > > > >> > > even >> > > > > >> > > > > > after >> > > > > >> > > > > > > he >> > > > > >> > > > > > > >>> has approved the PR before, generally takes a >> look >> > > at >> > > > > the >> > > > > >> > final >> > > > > >> > > > > state >> > > > > >> > > > > > > of >> > > > > >> > > > > > > >>> the PR before merging. Would it be fair to >> assume >> > > the >> > > > > >> > committer >> > > > > >> > > > > could >> > > > > >> > > > > > > take >> > > > > >> > > > > > > >>> the multiple re-trigger of a single job into >> > account >> > > > > >> before >> > > > > >> > > > > merging? >> > > > > >> > > > > > > The >> > > > > >> > > > > > > >>> committer then has the option to invoke >> > `@mxnet-bot >> > > > run >> > > > > ci >> > > > > >> > > [all] >> > > > > >> > > > ` >> > > > > >> > > > > to >> > > > > >> > > > > > > >>> trigger the entire build pipeline one last to >> > > counter >> > > > > the >> > > > > >> > > abuse. >> > > > > >> > > > > This >> > > > > >> > > > > > > is >> > > > > >> > > > > > > >>> aligned with what @Leonard said. >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> @Sandeep Thanks a lot for collecting and >> sharing >> > > > > valuable >> > > > > >> > data. >> > > > > >> > > > I'd >> > > > > >> > > > > > > concur >> > > > > >> > > > > > > >>> with the opinion that given the existing things >> > > > > >> committers & >> > > > > >> > PR >> > > > > >> > > > > > Authors >> > > > > >> > > > > > > >>> take care of, invoking CI shouldn't be that >> big of >> > > an >> > > > > >> > > additional >> > > > > >> > > > > > > burden. >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> @Marco With the opt-out, the onus remains on >> the >> > PR >> > > > > >> Author. >> > > > > >> > It >> > > > > >> > > > > > doesn't >> > > > > >> > > > > > > help >> > > > > >> > > > > > > >>> reduce the resource usage. Hence, it was >> suggested >> > > to >> > > > > >> switch >> > > > > >> > to >> > > > > >> > > > > > > >>> opt-in. @Leo's suggestion for proactive >> commenting >> > > on >> > > > > the >> > > > > >> > part >> > > > > >> > > of >> > > > > >> > > > > bot >> > > > > >> > > > > > > makes >> > > > > >> > > > > > > >>> sense and is doable. >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> Default : opt-out and User initiated opt-in >> (with >> > > > > >> addressing >> > > > > >> > > > Leo's >> > > > > >> > > > > > fix >> > > > > >> > > > > > > for >> > > > > >> > > > > > > >>> the usability issue you correctly pointed out ) >> > > > > >> > > > > > > >>> @Marco How does this sound to you? >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> Again, thank you all for chiming in and voicing >> > your >> > > > > >> opinion. >> > > > > >> > > > > > > Appreciate >> > > > > >> > > > > > > >>> it. >> > > > > >> > > > > > > >>> We can take ahead these discussions in today's >> > demo >> > > > > >> meeting. >> > > > > >> > > > > [Design >> > > > > >> > > > > > > Doc >> > > > > >> > > > > > > >>> < >> > > > > >> > > >> > https://cwiki.apache.org/confluence/display/MXNET/MXNet+CI+Bot >> > > > > >> > > > >] >> > > > > >> > > > > > > [Demo >> > > > > >> > > > > > > >>> Video < >> > https://www.youtube.com/watch?v=gfOGwZId8aU >> > > >] >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> Thanks, >> > > > > >> > > > > > > >>> Chai >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> On Fri, 13 Mar 2020 at 12:34, Marco de Abreu < >> > > > > >> > > > > > [email protected]> >> > > > > >> > > > > > > >>> wrote: >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>>> I'd recommend that the bot makes an initial >> > comment >> > > > > when >> > > > > >> a >> > > > > >> > PR >> > > > > >> > > > gets >> > > > > >> > > > > > > opened >> > > > > >> > > > > > > >>>> and informs the users of its commands. It then >> > > tells >> > > > > the >> > > > > >> > user >> > > > > >> > > > the >> > > > > >> > > > > > > commend >> > > > > >> > > > > > > >>>> to opt out of CI. >> > > > > >> > > > > > > >>>> >> > > > > >> > > > > > > >>>> -Marco >> > > > > >> > > > > > > >>>> >> > > > > >> > > > > > > >>>> Lausen, Leonard <[email protected]> >> > > schrieb >> > > > am >> > > > > >> Fr., >> > > > > >> > > 13. >> > > > > >> > > > > > März >> > > > > >> > > > > > > >>> 2020, >> > > > > >> > > > > > > >>>> 20:27: >> > > > > >> > > > > > > >>>> >> > > > > >> > > > > > > >>>>> On opt-out: People may be unaware of opt-out >> > would >> > > > not >> > > > > >> use >> > > > > >> > > it. >> > > > > >> > > > > > There >> > > > > >> > > > > > > is >> > > > > >> > > > > > > >>>> no >> > > > > >> > > > > > > >>>>> incentive to use opt-out, as the PR author >> > doesn't >> > > > pay >> > > > > >> any >> > > > > >> > > > money >> > > > > >> > > > > > for >> > > > > >> > > > > > > CI >> > > > > >> > > > > > > >>>>> run. >> > > > > >> > > > > > > >>>>> >> > > > > >> > > > > > > >>>>> I agree with Marco though that opt-in alone >> may >> > > > cause >> > > > > >> > > usability >> > > > > >> > > > > > > issues, >> > > > > >> > > > > > > >>>> as >> > > > > >> > > > > > > >>>>> contributors may not be aware of how to >> trigger >> > > the >> > > > > CI. >> > > > > >> > > > > > > >>>>> One solution is that the bot proactively >> > comments >> > > on >> > > > > >> the PR >> > > > > >> > > and >> > > > > >> > > > > > > reminds >> > > > > >> > > > > > > >>>> the >> > > > > >> > > > > > > >>>>> author to trigger running CI once the author >> > deems >> > > > the >> > > > > >> PR >> > > > > >> > > > ready. >> > > > > >> > > > > > > >>>>> >> > > > > >> > > > > > > >>>>> But even if we choose opt-out, the bot will >> > still >> > > > add >> > > > > a >> > > > > >> lot >> > > > > >> > > of >> > > > > >> > > > > > value, >> > > > > >> > > > > > > >>> as >> > > > > >> > > > > > > >>>> PR >> > > > > >> > > > > > > >>>>> authors can retrigger single jobs that have >> > failed >> > > > due >> > > > > >> to >> > > > > >> > > > > > flakiness. >> > > > > >> > > > > > > >>>>> >> > > > > >> > > > > > > >>>>>> Is it possible for re-triggering a single >> job >> > to >> > > be >> > > > > >> > abused? >> > > > > >> > > > For >> > > > > >> > > > > > > >>>> example, >> > > > > >> > > > > > > >>>>>> the author spends two days re-triggering a >> > flaky >> > > > job >> > > > > to >> > > > > >> > make >> > > > > >> > > > it >> > > > > >> > > > > > > pass. >> > > > > >> > > > > > > >>>>> >> > > > > >> > > > > > > >>>>> Yes, this is possible. I suggest the >> committer >> > who >> > > > > >> likes to >> > > > > >> > > > > merge a >> > > > > >> > > > > > > PR >> > > > > >> > > > > > > >>>>> needs to >> > > > > >> > > > > > > >>>>> make a good judgement here if a PR is abusing >> > the >> > > > > >> feature, >> > > > > >> > > and >> > > > > >> > > > if >> > > > > >> > > > > > so, >> > > > > >> > > > > > > >>>>> retrigger >> > > > > >> > > > > > > >>>>> all CI runs. >> > > > > >> > > > > > > >>>>> >> > > > > >> > > > > > > >>>>> Best regards >> > > > > >> > > > > > > >>>>> Leonard >> > > > > >> > > > > > > >>>>> >> > > > > >> > > > > > > >>>>> On Fri, 2020-03-13 at 08:07 +0100, Marco de >> > Abreu >> > > > > wrote: >> > > > > >> > > > > > > >>>>>> Thanks for the data Sandeep. In these cases >> it >> > > > sounds >> > > > > >> like >> > > > > >> > > it >> > > > > >> > > > > > would >> > > > > >> > > > > > > >>>> have >> > > > > >> > > > > > > >>>>>> rather been better when people explicitly >> > turned >> > > > off >> > > > > >> CI in >> > > > > >> > > > that >> > > > > >> > > > > > > case. >> > > > > >> > > > > > > >>>>>> What's the argument against an opt-out >> instead >> > of >> > > > an >> > > > > >> > opt-in? >> > > > > >> > > > > > > >>>>>> >> > > > > >> > > > > > > >>>>>> My intention is that I consider it quite >> > > cumbersome >> > > > > to >> > > > > >> > make >> > > > > >> > > > it a >> > > > > >> > > > > > > >>>>> *required* >> > > > > >> > > > > > > >>>>>> step to always trigger CI manually, even if >> > just >> > > > > >> > submitting >> > > > > >> > > a >> > > > > >> > > > > > small >> > > > > >> > > > > > > >>> PR. >> > > > > >> > > > > > > >>>>> I'd >> > > > > >> > > > > > > >>>>>> rather see people explicitly turning off CI >> if >> > > they >> > > > > >> > wouldn't >> > > > > >> > > > > like >> > > > > >> > > > > > to >> > > > > >> > > > > > > >>>> use >> > > > > >> > > > > > > >>>>> it >> > > > > >> > > > > > > >>>>>> - and there's also the "draft" stage for a >> PR >> > > which >> > > > > >> some >> > > > > >> > > > > > > contributors >> > > > > >> > > > > > > >>>> are >> > > > > >> > > > > > > >>>>>> using. >> > > > > >> > > > > > > >>>>>> >> > > > > >> > > > > > > >>>>>> With regards to WIP and do not review: I >> think >> > > > these >> > > > > >> are >> > > > > >> > use >> > > > > >> > > > > cases >> > > > > >> > > > > > > >>>> where >> > > > > >> > > > > > > >>>>>> you want CI feedback, as otherwise you >> wouldn't >> > > > have >> > > > > >> > opened >> > > > > >> > > > the >> > > > > >> > > > > > PR. >> > > > > >> > > > > > > >>> If >> > > > > >> > > > > > > >>>>> you >> > > > > >> > > > > > > >>>>>> don't want human feedback and neither >> machine >> > > > > feedback, >> > > > > >> > why >> > > > > >> > > > open >> > > > > >> > > > > > the >> > > > > >> > > > > > > >>> PR >> > > > > >> > > > > > > >>>>> at >> > > > > >> > > > > > > >>>>>> all? >> > > > > >> > > > > > > >>>>>> >> > > > > >> > > > > > > >>>>>> -Marco >> > > > > >> > > > > > > >>>>>> >> > > > > >> > > > > > > >>>>>> >> > > > > >> > > > > > > >>>>>> sandeep krishnamurthy < >> > > [email protected] >> > > > > >> > > > > >> > schrieb >> > > > > >> > > > am >> > > > > >> > > > > > Fr., >> > > > > >> > > > > > > >>>> 13. >> > > > > >> > > > > > > >>>>>> März 2020, 05:24: >> > > > > >> > > > > > > >>>>>> >> > > > > >> > > > > > > >>>>>>> I tried to gather some data for us to >> discuss >> > > this >> > > > > >> topic >> > > > > >> > in >> > > > > >> > > > > this >> > > > > >> > > > > > > >>>>> thread. I >> > > > > >> > > > > > > >>>>>>> tried to count number of un-necessary >> builds >> > by >> > > > > >> looking >> > > > > >> > at >> > > > > >> > > > most >> > > > > >> > > > > > > >>>> recent >> > > > > >> > > > > > > >>>>> (as >> > > > > >> > > > > > > >>>>>>> of 12, March 9 PM PST) 50 PRs merged to >> master >> > > and >> > > > > 50 >> > > > > >> > PRs. >> > > > > >> > > > > > > >>>>>>> Identifying un-necessary builds is bit >> > > > subjective. I >> > > > > >> > tried >> > > > > >> > > to >> > > > > >> > > > > be >> > > > > >> > > > > > > >>> more >> > > > > >> > > > > > > >>>>>>> conservative where I didn't count a build >> as >> > > > > >> un-necessary >> > > > > >> > > if >> > > > > >> > > > I >> > > > > >> > > > > > was >> > > > > >> > > > > > > >>> in >> > > > > >> > > > > > > >>>>>>> doubt. Hence, I was not able to automate, >> but >> > I >> > > > made >> > > > > >> an >> > > > > >> > > > effort >> > > > > >> > > > > to >> > > > > >> > > > > > > >>> go >> > > > > >> > > > > > > >>>>>>> through PRs manually and use below >> criteria to >> > > > > >> identify >> > > > > >> > > > > > > >>> un-necessary >> > > > > >> > > > > > > >>>>>>> commits triggering the builds. >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> 1. Explicitly marked as WIP / do not >> review >> > > PR >> > > > > >> > > > > > > >>>>>>> 2. Incremental WIP commit and finally >> > > > commenting a >> > > > > >> > commit >> > > > > >> > > > > > > >>> “trigger >> > > > > >> > > > > > > >>>>> CI” >> > > > > >> > > > > > > >>>>>>> 3. Multiple commits to address all >> comments >> > > from >> > > > > >> single >> > > > > >> > > > > review. >> > > > > >> > > > > > > >>>>> This is >> > > > > >> > > > > > > >>>>>>> assuming we see a comment, address them, >> > > commit, >> > > > > >> next >> > > > > >> > the >> > > > > >> > > > > > > >>>> following >> > > > > >> > > > > > > >>>>>>> comment >> > > > > >> > > > > > > >>>>>>> 4. Sequence of documentation only changes >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> I found there were around 42 avoidable >> builds >> > > from >> > > > > >> most >> > > > > >> > > > recent >> > > > > >> > > > > 50 >> > > > > >> > > > > > > >>>>> merged >> > > > > >> > > > > > > >>>>>>> PRs and around 86 builds from recent 50 >> open >> > > PRs. >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> I synced up with other contributors (Joe >> > Evans, >> > > > > Chai) >> > > > > >> > from >> > > > > >> > > > > Amazon >> > > > > >> > > > > > > >>> who >> > > > > >> > > > > > > >>>>> is >> > > > > >> > > > > > > >>>>>>> contributing to MXNet CI system. I was told >> > that >> > > > on >> > > > > an >> > > > > >> > > > average >> > > > > >> > > > > it >> > > > > >> > > > > > > >>>> costs >> > > > > >> > > > > > > >>>>>>> around $84 per build and on an average 6 >> > commits >> > > > per >> > > > > >> > merged >> > > > > >> > > > PR >> > > > > >> > > > > > (for >> > > > > >> > > > > > > >>>>> year >> > > > > >> > > > > > > >>>>>>> 2019). Going by that, it is approximately >> 1/6 >> > > > builds >> > > > > >> are >> > > > > >> > > > > > avoidable. >> > > > > >> > > > > > > >>>>> [100 / >> > > > > >> > > > > > > >>>>>>> 300 + 300 ] >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> Usability should be top most priority. But, >> > > since >> > > > > >> either >> > > > > >> > a >> > > > > >> > > > > > reviewer >> > > > > >> > > > > > > >>>> or >> > > > > >> > > > > > > >>>>> pr >> > > > > >> > > > > > > >>>>>>> author can trigger the bot, is it really a >> > > hurdle >> > > > > for >> > > > > >> pr >> > > > > >> > > > author >> > > > > >> > > > > > or >> > > > > >> > > > > > > >>>>> reviewer >> > > > > >> > > > > > > >>>>>>> to call a bot to trigger CI? Given that PR >> > > author >> > > > > and >> > > > > >> > > > reviewer >> > > > > >> > > > > is >> > > > > >> > > > > > > >>>>> already >> > > > > >> > > > > > > >>>>>>> actively commenting various details such >> as - >> > PR >> > > > > >> > > description, >> > > > > >> > > > > > > >>> review >> > > > > >> > > > > > > >>>>>>> comments and responses, adding labels etc. >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> Me too curious to know the behavior for >> Tao's >> > > > above >> > > > > >> use >> > > > > >> > > case. >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> Best, >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> Sandeep >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> On Thu, Mar 12, 2020 at 7:18 PM Tao Lv < >> > > > > >> > [email protected] >> > > > > >> > > > >> > > > > >> > > > > > wrote: >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>>> Is it possible for re-triggering a single >> job >> > > to >> > > > be >> > > > > >> > > abused? >> > > > > >> > > > > For >> > > > > >> > > > > > > >>>>> example, >> > > > > >> > > > > > > >>>>>>>> the author spends two days re-triggering a >> > > flaky >> > > > > job >> > > > > >> to >> > > > > >> > > make >> > > > > >> > > > > it >> > > > > >> > > > > > > >>>>> pass. But >> > > > > >> > > > > > > >>>>>>>> other jobs which have passed the >> validation >> > may >> > > > be >> > > > > >> > broken >> > > > > >> > > by >> > > > > >> > > > > > > >>> other >> > > > > >> > > > > > > >>>>>>> commits >> > > > > >> > > > > > > >>>>>>>> during the two day without being noticed. >> And >> > > > > finally >> > > > > >> > the >> > > > > >> > > PR >> > > > > >> > > > > is >> > > > > >> > > > > > > >>>>> merged >> > > > > >> > > > > > > >>>>>>> with >> > > > > >> > > > > > > >>>>>>>> underlying problems. >> > > > > >> > > > > > > >>>>>>>> >> > > > > >> > > > > > > >>>>>>>> On Fri, Mar 13, 2020 at 6:19 AM Marco de >> > Abreu >> > > < >> > > > > >> > > > > > > >>>>> [email protected]> >> > > > > >> > > > > > > >>>>>>>> wrote: >> > > > > >> > > > > > > >>>>>>>> >> > > > > >> > > > > > > >>>>>>>>> In the end it only comes down to money, >> > > > > considering >> > > > > >> > that >> > > > > >> > > > the >> > > > > >> > > > > > > >>>>> system is >> > > > > >> > > > > > > >>>>>>>> auto >> > > > > >> > > > > > > >>>>>>>>> scaling, making the execution time >> constant. >> > > > > >> > > > > > > >>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>> If we're trading money for usability, I >> > > > certainly >> > > > > >> would >> > > > > >> > > > > prefer >> > > > > >> > > > > > > >>>>>>> usability. >> > > > > >> > > > > > > >>>>>>>>> I'd rather recommend to spend time on >> > > > > parallelizing >> > > > > >> > test >> > > > > >> > > > > > > >>>> execution >> > > > > >> > > > > > > >>>>> or >> > > > > >> > > > > > > >>>>>>>>> getting rid of integration tests in the >> PR >> > > stage >> > > > > >> > instead >> > > > > >> > > > > > > >>> reducing >> > > > > >> > > > > > > >>>>> the >> > > > > >> > > > > > > >>>>>>>> costs >> > > > > >> > > > > > > >>>>>>>>> by making people not use it. But taking a >> > step >> > > > > back >> > > > > >> to >> > > > > >> > > > > > > >>> requiring >> > > > > >> > > > > > > >>>>> people >> > > > > >> > > > > > > >>>>>>>> to >> > > > > >> > > > > > > >>>>>>>>> manually trigger CI again doesn't feel >> > right. >> > > > > >> > > > > > > >>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>> I'm happy to see that bot deployed, but >> I do >> > > not >> > > > > >> agree >> > > > > >> > > with >> > > > > >> > > > > > > >>>>> removing >> > > > > >> > > > > > > >>>>>>> the >> > > > > >> > > > > > > >>>>>>>>> auto trigger functionality for new >> commits. >> > > > > >> > > > > > > >>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>> -Marco >> > > > > >> > > > > > > >>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>> Chaitanya Bapat <[email protected]> >> > > schrieb >> > > > am >> > > > > >> Do., >> > > > > >> > > 12. >> > > > > >> > > > > > > >>> März >> > > > > >> > > > > > > >>>>> 2020, >> > > > > >> > > > > > > >>>>>>>>> 22:47: >> > > > > >> > > > > > > >>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> @Marco Thanks for pointing that out. >> > > > > >> > > > > > > >>>>>>>>>> Tomorrow i.e. Friday, March 13, 2020 at >> > 3:00 >> > > > PM - >> > > > > >> 3:30 >> > > > > >> > > PM >> > > > > >> > > > in >> > > > > >> > > > > > > >>>>>>>> (UTC-08:00) >> > > > > >> > > > > > > >>>>>>>>>> Pacific Time (US & Canada). >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>> When do we expect this bot to be >> deployed? >> > > > > >> > > > > > > >>>>>>>>>> @Lin If all goes well in the next week I >> > can >> > > > > >> deploy it >> > > > > >> > > to >> > > > > >> > > > > > > >>>> public >> > > > > >> > > > > > > >>>>>>> Apache >> > > > > >> > > > > > > >>>>>>>>>> (provided I get permissions from Apache >> > > Infra) >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> @Marco Thanks for your feedback. >> > > > > >> > > > > > > >>>>>>>>>>> CI system has to support the community >> > > without >> > > > > >> > > requiring >> > > > > >> > > > > > > >>>>> people to >> > > > > >> > > > > > > >>>>>>>>>> constantly shepherd every single run >> > > > > >> > > > > > > >>>>>>>>>> We have data for the number of times CI >> was >> > > > > >> triggered >> > > > > >> > > > > > > >>>>> unnecessarily >> > > > > >> > > > > > > >>>>>>>> which >> > > > > >> > > > > > > >>>>>>>>>> includes >> > > > > >> > > > > > > >>>>>>>>>> - Entire build triggered instead of >> > specific >> > > > > build >> > > > > >> > > > > > > >>>>>>>>>> - CI triggered when PR is still work in >> > > > progress >> > > > > or >> > > > > >> > not >> > > > > >> > > > yet >> > > > > >> > > > > > > >>>> ready >> > > > > >> > > > > > > >>>>>>> (say >> > > > > >> > > > > > > >>>>>>>> - >> > > > > >> > > > > > > >>>>>>>>>> intermediate commits) >> > > > > >> > > > > > > >>>>>>>>>> At the end its a trade-off >> > > > > >> > > > > > > >>>>>>>>>> Money, Resources, Time to build for each >> > and >> > > > > every >> > > > > >> > > commit >> > > > > >> > > > vs >> > > > > >> > > > > > > >>>>> Pain of >> > > > > >> > > > > > > >>>>>>>>>> triggering builds >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>> Scan trigger plugin would poll SCM. >> Can we >> > > use >> > > > > >> plugin >> > > > > >> > > at >> > > > > >> > > > > > > >>>>> scale? >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> 1. I haven't tested it on scale. But I >> > think >> > > > with >> > > > > >> the >> > > > > >> > > > > current >> > > > > >> > > > > > > >>>>> scale >> > > > > >> > > > > > > >>>>>>> of >> > > > > >> > > > > > > >>>>>>>>>> MXNet repo (191 open PRs i.e. checking >> for >> > > > > changes >> > > > > >> to >> > > > > >> > > 191 >> > > > > >> > > > > > > >>>>> branches - >> > > > > >> > > > > > > >>>>>>> It >> > > > > >> > > > > > > >>>>>>>>>> should be manageable) >> > > > > >> > > > > > > >>>>>>>>>> 2. What's the purpose of the plugin? >> tldr; >> > > > Branch >> > > > > >> > > > discovery >> > > > > >> > > > > > > >>> or >> > > > > >> > > > > > > >>>>> branch >> > > > > >> > > > > > > >>>>>>>>>> indexing. >> > > > > >> > > > > > > >>>>>>>>>> Scan trigger plugin comes into the >> picture >> > > only >> > > > > >> once >> > > > > >> > per >> > > > > >> > > > PR >> > > > > >> > > > > > > >>> per >> > > > > >> > > > > > > >>>>> job >> > > > > >> > > > > > > >>>>>>>>> (i.e. 8 >> > > > > >> > > > > > > >>>>>>>>>> times per PR for 8 jobs). It is >> basically >> > > done >> > > > > >> when a >> > > > > >> > > new >> > > > > >> > > > PR >> > > > > >> > > > > > > >>> is >> > > > > >> > > > > > > >>>>> made >> > > > > >> > > > > > > >>>>>>>> and >> > > > > >> > > > > > > >>>>>>>>>> the job (say unix-cpu hasn't discovered >> the >> > > new >> > > > > PR >> > > > > >> > > branch >> > > > > >> > > > > > > >>> yet). >> > > > > >> > > > > > > >>>>>>> That's >> > > > > >> > > > > > > >>>>>>>>> it. >> > > > > >> > > > > > > >>>>>>>>>> So it shouldn't be a problem for public >> > MXNet >> > > > > repo. >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> Thanks, >> > > > > >> > > > > > > >>>>>>>>>> Chai >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> On Thu, 12 Mar 2020 at 14:22, Marco de >> > Abreu >> > > < >> > > > > >> > > > > > > >>>>>>> [email protected]> >> > > > > >> > > > > > > >>>>>>>>>> wrote: >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>> Btw you forgot to set a date and time >> for >> > > the >> > > > > >> metting >> > > > > >> > > > > > > >>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>> On Thu, Mar 12, 2020 at 10:18 PM Marco >> de >> > > > Abreu >> > > > > < >> > > > > >> > > > > > > >>>>>>>>> [email protected] >> > > > > >> > > > > > > >>>>>>>>>>> wrote: >> > > > > >> > > > > > > >>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>> Thanks Chai, I generally like the >> idea of >> > > the >> > > > > >> bot. >> > > > > >> > But >> > > > > >> > > > > > > >>> I'm >> > > > > >> > > > > > > >>>>> not a >> > > > > >> > > > > > > >>>>>>>>>>> supporter >> > > > > >> > > > > > > >>>>>>>>>>>> of the idea to disable any automatic >> > > > triggering >> > > > > >> > > > > > > >>> (disabling >> > > > > >> > > > > > > >>>>> the >> > > > > >> > > > > > > >>>>>>>>> webhook >> > > > > >> > > > > > > >>>>>>>>>> is >> > > > > >> > > > > > > >>>>>>>>>>>> also not an option, considering that >> this >> > > > will >> > > > > >> > disable >> > > > > >> > > > > > > >>>> master >> > > > > >> > > > > > > >>>>>>>>>> triggers). >> > > > > >> > > > > > > >>>>>>>>>>>> The CI system has to support the >> > community >> > > > > >> without >> > > > > >> > > > > > > >>>> requiring >> > > > > >> > > > > > > >>>>>>> people >> > > > > >> > > > > > > >>>>>>>>> to >> > > > > >> > > > > > > >>>>>>>>>>>> constantly shepherd every single run. >> > > > Disabling >> > > > > >> > > > automatic >> > > > > >> > > > > > > >>>>>>>> triggering >> > > > > >> > > > > > > >>>>>>>>>>> seems >> > > > > >> > > > > > > >>>>>>>>>>>> like a step back to me. >> > > > > >> > > > > > > >>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>> Instead, I'd recommend that CI gets >> > > triggered >> > > > > >> upon >> > > > > >> > > every >> > > > > >> > > > > > > >>>>> commit >> > > > > >> > > > > > > >>>>>>> as >> > > > > >> > > > > > > >>>>>>>>>> usual, >> > > > > >> > > > > > > >>>>>>>>>>>> but people have the possibility to >> call a >> > > > > >> "command" >> > > > > >> > > > (i.e. >> > > > > >> > > > > > > >>>>> make a >> > > > > >> > > > > > > >>>>>>>>>> message >> > > > > >> > > > > > > >>>>>>>>>>>> which results in the bot setting a >> label) >> > > to >> > > > > >> disable >> > > > > >> > > CI >> > > > > >> > > > > > > >>>> until >> > > > > >> > > > > > > >>>>>>> they >> > > > > >> > > > > > > >>>>>>>>>> revoke >> > > > > >> > > > > > > >>>>>>>>>>>> it. But the standard should still be >> > that a >> > > > new >> > > > > >> > commit >> > > > > >> > > > > > > >>>>> triggers a >> > > > > >> > > > > > > >>>>>>>> new >> > > > > >> > > > > > > >>>>>>>>>> CI >> > > > > >> > > > > > > >>>>>>>>>>>> run. >> > > > > >> > > > > > > >>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>> >> > > > > >> > > > > > > >>>> >> > > > > >> > https://plugins.jenkins.io/multibranch-scan-webhook-trigger/ >> > > > > >> > > > > > > >>>>>>>> seems >> > > > > >> > > > > > > >>>>>>>>>> like >> > > > > >> > > > > > > >>>>>>>>>>>> this would poll SCM. This will incur >> high >> > > > quota >> > > > > >> > > > > > > >>>>> restrictions. Are >> > > > > >> > > > > > > >>>>>>>> you >> > > > > >> > > > > > > >>>>>>>>>>> sure >> > > > > >> > > > > > > >>>>>>>>>>>> that you can use that plugin at scale? >> > > > > >> > > > > > > >>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>> -Marco >> > > > > >> > > > > > > >>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:04 PM Lin >> > Yuan < >> > > > > >> > > > > > > >>>>> [email protected]> >> > > > > >> > > > > > > >>>>>>>>> wrote: >> > > > > >> > > > > > > >>>>>>>>>>>>> Chai, >> > > > > >> > > > > > > >>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>> Awesome work. When do we expect this >> bot >> > > to >> > > > be >> > > > > >> > > > > > > >>> deployed? >> > > > > >> > > > > > > >>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>> Best, >> > > > > >> > > > > > > >>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>> Lin >> > > > > >> > > > > > > >>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>> On Thu, Mar 12, 2020 at 2:00 PM >> > Chaitanya >> > > > > Bapat >> > > > > >> < >> > > > > >> > > > > > > >>>>>>>>> [email protected] >> > > > > >> > > > > > > >>>>>>>>>>>>> wrote: >> > > > > >> > > > > > > >>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>>> Hello MXNet community, >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>>> I have built an MXNet Bot < >> > > > > >> > > > > > > >>>> https://github.com/mxnet-bot> >> > > > > >> > > > > > > >>>>> that >> > > > > >> > > > > > > >>>>>>>>>> allows >> > > > > >> > > > > > > >>>>>>>>>>> PR >> > > > > >> > > > > > > >>>>>>>>>>>>>> Authors, Committers and Jenkins >> Admins >> > to >> > > > > >> trigger >> > > > > >> > CI >> > > > > >> > > > > > > >>>>> manually. >> > > > > >> > > > > > > >>>>>>>>>>>>>> It handles 2 problems >> > > > > >> > > > > > > >>>>>>>>>>>>>> 1. Manual CI trigger instead of >> > existing >> > > > > >> automated >> > > > > >> > > CI >> > > > > >> > > > > > > >>>>> trigger >> > > > > >> > > > > > > >>>>>>>>>>>>>> 2. Gives permissions to PR Authors >> (in >> > > > > >> addition to >> > > > > >> > > > > > > >>>> MXNet >> > > > > >> > > > > > > >>>>>>>>> Committers >> > > > > >> > > > > > > >>>>>>>>>>> and >> > > > > >> > > > > > > >>>>>>>>>>>>>> Jenkins Admins) >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>>> Design Doc : >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > >> > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+CI+Bot >> > > > > >> > > > > > > >>>>>>>>>>>>>> I urge you all to attend the >> > > demonstration >> > > > > >> meeting >> > > > > >> > > > > > > >>> and >> > > > > >> > > > > > > >>>>> lend >> > > > > >> > > > > > > >>>>>>> your >> > > > > >> > > > > > > >>>>>>>>>> views >> > > > > >> > > > > > > >>>>>>>>>>>>> on >> > > > > >> > > > > > > >>>>>>>>>>>>>> the same. >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>>> Thank you, >> > > > > >> > > > > > > >>>>>>>>>>>>>> Chai >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>>> *Meeting Details*: >> > > > > >> > > > > > > >>>>>>>>>>>>>> ==============Conference Bridge >> > > > > >> > > > > > > >>>> Information============== >> > > > > >> > > > > > > >>>>>>>>>>>>>> You have been invited to an online >> > > meeting, >> > > > > >> > powered >> > > > > >> > > > > > > >>> by >> > > > > >> > > > > > > >>>>> Amazon >> > > > > >> > > > > > > >>>>>>>>> Chime. >> > > > > >> > > > > > > >>>>>>>>>>>>>> *Chime meeting ID*: *9272158344* >> > > > > >> > > > > > > >>>>>>>>>>>>>> Join via Chime clients (manually): >> > Select >> > > > > >> > 'Meetings >> > > > > >> > > > >> > > > > >> > > > > > > >>>>> Join a >> > > > > >> > > > > > > >>>>>>>>>> Meeting', >> > > > > >> > > > > > > >>>>>>>>>>>>> and >> > > > > >> > > > > > > >>>>>>>>>>>>>> enter 9272158344 >> > > > > >> > > > > > > >>>>>>>>>>>>>> Join via Chime clients (auto-call): >> If >> > > you >> > > > > >> invite >> > > > > >> > > > > > > >>>>> auto-call as >> > > > > >> > > > > > > >>>>>>>>>>> attendee, >> > > > > >> > > > > > > >>>>>>>>>>>>>> Chime will call you when the meeting >> > > > starts, >> > > > > >> > select >> > > > > >> > > > > > > >>>>> 'Answer' >> > > > > >> > > > > > > >>>>>>>>>>>>>> *Join via browser screen share*: >> > > > > >> > > > > > > >>>>> https://chime.aws/9272158344 >> > > > > >> > > > > > > >>>>>>>>>>>>>> *Join via phone* (US): >> > > > > >> > +1-929-432-4463,,,9272158344# >> > > > > >> > > > > > > >>>>>>>>>>>>>> *Join via phone (US toll-free)*: >> > > > > >> > > > > > > >>>>> +1-855-552-4463,,,9272158344# >> > > > > >> > > > > > > >>>>>>>>>>>>>> International dial-in: >> > > > > >> > > > > > > >>>> https://chime.aws/dialinnumbers/ >> > > > > >> > > > > > > >>>>>>>>>>>>>> In-room video system: Ext: 62000, >> > Meeting >> > > > > PIN: >> > > > > >> > > > > > > >>>>> 9272158344# >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>>> -- >> > > > > >> > > > > > > >>>>>>>>>>>>>> *Chaitanya Prakash Bapat* >> > > > > >> > > > > > > >>>>>>>>>>>>>> *+1 (973) 953-6299* >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>>>>>> [image: >> > > > > >> https://www.linkedin.com//in/chaibapat25] >> > > > > >> > > > > > > >>>>>>>>>>>>>> <https://github.com/ChaiBapchya >> > >[image: >> > > > > >> > > > > > > >>>>>>>>>>>>> https://www.facebook.com/chaibapat >> > > > > >> > > > > > > >>>>>>>>>>>>>> ] >> > > > > >> > > > > > > >>>>>>>>>>>>>> < >> https://www.facebook.com/chaibapchya >> > > > > >[image: >> > > > > >> > > > > > > >>>>>>>>>>>>>> https://twitter.com/ChaiBapchya] < >> > > > > >> > > > > > > >>>>>>>> https://twitter.com/ChaiBapchya >> > > > > >> > > > > > > >>>>>>>>>>>>>> [image: >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > https://www.linkedin.com//in/chaibapat25 >> > > ] >> > > > > >> > > > > > > >>>>>>>>>>>>>> < >> > > https://www.linkedin.com//in/chaibapchya/ >> > > > > >> > > > > >> > > > > > > >>>>>>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> -- >> > > > > >> > > > > > > >>>>>>>>>> *Chaitanya Prakash Bapat* >> > > > > >> > > > > > > >>>>>>>>>> *+1 (973) 953-6299* >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>>>>> [image: >> > > > https://www.linkedin.com//in/chaibapat25 >> > > > > ] >> > > > > >> > > > > > > >>>>>>>>>> <https://github.com/ChaiBapchya>[image: >> > > > > >> > > > > > > >>>>>>>>> https://www.facebook.com/chaibapat >> > > > > >> > > > > > > >>>>>>>>>> ] >> > > > > >> > > > > > > >>>>>>>>>> <https://www.facebook.com/chaibapchya >> > > >[image: >> > > > > >> > > > > > > >>>>>>>>>> https://twitter.com/ChaiBapchya] < >> > > > > >> > > > > > > >>>>> https://twitter.com/ChaiBapchya >> > > > > >> > > > > > > >>>>>>>>>> [image: >> > > > > >> > > > > > > >>>>>>>>>> >> https://www.linkedin.com//in/chaibapat25] >> > > > > >> > > > > > > >>>>>>>>>> < >> https://www.linkedin.com//in/chaibapchya/ >> > > >> > > > > >> > > > > > > >>>>>>>>>> >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>>>> -- >> > > > > >> > > > > > > >>>>>>> Sandeep Krishnamurthy >> > > > > >> > > > > > > >>>>>>> >> > > > > >> > > > > > > >>>>> >> > > > > >> > > > > > > >>>> >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> -- >> > > > > >> > > > > > > >>> *Chaitanya Prakash Bapat* >> > > > > >> > > > > > > >>> *+1 (973) 953-6299* >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >>> [image: >> https://www.linkedin.com//in/chaibapat25] >> > > > > >> > > > > > > >>> <https://github.com/ChaiBapchya>[image: >> > > > > >> > > > > > > https://www.facebook.com/chaibapat >> > > > > >> > > > > > > >>> ] >> > > > > >> > > > > > > >>> <https://www.facebook.com/chaibapchya>[image: >> > > > > >> > > > > > > >>> https://twitter.com/ChaiBapchya] < >> > > > > >> > > > https://twitter.com/ChaiBapchya >> > > > > >> > > > > > > >[image: >> > > > > >> > > > > > > >>> https://www.linkedin.com//in/chaibapat25] >> > > > > >> > > > > > > >>> <https://www.linkedin.com//in/chaibapchya/> >> > > > > >> > > > > > > >>> >> > > > > >> > > > > > > >> >> > > > > >> > > > > > > >> > > > > >> > > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > >> > > > > *Chaitanya Prakash Bapat* >> > > > > >> > > > > *+1 (973) 953-6299* >> > > > > >> > > > > >> > > > > >> > > > > [image: https://www.linkedin.com//in/chaibapat25] >> > > > > >> > > > > <https://github.com/ChaiBapchya>[image: >> > > > > >> > > > https://www.facebook.com/chaibapat >> > > > > >> > > > > ] >> > > > > >> > > > > <https://www.facebook.com/chaibapchya>[image: >> > > > > >> > > > > https://twitter.com/ChaiBapchya] < >> > > > > https://twitter.com/ChaiBapchya >> > > > > >> > > > >[image: >> > > > > >> > > > > https://www.linkedin.com//in/chaibapat25] >> > > > > >> > > > > <https://www.linkedin.com//in/chaibapchya/> >> > > > > >> > > > > >> > > > > >> > > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > -- >> > > > > >> > > *Chaitanya Prakash Bapat* >> > > > > >> > > *+1 (973) 953-6299* >> > > > > >> > > >> > > > > >> > > [image: https://www.linkedin.com//in/chaibapat25] >> > > > > >> > > <https://github.com/ChaiBapchya>[image: >> > > > > >> > https://www.facebook.com/chaibapat >> > > > > >> > > ] >> > > > > >> > > <https://www.facebook.com/chaibapchya>[image: >> > > > > >> > > https://twitter.com/ChaiBapchya] < >> > > https://twitter.com/ChaiBapchya >> > > > > >> > >[image: >> > > > > >> > > https://www.linkedin.com//in/chaibapat25] >> > > > > >> > > <https://www.linkedin.com//in/chaibapchya/> >> > > > > >> > > >> > > > > >> > >> > > > > >> >> > > > > >> >> > > > > >> -- >> > > > > >> *Chaitanya Prakash Bapat* >> > > > > >> *+1 (973) 953-6299* >> > > > > >> >> > > > > >> [image: https://www.linkedin.com//in/chaibapat25] >> > > > > >> <https://github.com/ChaiBapchya>[image: >> > > > > >> https://www.facebook.com/chaibapat] >> > > > > >> <https://www.facebook.com/chaibapchya>[image: >> > > > > >> https://twitter.com/ChaiBapchya] < >> https://twitter.com/ChaiBapchya >> > > > > >[image: >> > > > > >> https://www.linkedin.com//in/chaibapat25] >> > > > > >> <https://www.linkedin.com//in/chaibapchya/> >> > > > > >> >> > > > > > >> > > > > >> > > > >> > > > >> > > > -- >> > > > *Chaitanya Prakash Bapat* >> > > > *+1 (973) 953-6299* >> > > > >> > > > [image: https://www.linkedin.com//in/chaibapat25] >> > > > <https://github.com/ChaiBapchya>[image: >> > > https://www.facebook.com/chaibapat >> > > > ] >> > > > <https://www.facebook.com/chaibapchya>[image: >> > > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya >> > > >[image: >> > > > https://www.linkedin.com//in/chaibapat25] >> > > > <https://www.linkedin.com//in/chaibapchya/> >> > > > >> > > >> > >> > >> > -- >> > Sandeep Krishnamurthy >> > >> >> >> -- >> *Chaitanya Prakash Bapat* >> *+1 (973) 953-6299* >> >> [image: https://www.linkedin.com//in/chaibapat25] >> <https://github.com/ChaiBapchya>[image: >> https://www.facebook.com/chaibapat] >> <https://www.facebook.com/chaibapchya>[image: >> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image: >> https://www.linkedin.com//in/chaibapat25] >> <https://www.linkedin.com//in/chaibapchya/> >> > -- *Chaitanya Prakash Bapat* *+1 (973) 953-6299* [image: https://www.linkedin.com//in/chaibapat25] <https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat] <https://www.facebook.com/chaibapchya>[image: https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image: https://www.linkedin.com//in/chaibapat25] <https://www.linkedin.com//in/chaibapchya/>
