Great Job. Would be really helpful. Regards, Kshiteej
On Wed, 25 Mar, 2020, 7:29 AM Chaitanya Bapat, <chai.ba...@gmail.com> wrote: > Yes Denisa, you can use it on existing PRs as well. Sorry for miswording. > > Bot will comment instructions on how-to-use for every new PR. > You can invoke bot for all the PRs. > > > On Tue, 24 Mar 2020 at 18:00, Denisa Roberts <denisa.olte...@gmail.com> > wrote: > > > Hi- Only for new PRs? Can I use it for an existing PR to retrigger only > > specific tests? > > > > On Tue, Mar 24, 2020 at 8:27 PM Chaitanya Bapat <chai.ba...@gmail.com> > > wrote: > > > >> Hello MXNet community, > >> > >> Update: Bot has been deployed 🚀 on apache/incubator-mxnet. > >> > >> For every new PR, bot will comment with a help message (instructing what > >> command to comment) > >> It can trigger all jobs or specific jobs for users. > >> > >> Do use and if you find issues/suggestions do comment on this mail thread > >> or > >> the GitHub issue : > https://github.com/apache/incubator-mxnet/issues/17801 > >> > >> Thanks to Denis, Sandeep, Joe, Pedro, Marco and the design doc reviewers > >> for valuable feedback and guidance. > >> > >> Thank you, > >> Chai > >> > >> On Mon, 23 Mar 2020 at 13:08, sandeep krishnamurthy < > >> sandeep.krishn...@gmail.com> wrote: > >> > >> > Thank you Chaitanya and Marco for helping the MXNet community. > >> > > >> > On Mon, Mar 23, 2020 at 12:56 PM Marco de Abreu < > >> marco.g.ab...@gmail.com> > >> > wrote: > >> > > >> > > Sure, already done. > >> > > > >> > > -Marco > >> > > > >> > > On Mon, Mar 23, 2020 at 8:53 PM Chaitanya Bapat < > chai.ba...@gmail.com > >> > > >> > > wrote: > >> > > > >> > > > Hello, > >> > > > Update: Apache Infra Ticket for MXNet Bot > >> > > > Thanks once again, Marco for opening the ticket. But turns out, > >> Apache > >> > > > Infra folks closed it stating: "Security concerns around allowing > >> > unknown > >> > > > person to submit PR and run our hardware". Furthermore, it goes > onto > >> > > state > >> > > > that bot circumvents the dependence on Jenkins Admins which is > like > >> > > solving > >> > > > a problem that doesn't exist. > >> > > > > >> > > > I sense there is some confusion in the communication (maybe on my > >> > part). > >> > > It > >> > > > turns out the security concerns aren't actually correct. > >> > > > > >> > > > 1. Unknown person can submit a PR (before & after bot proposal), > and > >> > run > >> > > > our hardware (pre as well as post bot). > >> > > > 2. Code should be reviewed by somebody with an ICLA on file. This > >> > doesn't > >> > > > change either. Prior to merging a PR, code has to be approved by a > >> > > > committer just like before. > >> > > > Overall it looks like the job of the bot isn't clear to folks in > >> Apache > >> > > > Infra. Bot simply is a means for triggering CI (which could be > done > >> > > > manually by Log In to Jenkins -> PR -> Job -> Build) and doesn't > >> quite > >> > > > tweak with merging procedure. Yes, only addition is now unknown > >> person > >> > > (PR > >> > > > Author) can trigger CI with a message (but that was possible > anyway > >> by > >> > > > pushing a commit. Bot just prevents users from pushing empty > commits > >> > and > >> > > > building entire suite). > >> > > > > >> > > > As can be seen from last 10 open PRs as of Monday 23rd March, 12pm > >> PT > >> > > most > >> > > > PRs fail on 1/2 jobs. In such a scenario, the proposed MXNet bot > >> would > >> > > come > >> > > > in handy for just invoking CI on that specific build (instead of a > >> > > > non-committer PR Author to push empty commit : hurting on the > >> resource, > >> > > > time & cost considerations apart from undesirable dev experience) > >> > > > > >> > > > @Marco Since I am a non-committer, I guess these 2 clarifications > >> need > >> > to > >> > > > be conveyed to the Apache Infra by someone with Committer access. > >> > > > > >> > > > What do you think? > >> > > > > >> > > > Thanks, > >> > > > Chai > >> > > > > >> > > > On Sat, 21 Mar 2020 at 16:08, Marco de Abreu < > >> marco.g.ab...@gmail.com> > >> > > > wrote: > >> > > > > >> > > > > Hello, > >> > > > > > >> > > > > the ticket has been created: > >> > > > > https://issues.apache.org/jira/browse/INFRA-20005 > >> > > > > > >> > > > > Best regards, > >> > > > > Marco > >> > > > > > >> > > > > On Thu, Mar 19, 2020 at 11:49 PM Marco de Abreu < > >> > > marco.g.ab...@gmail.com > >> > > > > > >> > > > > wrote: > >> > > > > > >> > > > > > Sounds like a good plan! > >> > > > > > > >> > > > > > Please send me the URL (please make sure it's backed by DNS > and > >> not > >> > > > just > >> > > > > > the gateway URL) of the webhook handler, GitHub events you're > >> > > > interested > >> > > > > in > >> > > > > > and the shared secret in a private email to my personal email > >> > > address. > >> > > > I > >> > > > > > will then create the ticket with Apache infra. > >> > > > > > > >> > > > > > -Marco > >> > > > > > > >> > > > > > Chaitanya Bapat <chai.ba...@gmail.com> schrieb am Do., 19. > März > >> > > 2020, > >> > > > > > 23:07: > >> > > > > > > >> > > > > >> @Marco Alright, it makes total sense to test out the Bot > >> feature > >> > > > > alongside > >> > > > > >> auto-trigger as a transition. > >> > > > > >> > >> > > > > >> Path Forward: > >> > > > > >> 1. Setup MXNet Bot on apache/incubator-mxnet repo (GitHub > >> WebHook > >> > > and > >> > > > > >> Infra) > >> > > > > >> 2. We don't turn off automatic trigger of PR builds for now. > >> > > > > >> 3. Hopefully, bot is used by developers to trigger specific > >> jobs > >> > > > > >> 4. Later on (say around April 20), let's discuss the > >> possibility > >> > of > >> > > > > >> switching off auto-trigger (with appropriate data) if it > makes > >> > > sense. > >> > > > > >> Thanks Marco for volunteering to help enable the web hook on > >> > > > > >> apache/incubator-mxnet. Let me know if we can sync up on > Slack > >> > > channel > >> > > > > to > >> > > > > >> get the ball rolling. > >> > > > > >> > >> > > > > >> Thanks once again for the entire community to step in and > help > >> try > >> > > out > >> > > > > >> this > >> > > > > >> Bot. > >> > > > > >> Chai > >> > > > > >> > >> > > > > >> On Wed, 18 Mar 2020 at 17:07, Marco de Abreu < > >> > > marco.g.ab...@gmail.com > >> > > > > > >> > > > > >> wrote: > >> > > > > >> > >> > > > > >> > Hi, that's correct. But as stated previously, it's not an > >> option > >> > > to > >> > > > > >> remove > >> > > > > >> > the hook. For now, I'd like to see how the system behaves > >> while > >> > > it's > >> > > > > >> > optional. Later on, we can talk about revisiting this > >> decision. > >> > > But > >> > > > to > >> > > > > >> me > >> > > > > >> > it's not an option to deploy an entirely new system and > >> approach > >> > > > > without > >> > > > > >> > having a transition or even a timeframe in which we are > able > >> to > >> > > fall > >> > > > > >> back. > >> > > > > >> > > >> > > > > >> > I'm happy to support the deployment of the bot and add an > >> > > additional > >> > > > > >> > webhook to enable it's functionality to support selective > >> > > triggering > >> > > > > by > >> > > > > >> PR > >> > > > > >> > authors and committers, but I will not support the > disabling > >> of > >> > > > > >> automatic > >> > > > > >> > triggering of branches or PRs. > >> > > > > >> > > >> > > > > >> > -Marco > >> > > > > >> > > >> > > > > >> > Chaitanya Bapat <chai.ba...@gmail.com> schrieb am Mi., 18. > >> März > >> > > > 2020, > >> > > > > >> > 21:00: > >> > > > > >> > > >> > > > > >> > > Hey Marco, > >> > > > > >> > > > >> > > > > >> > > I thought currently every commit on PR and master > triggers > >> CI > >> > > > > >> > > because > >> > > > > >> > > a. github webhook points to Jenkins Server > >> > > > > >> > > b. GH Webhook events trigger builds on Jenkins for all > >> commits > >> > > to > >> > > > > any > >> > > > > >> > > branch in apache/incubator-mxnet > >> > > > > >> > > may it be master/PR/non-PR > >> > > > > >> > > Reason: > >> > > > > >> > > Because all the 3 types of branches are discovered by > >> Jenkins > >> > > > > (non-PR > >> > > > > >> > > (including master) and PR) > >> > > > > >> > > > >> > > > > >> > > Proposal: Remove GitHub WebHook to Jenkins and replace > >> with GH > >> > > > > >> Webhook to > >> > > > > >> > > Lambda > >> > > > > >> > > But after I remove the github webhook that points to > >> Jenkins : > >> > > > *N**o > >> > > > > >> > commit > >> > > > > >> > > will trigger Jenkins build by default* (as Jenkins wont > >> > receive > >> > > GH > >> > > > > >> > events) > >> > > > > >> > > Only those that Bot deems fit will be triggered (using > >> Jenkins > >> > > API > >> > > > > >> > invoked > >> > > > > >> > > by Lambda). > >> > > > > >> > > Hence its needed to handle that case of master merge. > >> > > > > >> > > Am I understanding this correctly? > >> > > > > >> > > > >> > > > > >> > > On Wed, 18 Mar 2020 at 04:23, Marco de Abreu < > >> > > > > marco.g.ab...@gmail.com > >> > > > > >> > > >> > > > > >> > > wrote: > >> > > > > >> > > > >> > > > > >> > > > Thanks Chai, sounds good to me. Could you elaborate a > >> bit on > >> > > the > >> > > > > >> point > >> > > > > >> > > > about triggering a CI run after the PR has been merged? > >> We > >> > > > already > >> > > > > >> to > >> > > > > >> > > that > >> > > > > >> > > > automatically for the master, so what's the benefit to > >> do it > >> > > > > twice? > >> > > > > >> > > > > >> > > > > >> > > > -Marco > >> > > > > >> > > > > >> > > > > >> > > > Chaitanya Bapat <chai.ba...@gmail.com> schrieb am Mi., > >> 18. > >> > > März > >> > > > > >> 2020, > >> > > > > >> > > > 09:30: > >> > > > > >> > > > > >> > > > > >> > > > > Update: > >> > > > > >> > > > > > >> > > > > >> > > > > > we can ensure that all CI runs ran on the commit > >> that > >> > > will > >> > > > be > >> > > > > >> > merged > >> > > > > >> > > > > @Sam Skalicky <samskali...@gmail.com> Branch > >> Protection > >> > is > >> > > > > added > >> > > > > >> to > >> > > > > >> > > > public > >> > > > > >> > > > > MXNet repo. It ensures that for every PR to be > merged, > >> the > >> > > CI > >> > > > > >> passes. > >> > > > > >> > > All > >> > > > > >> > > > > the jobs selected "required" jobs will have to be > green > >> > for > >> > > > the > >> > > > > >> PR to > >> > > > > >> > > be > >> > > > > >> > > > > merged. Ofcourse, users with "Adminstrator" access > can > >> > merge > >> > > > > >> without > >> > > > > >> > it > >> > > > > >> > > > but > >> > > > > >> > > > > that's just a backdoor. It is the case now and will > >> > continue > >> > > > to > >> > > > > be > >> > > > > >> > the > >> > > > > >> > > > case > >> > > > > >> > > > > with the inclusion of Bot. > >> > > > > >> > > > > > >> > > > > >> > > > > > easily verify that the CI has executed all runs on > >> the > >> > > > commit > >> > > > > >> that > >> > > > > >> > > will > >> > > > > >> > > > > be merged > >> > > > > >> > > > > GitHub UI shows all the jobs and the status > >> corresponding > >> > to > >> > > > it > >> > > > > on > >> > > > > >> > > every > >> > > > > >> > > > > commit. That should suffice. For the merged commits, > >> Repo > >> > -> > >> > > > > >> Commits > >> > > > > >> > -> > >> > > > > >> > > > > Commit ID (Status) can be tracked currently (only way > >> > that I > >> > > > > know > >> > > > > >> > of). > >> > > > > >> > > > > Moreover, it is beyond the scope of this project (and > >> > > possibly > >> > > > > >> out of > >> > > > > >> > > our > >> > > > > >> > > > > control since this is purely GitHub UI specific > >> use-case). > >> > > > > >> > > > > > >> > > > > >> > > > > Thanks @przemyslaw for supporting the opt-in. > >> > > > > >> > > > > > >> > > > > >> > > > > Thanks everyone in the community for sharing > concerns, > >> > > voicing > >> > > > > >> your > >> > > > > >> > > > opinion > >> > > > > >> > > > > and participating in the discussion. > >> > > > > >> > > > > Thanks to those who attended the demo last Friday. > >> > > > > >> > > > > > >> > > > > >> > > > > Action items from that discussion > >> > > > > >> > > > > 1. Handle master merge builds [Done] > >> > > > > >> > > > > Bot runs entire CI suite after the PR is merged and > >> > comments > >> > > > on > >> > > > > >> the > >> > > > > >> > PR > >> > > > > >> > > > > about the same. > >> > > > > >> > > > > Design decision : > >> > > > > >> > > > > MXNet Bot comment about master merge build on the > >> *merge > >> > > > commit > >> > > > > vs > >> > > > > >> > PR*. > >> > > > > >> > > > > After the PR is merged, Bot runs entire CI and > comments > >> > the > >> > > > > >> result of > >> > > > > >> > > CI > >> > > > > >> > > > > trigger on the PR (because it is easy to track on a > PR > >> > > rather > >> > > > > than > >> > > > > >> > > > > commenting inside the merge commit) > >> > > > > >> > > > > > >> > > > > >> > > > > 2. Idempotent condition > >> > > > > >> > > > > In case of already running build, if an attempt is > >> made to > >> > > > > >> retrigger > >> > > > > >> > > the > >> > > > > >> > > > > job then what should be the response > >> > > > > >> > > > > a. Not to re-trigger, let the ongoing build continue > >> till > >> > > > > >> completion > >> > > > > >> > > > > b. End the ongoing build and re-trigger > >> > > > > >> > > > > c. Let the ongoing build continue, re-trigger new > build > >> > > > > >> > > > > > >> > > > > >> > > > > From resource saving point of view, *c* looks costly > >> and a > >> > > can > >> > > > > be > >> > > > > >> > > > > avoided/optimized by B. > >> > > > > >> > > > > In case when a re-trigger was started "erroneously" > >> then > >> > > > killing > >> > > > > >> > > ongoing > >> > > > > >> > > > > build and re-trigger is a waste. > >> > > > > >> > > > > In case when ongoing build failed in one sub-part, > then > >> > > > > >> re-triggering > >> > > > > >> > > is > >> > > > > >> > > > > justified. > >> > > > > >> > > > > Erroneous re-triggers would be less often while > >> conscious > >> > > > > >> re-triggers > >> > > > > >> > > to > >> > > > > >> > > > > suppress failure is more common use-case. It looks > >> like a > >> > > safe > >> > > > > >> > > assumption > >> > > > > >> > > > > to make given the trade-off. > >> > > > > >> > > > > [Open to debate] > >> > > > > >> > > > > > >> > > > > >> > > > > 3. Add security consideration [Use of secret manager, > >> but > >> > > > > without > >> > > > > >> > > > > auto-rotation due to Jenkins manual config > requirement] > >> > > [Done] > >> > > > > >> > > > > 4. New PR Instruction message by the Bot [Done] > >> > > > > >> > > > > Thanks to the suggestion of Leonard, supported by > >> others. > >> > > I've > >> > > > > now > >> > > > > >> > > added > >> > > > > >> > > > > the feature where the Bot comments a help message. > [For > >> > > > > reference > >> > > > > >> - > >> > > > > >> > > > > > https://github.com/ChaiBapchya/incubator-mxnet/pull/52 > >> ] > >> > > > > >> > > > > > >> > > > > >> > > > > Barring the opt-in vs opt-out debate & idempotency, > >> > > consensus > >> > > > > was > >> > > > > >> > > quickly > >> > > > > >> > > > > reached for the rest. > >> > > > > >> > > > > > >> > > > > >> > > > > In the coming days, I hope to roll-out this feature > >> into > >> > > Prod > >> > > > > >> (public > >> > > > > >> > > > > MXNet) for all devs to use. > >> > > > > >> > > > > > >> > > > > >> > > > > Thanks, > >> > > > > >> > > > > Chai > >> > > > > >> > > > > > >> > > > > >> > > > > On Mon, 16 Mar 2020 at 11:57, Marco de Abreu < > >> > > > > >> > marco.g.ab...@gmail.com> > >> > > > > >> > > > > wrote: > >> > > > > >> > > > > > >> > > > > >> > > > > > Well that's generally a problem with a deferred CI > >> > > approach > >> > > > > (CI > >> > > > > >> is > >> > > > > >> > > run > >> > > > > >> > > > at > >> > > > > >> > > > > > commit and not at merge time). This can either be > >> solved > >> > > > > through > >> > > > > >> > the > >> > > > > >> > > > > other > >> > > > > >> > > > > > proposal that's currently on dev@, by having a bot > >> > which > >> > > > does > >> > > > > >> > merges > >> > > > > >> > > > by > >> > > > > >> > > > > > having a global lock and a merge queue or by > >> accepting > >> > the > >> > > > > >> issue. > >> > > > > >> > > > Reality > >> > > > > >> > > > > > right now is that we're running that model where > two > >> PRs > >> > > > which > >> > > > > >> are > >> > > > > >> > > > merged > >> > > > > >> > > > > > in parallel might break one another. One thing to > >> > consider > >> > > > > >> though > >> > > > > >> > is > >> > > > > >> > > > that > >> > > > > >> > > > > > this breakage would have to be introduced in two > >> > separate > >> > > > > parts > >> > > > > >> > since > >> > > > > >> > > > > > otherwise there'd be merge conflicts. I think we > had > >> > that > >> > > > > >> situation > >> > > > > >> > > > twice > >> > > > > >> > > > > > so far and the result was a quick revert, so I'd > say > >> > that > >> > > > > it's a > >> > > > > >> > > > problem > >> > > > > >> > > > > > that can happily be accepted. All other solutions > >> > > basically > >> > > > > >> require > >> > > > > >> > > > some > >> > > > > >> > > > > > form of single-threaded and globally locked > solution > >> > which > >> > > > > >> limits > >> > > > > >> > us > >> > > > > >> > > in > >> > > > > >> > > > > > scalability. I'd recommend to just accept that risk > >> and > >> > > > revert > >> > > > > >> a PR > >> > > > > >> > > in > >> > > > > >> > > > > case > >> > > > > >> > > > > > it actually had a conflict. > >> > > > > >> > > > > > > >> > > > > >> > > > > > -Marco > >> > > > > >> > > > > > > >> > > > > >> > > > > > On Mon, Mar 16, 2020 at 6:29 PM Skalicky, Sam > >> > > > > >> > > > <sska...@amazon.com.invalid > >> > > > > >> > > > > > > >> > > > > >> > > > > > wrote: > >> > > > > >> > > > > > > >> > > > > >> > > > > > > We probably need some way to track which CI runs > >> ran > >> > for > >> > > > > which > >> > > > > >> > > commit > >> > > > > >> > > > > > too, > >> > > > > >> > > > > > > that way we can ensure that all CI runs ran on > the > >> > > commit > >> > > > > that > >> > > > > >> > will > >> > > > > >> > > > be > >> > > > > >> > > > > > > merged. Maybe the bot can comment with the > commit > >> > hash > >> > > > when > >> > > > > >> > users > >> > > > > >> > > > > > command > >> > > > > >> > > > > > > it to do something. Although since users can > >> trigger > >> > > > > >> individual > >> > > > > >> > CI > >> > > > > >> > > > runs > >> > > > > >> > > > > > its > >> > > > > >> > > > > > > possible to have some commits run some CI runs > but > >> not > >> > > > > >> others. We > >> > > > > >> > > > need > >> > > > > >> > > > > > some > >> > > > > >> > > > > > > way to easily verify that the CI has executed all > >> runs > >> > > on > >> > > > > the > >> > > > > >> > > commit > >> > > > > >> > > > > that > >> > > > > >> > > > > > > will be merged. > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > Sam > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > > On Mar 13, 2020, at 8:28 PM, Przemysław Trędak > < > >> > > > > >> > > ptre...@apache.org > >> > > > > >> > > > > > >> > > > > >> > > > > > > wrote: > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > CAUTION: This email originated from outside of > >> the > >> > > > > >> > organization. > >> > > > > >> > > Do > >> > > > > >> > > > > not > >> > > > > >> > > > > > > click links or open attachments unless you can > >> confirm > >> > > the > >> > > > > >> sender > >> > > > > >> > > and > >> > > > > >> > > > > > know > >> > > > > >> > > > > > > the content is safe. > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > I personally like the idea of opt-in more than > >> > > opt-out: > >> > > > > >> > > > > > > > - ultimately PR author wants the PR to be > merged > >> so > >> > > they > >> > > > > (or > >> > > > > >> > > > > committer > >> > > > > >> > > > > > > reviewing the PR) will trigger the CI > >> > > > > >> > > > > > > > - if it is easy to trigger the PR via the bot > >> > command > >> > > > then > >> > > > > >> the > >> > > > > >> > > > amount > >> > > > > >> > > > > > of > >> > > > > >> > > > > > > work per PR should be less than with opt-out > (since > >> > most > >> > > > of > >> > > > > >> the > >> > > > > >> > > > commits > >> > > > > >> > > > > > > should then be marked as [skip ci] or something > >> > similar > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > +1 to the bot making a comment on each new PR > >> with > >> > its > >> > > > > >> commands > >> > > > > >> > > > (and > >> > > > > >> > > > > > > also explaining, or at least giving links to the > >> > general > >> > > > PR > >> > > > > >> > process > >> > > > > >> > > > so > >> > > > > >> > > > > > new > >> > > > > >> > > > > > > PR authors are not lost). Maybe we could make the > >> bot > >> > > > > >> recognize > >> > > > > >> > if > >> > > > > >> > > > the > >> > > > > >> > > > > PR > >> > > > > >> > > > > > > author is new or existing contributor and offer > >> advice > >> > > > based > >> > > > > >> on > >> > > > > >> > > that? > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > Thanks > >> > > > > >> > > > > > > > Przemek > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > On 2020/03/13 22:06:58, Marco de Abreu < > >> > > > > >> > marco.g.ab...@gmail.com> > >> > > > > >> > > > > > wrote: > >> > > > > >> > > > > > > >> Hi, > >> > > > > >> > > > > > > >> > >> > > > > >> > > > > > > >> since it's no longer necessary to push a new > >> commit > >> > > to > >> > > > > >> trigger > >> > > > > >> > > CI, > >> > > > > >> > > > > it > >> > > > > >> > > > > > > will > >> > > > > >> > > > > > > >> already reduce the costs. But to me, requiring > >> an > >> > > > action > >> > > > > to > >> > > > > >> > > enable > >> > > > > >> > > > > CI > >> > > > > >> > > > > > > after > >> > > > > >> > > > > > > >> a PR has been created initially, is a no go. > >> User > >> > can > >> > > > opt > >> > > > > >> out > >> > > > > >> > of > >> > > > > >> > > > CI, > >> > > > > >> > > > > > but > >> > > > > >> > > > > > > >> the default has to be CI being triggered > >> > > automatically > >> > > > > for > >> > > > > >> > every > >> > > > > >> > > > > > commit > >> > > > > >> > > > > > > >> unless specifically disabled by a participant. > >> I'm > >> > > also > >> > > > > >> fine > >> > > > > >> > > with > >> > > > > >> > > > > > > >> triggering certain additional jobs (think > about > >> > > > running a > >> > > > > >> > > nightly > >> > > > > >> > > > > job > >> > > > > >> > > > > > > upon > >> > > > > >> > > > > > > >> request for a PR) to require a manual step, > but > >> the > >> > > PR > >> > > > > >> > > validation > >> > > > > >> > > > > > > pipelines > >> > > > > >> > > > > > > >> have to run automatically. Every check that is > >> > marked > >> > > > as > >> > > > > >> > > > "Required" > >> > > > > >> > > > > in > >> > > > > >> > > > > > > >> GitHub has to be automatically kicked off. > >> > > > > >> > > > > > > >> > >> > > > > >> > > > > > > >> -Marco > >> > > > > >> > > > > > > >> > >> > > > > >> > > > > > > >> On Fri, Mar 13, 2020 at 9:50 PM Chaitanya > Bapat > >> < > >> > > > > >> > > > > chai.ba...@gmail.com > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > >> wrote: > >> > > > > >> > > > > > > >> > >> > > > > >> > > > > > > >>> Firstly, > >> > > > > >> > > > > > > >>> Sorry I missed out on attaching the mail > thread > >> > that > >> > > > was > >> > > > > >> sent > >> > > > > >> > > on > >> > > > > >> > > > > 12th > >> > > > > >> > > > > > > >>> February for notifying the community of the > >> > upcoming > >> > > > > >> changes > >> > > > > >> > to > >> > > > > >> > > > the > >> > > > > >> > > > > > > MXNet > >> > > > > >> > > > > > > >>> CI > >> > > > > >> > > > > > > >>> For reference : > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > > >> > > > > >> > > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > > >> > > >> > > > > >> > >> > > > > > >> > > > > >> > > > >> > > >> > https://lists.apache.org/thread.html/r09a6ab2803a996fc80e00fe39ed312fa4865e8805e08df847f1addad%40%3Cdev.mxnet.apache.org%3E > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>> Now to the questions, > >> > > > > >> > > > > > > >>>> Is it possible for re-triggering a single > job > >> to > >> > be > >> > > > > >> abused? > >> > > > > >> > > > > > > >>> @Tao In the case when a user re-triggers a > >> single > >> > > job > >> > > > > >> > multiple > >> > > > > >> > > > > times, > >> > > > > >> > > > > > > that > >> > > > > >> > > > > > > >>> will be visible in the PR conversation > thread. > >> A > >> > > > > >> committer, > >> > > > > >> > > even > >> > > > > >> > > > > > after > >> > > > > >> > > > > > > he > >> > > > > >> > > > > > > >>> has approved the PR before, generally takes a > >> look > >> > > at > >> > > > > the > >> > > > > >> > final > >> > > > > >> > > > > state > >> > > > > >> > > > > > > of > >> > > > > >> > > > > > > >>> the PR before merging. Would it be fair to > >> assume > >> > > the > >> > > > > >> > committer > >> > > > > >> > > > > could > >> > > > > >> > > > > > > take > >> > > > > >> > > > > > > >>> the multiple re-trigger of a single job into > >> > account > >> > > > > >> before > >> > > > > >> > > > > merging? > >> > > > > >> > > > > > > The > >> > > > > >> > > > > > > >>> committer then has the option to invoke > >> > `@mxnet-bot > >> > > > run > >> > > > > ci > >> > > > > >> > > [all] > >> > > > > >> > > > ` > >> > > > > >> > > > > to > >> > > > > >> > > > > > > >>> trigger the entire build pipeline one last to > >> > > counter > >> > > > > the > >> > > > > >> > > abuse. > >> > > > > >> > > > > This > >> > > > > >> > > > > > > is > >> > > > > >> > > > > > > >>> aligned with what @Leonard said. > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>> @Sandeep Thanks a lot for collecting and > >> sharing > >> > > > > valuable > >> > > > > >> > data. > >> > > > > >> > > > I'd > >> > > > > >> > > > > > > concur > >> > > > > >> > > > > > > >>> with the opinion that given the existing > things > >> > > > > >> committers & > >> > > > > >> > PR > >> > > > > >> > > > > > Authors > >> > > > > >> > > > > > > >>> take care of, invoking CI shouldn't be that > >> big of > >> > > an > >> > > > > >> > > additional > >> > > > > >> > > > > > > burden. > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>> @Marco With the opt-out, the onus remains on > >> the > >> > PR > >> > > > > >> Author. > >> > > > > >> > It > >> > > > > >> > > > > > doesn't > >> > > > > >> > > > > > > help > >> > > > > >> > > > > > > >>> reduce the resource usage. Hence, it was > >> suggested > >> > > to > >> > > > > >> switch > >> > > > > >> > to > >> > > > > >> > > > > > > >>> opt-in. @Leo's suggestion for proactive > >> commenting > >> > > on > >> > > > > the > >> > > > > >> > part > >> > > > > >> > > of > >> > > > > >> > > > > bot > >> > > > > >> > > > > > > makes > >> > > > > >> > > > > > > >>> sense and is doable. > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>> Default : opt-out and User initiated opt-in > >> (with > >> > > > > >> addressing > >> > > > > >> > > > Leo's > >> > > > > >> > > > > > fix > >> > > > > >> > > > > > > for > >> > > > > >> > > > > > > >>> the usability issue you correctly pointed > out ) > >> > > > > >> > > > > > > >>> @Marco How does this sound to you? > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>> Again, thank you all for chiming in and > voicing > >> > your > >> > > > > >> opinion. > >> > > > > >> > > > > > > Appreciate > >> > > > > >> > > > > > > >>> it. > >> > > > > >> > > > > > > >>> We can take ahead these discussions in > today's > >> > demo > >> > > > > >> meeting. > >> > > > > >> > > > > [Design > >> > > > > >> > > > > > > Doc > >> > > > > >> > > > > > > >>> < > >> > > > > >> > > > >> > https://cwiki.apache.org/confluence/display/MXNET/MXNet+CI+Bot > >> > > > > >> > > > >] > >> > > > > >> > > > > > > [Demo > >> > > > > >> > > > > > > >>> Video < > >> > https://www.youtube.com/watch?v=gfOGwZId8aU > >> > > >] > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>> Thanks, > >> > > > > >> > > > > > > >>> Chai > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>> On Fri, 13 Mar 2020 at 12:34, Marco de Abreu > < > >> > > > > >> > > > > > marco.g.ab...@gmail.com> > >> > > > > >> > > > > > > >>> wrote: > >> > > > > >> > > > > > > >>> > >> > > > > >> > > > > > > >>>> I'd recommend that the bot makes an initial > >> > comment > >> > > > > when > >> > > > > >> a > >> > > > > >> > PR > >> > > > > >> > > > gets > >> > > > > >> > > > > > > opened > >> > > > > >> > > > > > > >>>> and informs the users of its commands. It > then > >> > > tells > >> > > > > the > >> > > > > >> > user > >> > > > > >> > > > the > >> > > > > >> > > > > > > commend > >> > > > > >> > > > > > > >>>> to opt out of CI. > >> > > > > >> > > > > > > >>>> > >> > > > > >> > > > > > > >>>> -Marco > >> > > > > >> > > > > > > >>>> > >> > > > > >> > > > > > > >>>> Lausen, Leonard <lau...@amazon.com.invalid> > >> > > schrieb > >> > > > am > >> > > > > >> Fr., > >> > > > > >> > > 13. > >> > > > > >> > > > > > März > >> > > > > >> > > > > > > >>> 2020, > >> > > > > >> > > > > > > >>>> 20:27: > >> > > > > >> > > > > > > >>>> > >> > > > > >> > > > > > > >>>>> On opt-out: People may be unaware of > opt-out > >> > would > >> > > > not > >> > > > > >> use > >> > > > > >> > > it. > >> > > > > >> > > > > > There > >> > > > > >> > > > > > > is > >> > > > > >> > > > > > > >>>> no > >> > > > > >> > > > > > > >>>>> incentive to use opt-out, as the PR author > >> > doesn't > >> > > > pay > >> > > > > >> any > >> > > > > >> > > > money > >> > > > > >> > > > > > for > >> > > > > >> > > > > > > CI > >> > > > > >> > > > > > > >>>>> run. > >> > > > > >> > > > > > > >>>>> > >> > > > > >> > > > > > > >>>>> I agree with Marco though that opt-in alone > >> may > >> > > > cause > >> > > > > >> > > usability > >> > > > > >> > > > > > > issues, > >> > > > > >> > > > > > > >>>> as > >> > > > > >> > > > > > > >>>>> contributors may not be aware of how to > >> trigger > >> > > the > >> > > > > CI. > >> > > > > >> > > > > > > >>>>> One solution is that the bot proactively > >> > comments > >> > > on > >> > > > > >> the PR > >> > > > > >> > > and > >> > > > > >> > > > > > > reminds > >> > > > > >> > > > > > > >>>> the > >> > > > > >> > > > > > > >>>>> author to trigger running CI once the > author > >> > deems > >> > > > the > >> > > > > >> PR > >> > > > > >> > > > ready. > >> > > > > >> > > > > > > >>>>> > >> > > > > >> > > > > > > >>>>> But even if we choose opt-out, the bot will > >> > still > >> > > > add > >> > > > > a > >> > > > > >> lot > >> > > > > >> > > of > >> > > > > >> > > > > > value, > >> > > > > >> > > > > > > >>> as > >> > > > > >> > > > > > > >>>> PR > >> > > > > >> > > > > > > >>>>> authors can retrigger single jobs that have > >> > failed > >> > > > due > >> > > > > >> to > >> > > > > >> > > > > > flakiness. > >> > > > > >> > > > > > > >>>>> > >> > > > > >> > > > > > > >>>>>> Is it possible for re-triggering a single > >> job > >> > to > >> > > be > >> > > > > >> > abused? > >> > > > > >> > > > For > >> > > > > >> > > > > > > >>>> example, > >> > > > > >> > > > > > > >>>>>> the author spends two days re-triggering a > >> > flaky > >> > > > job > >> > > > > to > >> > > > > >> > make > >> > > > > >> > > > it > >> > > > > >> > > > > > > pass. > >> > > > > >> > > > > > > >>>>> > >> > > > > >> > > > > > > >>>>> Yes, this is possible. I suggest the > >> committer > >> > who > >> > > > > >> likes to > >> > > > > >> > > > > merge a > >> > > > > >> > > > > > > PR > >> > > > > >> > > > > > > >>>>> needs to > >> > > > > >> > > > > > > >>>>> make a good judgement here if a PR is > abusing > >> > the > >> > > > > >> feature, > >> > > > > >> > > and > >> > > > > >> > > > if > >> > > > > >> > > > > > so, > >> > > > > >> > > > > > > >>>>> retrigger > >> > > > > >> > > > > > > >>>>> all CI runs. > >> > > > > >> > > > > > > >>>>> > >> > > > > >> > > > > > > >>>>> Best regards > >> > > > > >> > > > > > > >>>>> Leonard > >> > > > > >> > > > > > > >>>>> > >> > > > > >> > > > > > > >>>>> On Fri, 2020-03-13 at 08:07 +0100, Marco de > >> > Abreu > >> > > > > wrote: > >> > > > > >> > > > > > > >>>>>> Thanks for the data Sandeep. In these > cases > >> it > >> > > > sounds > >> > > > > >> like > >> > > > > >> > > it > >> > > > > >> > > > > > would > >> > > > > >> > > > > > > >>>> have > >> > > > > >> > > > > > > >>>>>> rather been better when people explicitly > >> > turned > >> > > > off > >> > > > > >> CI in > >> > > > > >> > > > that > >> > > > > >> > > > > > > case. > >> > > > > >> > > > > > > >>>>>> What's the argument against an opt-out > >> instead > >> > of > >> > > > an > >> > > > > >> > opt-in? > >> > > > > >> > > > > > > >>>>>> > >> > > > > >> > > > > > > >>>>>> My intention is that I consider it quite > >> > > cumbersome > >> > > > > to > >> > > > > >> > make > >> > > > > >> > > > it a > >> > > > > >> > > > > > > >>>>> *required* > >> > > > > >> > > > > > > >>>>>> step to always trigger CI manually, even > if > >> > just > >> > > > > >> > submitting > >> > > > > >> > > a > >> > > > > >> > > > > > small > >> > > > > >> > > > > > > >>> PR. > >> > > > > >> > > > > > > >>>>> I'd > >> > > > > >> > > > > > > >>>>>> rather see people explicitly turning off > CI > >> if > >> > > they > >> > > > > >> > wouldn't > >> > > > > >> > > > > like > >> > > > > >> > > > > > to > >> > > > > >> > > > > > > >>>> use > >> > > > > >> > > > > > > >>>>> it > >> > > > > >> > > > > > > >>>>>> - and there's also the "draft" stage for a > >> PR > >> > > which > >> > > > > >> some > >> > > > > >> > > > > > > contributors > >> > > > > >> > > > > > > >>>> are > >> > > > > >> > > > > > > >>>>>> using. > >> > > > > >> > > > > > > >>>>>> > >> > > > > >> > > > > > > >>>>>> With regards to WIP and do not review: I > >> think > >> > > > these > >> > > > > >> are > >> > > > > >> > use > >> > > > > >> > > > > cases > >> > > > > >> > > > > > > >>>> where > >> > > > > >> > > > > > > >>>>>> you want CI feedback, as otherwise you > >> wouldn't > >> > > > have > >> > > > > >> > opened > >> > > > > >> > > > the > >> > > > > >> > > > > > PR. > >> > > > > >> > > > > > > >>> If > >> > > > > >> > > > > > > >>>>> you > >> > > > > >> > > > > > > >>>>>> don't want human feedback and neither > >> machine > >> > > > > feedback, > >> > > > > >> > why > >> > > > > >> > > > open > >> > > > > >> > > > > > the > >> > > > > >> > > > > > > >>> PR > >> > > > > >> > > > > > > >>>>> at > >> > > > > >> > > > > > > >>>>>> all? > >> > > > > >> > > > > > > >>>>>> > >> > > > > >> > > > > > > >>>>>> -Marco > >> > > > > >> > > > > > > >>>>>> > >> > > > > >> > > > > > > >>>>>> > >> > > > > >> > > > > > > >>>>>> sandeep krishnamurthy < > >> > > sandeep.krishn...@gmail.com > >> > > > > > >> > > > > >> > schrieb > >> > > > > >> > > > am > >> > > > > >> > > > > > Fr., > >> > > > > >> > > > > > > >>>> 13. > >> > > > > >> > > > > > > >>>>>> März 2020, 05:24: > >> > > > > >> > > > > > > >>>>>> > >> > > > > >> > > > > > > >>>>>>> I tried to gather some data for us to > >> discuss > >> > > this > >> > > > > >> topic > >> > > > > >> > in > >> > > > > >> > > > > this > >> > > > > >> > > > > > > >>>>> thread. I > >> > > > > >> > > > > > > >>>>>>> tried to count number of un-necessary > >> builds > >> > by > >> > > > > >> looking > >> > > > > >> > at > >> > > > > >> > > > most > >> > > > > >> > > > > > > >>>> recent > >> > > > > >> > > > > > > >>>>> (as > >> > > > > >> > > > > > > >>>>>>> of 12, March 9 PM PST) 50 PRs merged to > >> master > >> > > and > >> > > > > 50 > >> > > > > >> > PRs. > >> > > > > >> > > > > > > >>>>>>> Identifying un-necessary builds is bit > >> > > > subjective. I > >> > > > > >> > tried > >> > > > > >> > > to > >> > > > > >> > > > > be > >> > > > > >> > > > > > > >>> more > >> > > > > >> > > > > > > >>>>>>> conservative where I didn't count a build > >> as > >> > > > > >> un-necessary > >> > > > > >> > > if > >> > > > > >> > > > I > >> > > > > >> > > > > > was > >> > > > > >> > > > > > > >>> in > >> > > > > >> > > > > > > >>>>>>> doubt. Hence, I was not able to automate, > >> but > >> > I > >> > > > made > >> > > > > >> an > >> > > > > >> > > > effort > >> > > > > >> > > > > to > >> > > > > >> > > > > > > >>> go > >> > > > > >> > > > > > > >>>>>>> through PRs manually and use below > >> criteria to > >> > > > > >> identify > >> > > > > >> > > > > > > >>> un-necessary > >> > > > > >> > > > > > > >>>>>>> commits triggering the builds. > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> 1. Explicitly marked as WIP / do not > >> review > >> > > PR > >> > > > > >> > > > > > > >>>>>>> 2. Incremental WIP commit and finally > >> > > > commenting a > >> > > > > >> > commit > >> > > > > >> > > > > > > >>> “trigger > >> > > > > >> > > > > > > >>>>> CI” > >> > > > > >> > > > > > > >>>>>>> 3. Multiple commits to address all > >> comments > >> > > from > >> > > > > >> single > >> > > > > >> > > > > review. > >> > > > > >> > > > > > > >>>>> This is > >> > > > > >> > > > > > > >>>>>>> assuming we see a comment, address > them, > >> > > commit, > >> > > > > >> next > >> > > > > >> > the > >> > > > > >> > > > > > > >>>> following > >> > > > > >> > > > > > > >>>>>>> comment > >> > > > > >> > > > > > > >>>>>>> 4. Sequence of documentation only > changes > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> I found there were around 42 avoidable > >> builds > >> > > from > >> > > > > >> most > >> > > > > >> > > > recent > >> > > > > >> > > > > 50 > >> > > > > >> > > > > > > >>>>> merged > >> > > > > >> > > > > > > >>>>>>> PRs and around 86 builds from recent 50 > >> open > >> > > PRs. > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> I synced up with other contributors (Joe > >> > Evans, > >> > > > > Chai) > >> > > > > >> > from > >> > > > > >> > > > > Amazon > >> > > > > >> > > > > > > >>> who > >> > > > > >> > > > > > > >>>>> is > >> > > > > >> > > > > > > >>>>>>> contributing to MXNet CI system. I was > told > >> > that > >> > > > on > >> > > > > an > >> > > > > >> > > > average > >> > > > > >> > > > > it > >> > > > > >> > > > > > > >>>> costs > >> > > > > >> > > > > > > >>>>>>> around $84 per build and on an average 6 > >> > commits > >> > > > per > >> > > > > >> > merged > >> > > > > >> > > > PR > >> > > > > >> > > > > > (for > >> > > > > >> > > > > > > >>>>> year > >> > > > > >> > > > > > > >>>>>>> 2019). Going by that, it is approximately > >> 1/6 > >> > > > builds > >> > > > > >> are > >> > > > > >> > > > > > avoidable. > >> > > > > >> > > > > > > >>>>> [100 / > >> > > > > >> > > > > > > >>>>>>> 300 + 300 ] > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> Usability should be top most priority. > But, > >> > > since > >> > > > > >> either > >> > > > > >> > a > >> > > > > >> > > > > > reviewer > >> > > > > >> > > > > > > >>>> or > >> > > > > >> > > > > > > >>>>> pr > >> > > > > >> > > > > > > >>>>>>> author can trigger the bot, is it really > a > >> > > hurdle > >> > > > > for > >> > > > > >> pr > >> > > > > >> > > > author > >> > > > > >> > > > > > or > >> > > > > >> > > > > > > >>>>> reviewer > >> > > > > >> > > > > > > >>>>>>> to call a bot to trigger CI? Given that > PR > >> > > author > >> > > > > and > >> > > > > >> > > > reviewer > >> > > > > >> > > > > is > >> > > > > >> > > > > > > >>>>> already > >> > > > > >> > > > > > > >>>>>>> actively commenting various details such > >> as - > >> > PR > >> > > > > >> > > description, > >> > > > > >> > > > > > > >>> review > >> > > > > >> > > > > > > >>>>>>> comments and responses, adding labels > etc. > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> Me too curious to know the behavior for > >> Tao's > >> > > > above > >> > > > > >> use > >> > > > > >> > > case. > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> Best, > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> Sandeep > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>> On Thu, Mar 12, 2020 at 7:18 PM Tao Lv < > >> > > > > >> > mutou...@gmail.com > >> > > > > >> > > > > >> > > > > >> > > > > > wrote: > >> > > > > >> > > > > > > >>>>>>> > >> > > > > >> > > > > > > >>>>>>>> Is it possible for re-triggering a > single > >> job > >> > > to > >> > > > be > >> > > > > >> > > abused? > >> > > > > >> > > > > For > >> > > > > >> > > > > > > >>>>> example, > >> > > > > >> > > > > > > >>>>>>>> the author spends two days > re-triggering a > >> > > flaky > >> > > > > job > >> > > > > >> to > >> > > > > >> > > make > >> > > > > >> > > > > it > >> > > > > >> > > > > > > >>>>> pass. But > >> > > > > >> > > > > > > >>>>>>>> other jobs which have passed the > >> validation > >> > may > >> > > > be > >> > > > > >> > broken > >> > > > > >> > > by > >> > > > > >> > > > > > > >>> other > >> > > > > >> > > > > > > >>>>>>> commits > >> > > > > >> > > > > > > >>>>>>>> during the two day without being > noticed. > >> And > >> > > > > finally > >> > > > > >> > the > >> > > > > >> > > PR > >> > > > > >> > > > > is > >> > > > > >> > > > > > > >>>>> merged > >> > > > > >> > > > > > > >>>>>>> with > >> > > > > >> > > > > > > >>>>>>>> underlying problems. > >> > > > > >> > > > > > > >>>>>>>> > >> > > > > >> > > > > > > >>>>>>>> On Fri, Mar 13, 2020 at 6:19 AM Marco de > >> > Abreu > >> > > < > >> > > > > >> > > > > > > >>>>> marco.g.ab...@gmail.com> > >> > > > > >> > > > > > > >>>>>>>> wrote: > >> > > > > >> > > > > > > >>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>> In the end it only comes down to money, > >> > > > > considering > >> > > > > >> > that > >> > > > > >> > > > the > >> > > > > >> > > > > > > >>>>> system is > >> > > > > >> > > > > > > >>>>>>>> auto > >> > > > > >> > > > > > > >>>>>>>>> scaling, making the execution time > >> constant. > >> > > > > >> > > > > > > >>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>> If we're trading money for usability, I > >> > > > certainly > >> > > > > >> would > >> > > > > >> > > > > prefer > >> > > > > >> > > > > > > >>>>>>> usability. > >> > > > > >> > > > > > > >>>>>>>>> I'd rather recommend to spend time on > >> > > > > parallelizing > >> > > > > >> > test > >> > > > > >> > > > > > > >>>> execution > >> > > > > >> > > > > > > >>>>> or > >> > > > > >> > > > > > > >>>>>>>>> getting rid of integration tests in the > >> PR > >> > > stage > >> > > > > >> > instead > >> > > > > >> > > > > > > >>> reducing > >> > > > > >> > > > > > > >>>>> the > >> > > > > >> > > > > > > >>>>>>>> costs > >> > > > > >> > > > > > > >>>>>>>>> by making people not use it. But > taking a > >> > step > >> > > > > back > >> > > > > >> to > >> > > > > >> > > > > > > >>> requiring > >> > > > > >> > > > > > > >>>>> people > >> > > > > >> > > > > > > >>>>>>>> to > >> > > > > >> > > > > > > >>>>>>>>> manually trigger CI again doesn't feel > >> > right. > >> > > > > >> > > > > > > >>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>> I'm happy to see that bot deployed, but > >> I do > >> > > not > >> > > > > >> agree > >> > > > > >> > > with > >> > > > > >> > > > > > > >>>>> removing > >> > > > > >> > > > > > > >>>>>>> the > >> > > > > >> > > > > > > >>>>>>>>> auto trigger functionality for new > >> commits. > >> > > > > >> > > > > > > >>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>> -Marco > >> > > > > >> > > > > > > >>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>> Chaitanya Bapat <chai.ba...@gmail.com> > >> > > schrieb > >> > > > am > >> > > > > >> Do., > >> > > > > >> > > 12. > >> > > > > >> > > > > > > >>> März > >> > > > > >> > > > > > > >>>>> 2020, > >> > > > > >> > > > > > > >>>>>>>>> 22:47: > >> > > > > >> > > > > > > >>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>> @Marco Thanks for pointing that out. > >> > > > > >> > > > > > > >>>>>>>>>> Tomorrow i.e. Friday, March 13, 2020 > at > >> > 3:00 > >> > > > PM - > >> > > > > >> 3:30 > >> > > > > >> > > PM > >> > > > > >> > > > in > >> > > > > >> > > > > > > >>>>>>>> (UTC-08:00) > >> > > > > >> > > > > > > >>>>>>>>>> Pacific Time (US & Canada). > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>>> When do we expect this bot to be > >> deployed? > >> > > > > >> > > > > > > >>>>>>>>>> @Lin If all goes well in the next > week I > >> > can > >> > > > > >> deploy it > >> > > > > >> > > to > >> > > > > >> > > > > > > >>>> public > >> > > > > >> > > > > > > >>>>>>> Apache > >> > > > > >> > > > > > > >>>>>>>>>> (provided I get permissions from > Apache > >> > > Infra) > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>> @Marco Thanks for your feedback. > >> > > > > >> > > > > > > >>>>>>>>>>> CI system has to support the > community > >> > > without > >> > > > > >> > > requiring > >> > > > > >> > > > > > > >>>>> people to > >> > > > > >> > > > > > > >>>>>>>>>> constantly shepherd every single run > >> > > > > >> > > > > > > >>>>>>>>>> We have data for the number of times > CI > >> was > >> > > > > >> triggered > >> > > > > >> > > > > > > >>>>> unnecessarily > >> > > > > >> > > > > > > >>>>>>>> which > >> > > > > >> > > > > > > >>>>>>>>>> includes > >> > > > > >> > > > > > > >>>>>>>>>> - Entire build triggered instead of > >> > specific > >> > > > > build > >> > > > > >> > > > > > > >>>>>>>>>> - CI triggered when PR is still work > in > >> > > > progress > >> > > > > or > >> > > > > >> > not > >> > > > > >> > > > yet > >> > > > > >> > > > > > > >>>> ready > >> > > > > >> > > > > > > >>>>>>> (say > >> > > > > >> > > > > > > >>>>>>>> - > >> > > > > >> > > > > > > >>>>>>>>>> intermediate commits) > >> > > > > >> > > > > > > >>>>>>>>>> At the end its a trade-off > >> > > > > >> > > > > > > >>>>>>>>>> Money, Resources, Time to build for > each > >> > and > >> > > > > every > >> > > > > >> > > commit > >> > > > > >> > > > vs > >> > > > > >> > > > > > > >>>>> Pain of > >> > > > > >> > > > > > > >>>>>>>>>> triggering builds > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>>> Scan trigger plugin would poll SCM. > >> Can we > >> > > use > >> > > > > >> plugin > >> > > > > >> > > at > >> > > > > >> > > > > > > >>>>> scale? > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>> 1. I haven't tested it on scale. But I > >> > think > >> > > > with > >> > > > > >> the > >> > > > > >> > > > > current > >> > > > > >> > > > > > > >>>>> scale > >> > > > > >> > > > > > > >>>>>>> of > >> > > > > >> > > > > > > >>>>>>>>>> MXNet repo (191 open PRs i.e. checking > >> for > >> > > > > changes > >> > > > > >> to > >> > > > > >> > > 191 > >> > > > > >> > > > > > > >>>>> branches - > >> > > > > >> > > > > > > >>>>>>> It > >> > > > > >> > > > > > > >>>>>>>>>> should be manageable) > >> > > > > >> > > > > > > >>>>>>>>>> 2. What's the purpose of the plugin? > >> tldr; > >> > > > Branch > >> > > > > >> > > > discovery > >> > > > > >> > > > > > > >>> or > >> > > > > >> > > > > > > >>>>> branch > >> > > > > >> > > > > > > >>>>>>>>>> indexing. > >> > > > > >> > > > > > > >>>>>>>>>> Scan trigger plugin comes into the > >> picture > >> > > only > >> > > > > >> once > >> > > > > >> > per > >> > > > > >> > > > PR > >> > > > > >> > > > > > > >>> per > >> > > > > >> > > > > > > >>>>> job > >> > > > > >> > > > > > > >>>>>>>>> (i.e. 8 > >> > > > > >> > > > > > > >>>>>>>>>> times per PR for 8 jobs). It is > >> basically > >> > > done > >> > > > > >> when a > >> > > > > >> > > new > >> > > > > >> > > > PR > >> > > > > >> > > > > > > >>> is > >> > > > > >> > > > > > > >>>>> made > >> > > > > >> > > > > > > >>>>>>>> and > >> > > > > >> > > > > > > >>>>>>>>>> the job (say unix-cpu hasn't > discovered > >> the > >> > > new > >> > > > > PR > >> > > > > >> > > branch > >> > > > > >> > > > > > > >>> yet). > >> > > > > >> > > > > > > >>>>>>> That's > >> > > > > >> > > > > > > >>>>>>>>> it. > >> > > > > >> > > > > > > >>>>>>>>>> So it shouldn't be a problem for > public > >> > MXNet > >> > > > > repo. > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>> Thanks, > >> > > > > >> > > > > > > >>>>>>>>>> Chai > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>> On Thu, 12 Mar 2020 at 14:22, Marco de > >> > Abreu > >> > > < > >> > > > > >> > > > > > > >>>>>>> marco.g.ab...@gmail.com> > >> > > > > >> > > > > > > >>>>>>>>>> wrote: > >> > > > > >> > > > > > > >>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>>> Btw you forgot to set a date and time > >> for > >> > > the > >> > > > > >> metting > >> > > > > >> > > > > > > >>>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>>> On Thu, Mar 12, 2020 at 10:18 PM > Marco > >> de > >> > > > Abreu > >> > > > > < > >> > > > > >> > > > > > > >>>>>>>>> marco.g.ab...@gmail.com > >> > > > > >> > > > > > > >>>>>>>>>>> wrote: > >> > > > > >> > > > > > > >>>>>>>>>>> > >> > > > > >> > > > > > > >>>>>>>>>>>> Thanks Chai, I generally like the > >> idea of > >> > > the > >> > > > > >> bot. > >> > > > > >> > But > >> > > > > >> > > > > > > >>> I'm > >> > > > > >> > > > > > > >>>>> not a > >> > > > > >> > > > > > > >>>>>>>>>>> supporter > >> > > > > >> > > > > > > >>>>>>>>>>>> of the idea to disable any automatic > >> > > > triggering > >> > > > > >> > > > > > > >>> (disabling > >> > > > > >> > > > > > > >>>>> the > >> > > > > >> > > > > > > >>>>>>>>> webhook > >> > > > > >> > > > > > > >>>>>>>>>> is > >> > > > > >> > > > > > > >>>>>>>>>>>> also not an option, considering that > >> this > >> > > > will > >> > > > > >> > disable > >> > > > > >> > > > > > > >>>> master > >> > > > > >> > > > > > > >>>>>>>>>> triggers). > >> > > > > >> > > > > > > >>>>>>>>>>>> The CI system has to support the > >> > community > >> > > > > >> without > >> > > > > >> > > > > > > >>>> requiring > >> > > > > >> > > > > > > >>>>>>> people > >> > > > > >> > > > > > > >>>>>>>>> to > >> > > > > >> > > > > > > >>>>>>>>>>>> constantly shepherd every single > run. > >> > > > Disabling > >> > > > > >> > > > automatic > >> > > > > >> > > > > > > >>>>>>
