Hello James! Thanks for the reminder and the bump.
We saw the peak this morning: The ATH is not using container agents (Is it because it uses Docker, a bunch of memory or both?) but VMs of kind "higmem". It means that the work we’ve done on the Kubernetes agent scaling is not used. We are working on increasing the limit on VMs, but we hit cloud limitations for now (to be quite transparent, the setup of VM agent is not really fine tuned). Today shall see the ability to spawn 50 highmem VM in Azure and 30 in AWS EC2 (instead of 15/15 this morning). Hope it will help. Damien > Le 11 oct. 2021 à 23:30, '[email protected]' via Jenkins Developers > <[email protected]> a écrit : > > Hi Damien, > > Did this lead to anything? The CI servers resources seem to be already > exhausted with a few PRs to ATH. > > /James > > On Monday, September 20, 2021 at 8:25:12 AM UTC+1 [email protected] > <http://gmail.com/> wrote: > Hello there ! We (the Jenkins admin) we'll try to add more capability for the > month of October. We'll have to plan but Digital Ocean (:lauve: Gavin) is the > first step, along with exteding the current limitation for VMs and containers. > > > This topic has been added to the upcoming Infra meeting: > https://hackmd.io/tYSCkhn7RSu7wREEKH9QIg > <https://hackmd.io/tYSCkhn7RSu7wREEKH9QIg> . > > Sorry for the delay of answer from the infra team, we were quite busy with > the consequences of the wiki thing. > > This email thread will be updated with the conclusions from the team meeting. > > Cheers > > Damien > > Le mardi 14 septembre 2021 à 12:04:14 UTC+2, Oleg Nenashev a écrit : > It is a good question to discuss with the Infra Team. I have added it to the > contributor summit logistics Google Doc, because this is the weekend when the > load is expected to spike the most. > https://docs.google.com/document/d/1QLWXNG23ui-LvQXth3UREzOvLYgTMSZcv-El0H141a4/edit#heading=h.jhodwcaw6jcl > > <https://docs.google.com/document/d/1QLWXNG23ui-LvQXth3UREzOvLYgTMSZcv-El0H141a4/edit#heading=h.jhodwcaw6jcl> > > FTR last year we did not have serious issues with CI, because we did not > promote the event too much. In 2019 we had a massive CI outage in the > beginning of the event, so James definitely has a point. We are interested to > keep our CI stable to provide good contributor experience, whether they > participate in Hacktoberfest or not > > > > On Friday, September 10, 2021 at 7:06:55 PM UTC+2 [email protected] <> > wrote: > ooh maybe use some of our digital ocean credits to expand build capacity > > On Fri, Sep 10, 2021 at 10:00 AM Jesse Glick <[email protected] <>> wrote: > On Fri, Sep 10, 2021 at 11:32 AM [email protected] <> <[email protected] > <>> wrote: > I at least hope to find some time to get the ATH to zero failures at some > point. > > That would be great! > > Is there a plan to increase the capacity (temporarily or not) > > or INFRA-1633 <https://issues.jenkins.io/browse/INFRA-1633> > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1ZvaDVX-MyDgF2CtDD5eNSAcxeMVSr4uk_5RPgCEyt1A%40mail.gmail.com > > <https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1ZvaDVX-MyDgF2CtDD5eNSAcxeMVSr4uk_5RPgCEyt1A%40mail.gmail.com?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to a topic in the Google > Groups "Jenkins Developers" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/jenkinsci-dev/7jitt4zBZvo/unsubscribe > <https://groups.google.com/d/topic/jenkinsci-dev/7jitt4zBZvo/unsubscribe>. > To unsubscribe from this group and all its topics, send an email to > [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-dev/cd7f6d44-d921-4cb8-ad63-e8b8ad3bdcebn%40googlegroups.com > > <https://groups.google.com/d/msgid/jenkinsci-dev/cd7f6d44-d921-4cb8-ad63-e8b8ad3bdcebn%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA34BD95-B624-4170-A710-0268D325C698%40gmail.com.
