To be extra explicit/clear, what I believe to be true is, per material (i.e., repo):
100 webhook POSTs != 100 git fetch Therefore: 100 webhook POSTs != 100 pipeline runs On Thu, May 27, 2021 at 8:24 PM Marques Lee <[email protected]> wrote: > Hmm, I actually don’t know the exact answer. Probably @arvindsv, @maheshp, > @ketan might of off the top. They’ve all been on the codebase way earlier > than I have. I would guess that the answer to the question “will GoCD > throttle messaging for me” is “no” but I defer to them. > > What I do know is this — if your web hooks are firing off and the messages > build up in the queue, the MaterialUpdateService won’t necessarily hammer > the git repos. If messages are consumed while the material service is > updating, it will log a message that an update is already in progress and > do nothing. So that will help in that scenario, at least partly; 100 > commits != 100 `git fetch` operations, especially if they come all at once. > Quite possibly they’ll be effectively reduced to a single fetch operation > per repo. > > On Wed, May 26, 2021 at 10:09 PM HUSSEIN KADIRI <[email protected]> > wrote: > >> Would you happen to know the answer to this question? >> Oftentimes, in other tools, disabling maintenance mode after its been >> enabled for a long time leads to a flood of messages the server has to >> handle. Sometimes the server can't handle that much load and it craps out. >> Being able to control the flow while disabling maintenance would be great. >> Is that something that can be done in GoCd? >> >> On Wed, May 26, 2021, 10:07 PM Marques Lee <[email protected]> >> wrote: >> >>> Anytime 👍 >>> >>> On Wed, May 26, 2021 at 10:04 PM HUSSEIN KADIRI <[email protected]> >>> wrote: >>> >>>> Got it. As usual thanks for your helpful answers Marques. >>>> >>>> On Wed, May 26, 2021, 8:27 PM Marques Lee <[email protected]> >>>> wrote: >>>> >>>>> > Does the new server pick up from where the old one left off? >>>>> >>>>> Kind of. If you have a stage that’s in progress, and 3/5 jobs finished >>>>> and 2 are still running, if the server dies and you replace it with a new >>>>> instance it should start running those 2 jobs that never finished. But >>>>> that’s not to say they pick up exactly where they left off; for example, >>>>> if >>>>> one of those jobs was halfway through running a gradle task, it won’t >>>>> magically continue and finish the rest of that task; that job is basically >>>>> restarted when the server comes back up and quite possibly with a >>>>> different >>>>> agent. But you wouldn’t need to rerun the other jobs that had previously >>>>> finished in that stage run. >>>>> >>>>> On Wed, May 26, 2021 at 5:26 PM HUSSEIN KADIRI <[email protected]> >>>>> wrote: >>>>> >>>>>> I haven't given it a try. I understand how that process works with >>>>>> maintenance. My question is around when the server is unresponsive (for >>>>>> whatever reason) and so we're unable to put it in maintenance mode via >>>>>> the >>>>>> normal means. Fixing the issue might involve rebooting (for physical/vm >>>>>> servers) or recreating the server pod (for kubernetes use case). >>>>>> >>>>>> What's the behavior when the new server comes online? Does the new >>>>>> server pick up from where the old one left off? >>>>>> >>>>>> On Wednesday, 26 May 2021 at 14:52:53 UTC-7 [email protected] wrote: >>>>>> >>>>>>> We have a maintenance mode in gocd which will for you in this case I >>>>>>> believe. Have you tried it? >>>>>>> >>>>>>> On Thu, May 27, 2021, 1:56 AM HUSSEIN KADIRI <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> We have a push based CICD workflow where an external system >>>>>>>> triggers CICD workflows. >>>>>>>> >>>>>>>> Sometimes the CICD server is temporarily offline (for a reboot, >>>>>>>> upgrade or whatever reason). Other times the CICD server is or unable >>>>>>>> to >>>>>>>> process the vast amount of jobs the external system is sending to it. >>>>>>>> >>>>>>>> Having a buffer (e.g a queue) in-front of the GoCD server would >>>>>>>> beneficial because: >>>>>>>> - Jobs can still be queued up and resumed when the server is back >>>>>>>> up. >>>>>>>> - Job flow to GoCD server can be controlled letting it recover at a >>>>>>>> pace it can handle. >>>>>>>> >>>>>>>> Does this buffer implementation exist in GoCD. Or is it something >>>>>>>> we'll have to develop outside? >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "go-cd" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/go-cd/6564a7d3-98ae-4ae5-b9a8-83fbc9fc927fn%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/go-cd/6564a7d3-98ae-4ae5-b9a8-83fbc9fc927fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "go-cd" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/go-cd/fbc863f4-bdac-4686-87d5-b9d65ebb00dcn%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/go-cd/fbc863f4-bdac-4686-87d5-b9d65ebb00dcn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "go-cd" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/go-cd/Ak-T_vvwKKE/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jYAQLtRiDTWahcBHxGcJ57XHnP%2BnZscbZ1rjVuEKGE8Zw%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jYAQLtRiDTWahcBHxGcJ57XHnP%2BnZscbZ1rjVuEKGE8Zw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "go-cd" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dn738G7S8k15SLaC%2Bo0Gxik1EkAiYboFR4Ps7WLPuCK%3Dw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dn738G7S8k15SLaC%2Bo0Gxik1EkAiYboFR4Ps7WLPuCK%3Dw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "go-cd" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/go-cd/Ak-T_vvwKKE/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/go-cd/CAPKX9jau0hpCRgo%3DjKYEaqaaRc0tULCtxcaT8VWnoVvmeyN_vg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jau0hpCRgo%3DjKYEaqaaRc0tULCtxcaT8VWnoVvmeyN_vg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/CAFD%2B7D%3Db%3D60pM_08XZVGN4N0-Pzgd3UtozNNeO_tQegptcre%2Bw%40mail.gmail.com >> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7D%3Db%3D60pM_08XZVGN4N0-Pzgd3UtozNNeO_tQegptcre%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CAPKX9jbk-HPfedOyn1GY1aNeE%2BqngcQvaWJkqn4Ve%3Dc%3DaHT72w%40mail.gmail.com.
