Hi Chad, I was just reading up on the changes for Kubernetes. Looks 1.24 moves to containerd runtime images and we are still using DIND for our elastic agents. That seems like it could be the culprit. I'll do some testing and change our elastic agent images. Thanks for pointing that out.
On Thu, Oct 26, 2023 at 10:46 AM Chad Wilson <[email protected]> wrote: > Unfortunately the error message is a bit mysterious and useless. Which > agent image you are using? Anything special in the elastic agent pod spec > that might no longer work as expected on Kubernetes 1.24? (e.g use of > docker dind images) > > Does the pod get created (and fail) if you look at the events on the > kubernetes side, or does it never get that far? > > -Chad > > On Thu, Oct 26, 2023 at 11:37 PM Chad Wilson <[email protected]> > wrote: > >> Just curious - were the errors/stack traces on failure essentially >> identical before and after you upgraded your gocd and elastic agent plugin >> versions? >> >> On Thu, Oct 26, 2023 at 11:33 PM Kim Pham <[email protected]> wrote: >> >>> Hi Ashwanth, >>> >>> I checked the clusterrole of the service account it's using and it >>> basically has full access atm. >>> >>> PolicyRule: >>> Resources Non-Resource URLs Resource Names Verbs >>> --------- ----------------- -------------- ----- >>> events [] [] [*] >>> namespaces [] [] [*] >>> nodes [] [] [*] >>> pods/log [] [] [*] >>> pods [] [] [*] >>> >>> On Thu, Oct 26, 2023 at 10:21 AM 'Ashwanth Kumar' via go-cd < >>> [email protected]> wrote: >>> >>>> A wild guess, anything changed on the service account side or a custom >>>> role being added as part of the upgrade that is probably not allowing the >>>> gocd plugin to create the pod? >>>> >>>> Thanks, >>>> >>>> >>>> On Thu, 26 Oct 2023 at 20:27, Kim Pham <[email protected]> wrote: >>>> >>>>> Hi All, >>>>> >>>>> We recently began to encounter issues where pods were unable to be >>>>> created. Nothing has changed in terms of GoCD server, agent, and >>>>> Kubernetes elastic agent plugin versions. However, we did notice that the >>>>> cluster went through an automatic upgrade and updated gke version to >>>>> 1.24.14. GoCD is able to see the node pools through the 'Status Report' >>>>> button. >>>>> >>>>> When attempting to create an agent on those node pools, I do see a 500 >>>>> in the plugin logs and gocd-server logs. Attached are logs. >>>>> >>>>> I've tried updating GoCD and the plugins to latest release versions. >>>>> Our static agents that are running on older gke versions aren't having any >>>>> issues. >>>>> >>>>> Has anyone encountered this? >>>>> >>>>> Thanks in advance. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "go-cd" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>>> >>>> -- >>>> >>>> Ashwanth Kumar / ashwanthkumar.in >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "go-cd" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "go-cd" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAA1RwH-tw%3D0MFCgWRQoT8ZwFYcyJsGT8T5pPxrBzzu46Nn960w%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAA1RwH-tw%3D0MFCgWRQoT8ZwFYcyJsGT8T5pPxrBzzu46Nn960w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4fGOjEWD-epsn0eNSzJuWJermoKjK2ed4y11y5YPARGA%40mail.gmail.com.
