My guess is that this is due to running this both on GitHub Actions and Jenkins. The Actions run succeeded, so I don't think we need to worry about this - https://github.com/apache/beam/actions/runs/6714783844
It seems like for the metrics job the opposite happened - the Actions run failing triggering an email <https://lists.apache.org/thread/1mw4xq4tlp7htfxnq3cswqdwb0g17gbq> and the Jenkins job succeeded - https://ci-beam.apache.org/job/Rotate%20Metrics%20Cluster%20Credentials/ I put up https://github.com/apache/beam/pull/29243 to remove the Jenkins jobs in favor of the Actions jobs, which should fix the issue. Thanks, Danny On Tue, Oct 31, 2023 at 10:41 PM Svetak Sundhar via dev <dev@beam.apache.org> wrote: > I took a quick look -- the error is the following: > > *22:17:26* ERROR: (gcloud.container.clusters.update) ResponseError: code=400, > message=Operation > operation-1698804621818-e9c8fe33-d4a2-44cd-86aa-9c4e09dea259 is currently > upgrading cluster io-datastores. Please wait and try again once it is done. > > > > > This is different than the last time this error happened > (https://lists.apache.org/thread/xw2hx8yycpfmhf64w0vyt96r0d8zwnyg) > > > I noticed node pool pool-1 was still updating when this error was sent, so I > think it should succeed now. > > > Should we retrigger the seed job manually? > > > > Svetak Sundhar > > Data Engineer > s <nellywil...@google.com>vetaksund...@google.com > > > > On Tue, Oct 31, 2023 at 10:17 PM Apache Jenkins Server < > jenk...@builds.apache.org> wrote: > >> Something went wrong during the automatic credentials rotation for >> IO-Datastores Cluster, performed at Wed Nov 01 00:52:45 UTC 2023. It may be >> necessary to check the state of the cluster certificates. For further >> details refer to the following links: >> * Failing job: >> https://ci-beam.apache.org/job/Rotate%20IO-Datastores%20Cluster%20Credentials/ >> * Job configuration: >> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_IODatastoresCredentialsRotation.groovy >> * Cluster URL: >> https://pantheon.corp.google.com/kubernetes/clusters/details/us-central1-a/io-datastores/details?mods=dataflow_dev&project=apache-beam-testing > >
