+1. I believe there were too many dataflow jobs running at that moment that exceeded the compute engine quota of us-central1 ( the current quota is 100).
Yifan On Fri, Nov 2, 2018 at 1:49 PM Alan Myrvold <amyrv...@google.com> wrote: > Not sure, but I expect that each dataflow job needs a managed instance > group on the apache-beam-testing project, so it would depend on how many > dataflow jobs are being run in parallel. > The quota for concurrent dataflow jobs is 300, the quota for managed > instance groups is 100. It would make sense to increase the managed > instance group quota or reduce the number of concurrent jobs. > > On Fri, Nov 2, 2018 at 1:26 PM Mikhail Gryzykhin <mig...@google.com> > wrote: > >> Hi everyone, >> >> I was looking through (now 33) tickets open based on our post-commit >> tests failures and noticed one interesting. >> >> Test failed due to INSTANCE_GROUP_MANAGERS quota exceeding. >> BEAM-5938 <https://issues.apache.org/jira/browse/BEAM-5938> Thank you >> Daniel for triaging failure. >> >> Even thought this seem to happen only once so far, it might become a >> bigger issue since we are adding more tests. >> >> I've checked on the GCloud instance groups >> <https://cloud.google.com/compute/docs/instance-groups/>, but it didn't >> give me good insight into what might have triggered the quota issue. >> >> Does anyone know how we use instance group managers and triage whether >> this might hit us once we increase amount of tests? >> >> Regards, >> --Mikhail >> >> Have feedback <http://go/migryz-feedback>? >> >
