Oops, I'm mixing up terms here. Topics != Subscriptions. We shouldn't be leaking topics.
Andrew On Wed, Feb 6, 2019 at 9:04 AM Kenneth Knowles <k...@google.com> wrote: > To clarify, PubsubIO does not clean up auto-created subscriptions, and SQL > doesn't compensate for that. > > On Wed, Feb 6, 2019 at 8:45 AM Mikhail Gryzykhin <mig...@google.com> > wrote: > >> Thank you for quick response Andrew. >> >> I'll cleanup these. I'll keep the bug open and assign it to @Kenneth >> Knowles <k...@google.com> who's working on SQK for follow up: we need a >> way to automatically cleanup topics. >> >> Current suggestions: >> 1. Make SQL cleanup created topics >> 2. Cleanup topics created by SQL in tests >> 3. Increase quota so that topics have enough time to be cleaned up >> automatically. >> >> Regards, >> --Mikhail >> >> Have feedback <http://go/migryz-feedback>? >> >> >> On Wed, Feb 6, 2019 at 8:05 AM Andrew Pilloud <apill...@google.com> >> wrote: >> >>> SQL doesn't cleanup pubsub subscriptions. Feel free to delete those. >>> >>> Andrew >>> >>> On Wed, Feb 6, 2019, 8:01 AM Mikhail Gryzykhin <mig...@google.com> >>> wrote: >>> >>>> +Kenneth Knowles <k...@google.com> you're working on SQL recently, so >>>> might provide some info. >>>> >>>> I see a lot of topics of format >>>> rojects/apache-beam-testing/topics/integ-test-PubsubJsonIT-testSQLLimit-2018-08-27-16-55-44-342-start-7392789257486934721 >>>> <https://pantheon.corp.google.com/cloudpubsub/topics/integ-test-PubsubJsonIT-testSQLLimit-2018-08-27-16-55-44-342-start-7392789257486934721?project=apache-beam-testing>. >>>> Seems we do not cleanup properly. >>>> >>>> Is it safe to cleanup topics with this name? >>>> >>>> --Mikhail >>>> >>>> Have feedback <http://go/migryz-feedback>? >>>> >>>> >>>> On Wed, Feb 6, 2019 at 7:46 AM Mikhail Gryzykhin <mig...@google.com> >>>> wrote: >>>> >>>>> Minor UPD: >>>>> As expected it fails most of our test jobs, since we use Pub/Subs in >>>>> many tests. >>>>> >>>>> --Mikhail >>>>> >>>>> Have feedback <http://go/migryz-feedback>? >>>>> >>>>> >>>>> On Wed, Feb 6, 2019 at 7:38 AM Mikhail Gryzykhin <mig...@google.com> >>>>> wrote: >>>>> >>>>>> Hi everyone, >>>>>> >>>>>> Our python pipelines failed with limit exceeded error >>>>>> <https://builds.apache.org/job/beam_PostCommit_Python_Verify/7313/console> >>>>>> : >>>>>> >>>>>> ResourceExhausted: 429 Your project has exceeded a limit: >>>>>> (type="topics-per-project", current=10000, maximum=10000). >>>>>> >>>>>> >>>>>> Does anyone know if there were new tests that use topics added >>>>>> recently? >>>>>> >>>>>> I tried to see list of topics, but UI fails >>>>>> <https://pantheon.corp.google.com/cloudpubsub/topicList?project=apache-beam-testing> >>>>>> to load. Will see if I can use APIs to investigate. >>>>>> >>>>>> If anyone has good insight, please, pick up BEAM-6610 >>>>>> <https://issues.apache.org/jira/browse/BEAM-6610>. >>>>>> >>>>>> --Mikhail >>>>>> >>>>>> Have feedback <http://go/migryz-feedback>? >>>>>> >>>>>
