Re: Resource usage exceeded: topics-per-project

Ah, I jumped to the wrong vocabulary. There is no auto-created topic. That is part of TestPubsub and TestPubsubSignal. It should be cleaned up. The only use of those is in SQL ITs. But PubsubIO _should_ use it. Kenn On Wed, Feb 6, 2019 at 9:04 AM Kenneth Knowles wrote: > To clarify, PubsubIO

Re: Resource usage exceeded: topics-per-project

Oops, I'm mixing up terms here. Topics != Subscriptions. We shouldn't be leaking topics. Andrew On Wed, Feb 6, 2019 at 9:04 AM Kenneth Knowles wrote: > To clarify, PubsubIO does not clean up auto-created subscriptions, and SQL > doesn't compensate for that. > > On Wed, Feb 6, 2019 at 8:45 AM

Re: Resource usage exceeded: topics-per-project

To clarify, PubsubIO does not clean up auto-created subscriptions, and SQL doesn't compensate for that. On Wed, Feb 6, 2019 at 8:45 AM Mikhail Gryzykhin wrote: > Thank you for quick response Andrew. > > I'll cleanup these. I'll keep the bug open and assign it to @Kenneth > Knowles who's

Re: Resource usage exceeded: topics-per-project

Thank you for quick response Andrew. I'll cleanup these. I'll keep the bug open and assign it to @Kenneth Knowles who's working on SQK for follow up: we need a way to automatically cleanup topics. Current suggestions: 1. Make SQL cleanup created topics 2. Cleanup topics created by SQL in tests

Re: Resource usage exceeded: topics-per-project

SQL doesn't cleanup pubsub subscriptions. Feel free to delete those. Andrew On Wed, Feb 6, 2019, 8:01 AM Mikhail Gryzykhin wrote: > +Kenneth Knowles you're working on SQL recently, so > might provide some info. > > I see a lot of topics of format >

Re: Resource usage exceeded: topics-per-project

+Kenneth Knowles you're working on SQL recently, so might provide some info. I see a lot of topics of format rojects/apache-beam-testing/topics/integ-test-PubsubJsonIT-testSQLLimit-2018-08-27-16-55-44-342-start-7392789257486934721

Re: Resource usage exceeded: topics-per-project

Minor UPD: As expected it fails most of our test jobs, since we use Pub/Subs in many tests. --Mikhail Have feedback ? On Wed, Feb 6, 2019 at 7:38 AM Mikhail Gryzykhin wrote: > Hi everyone, > > Our python pipelines failed with limit exceeded error >