Re: BQ IO GC thrashing when specifying .withMethod(STREAMING_INSERTS)

This is not really designed to be used from batch pipelines. From batch, you will definitely overwhelm BQ's quota, causing all sorts of problems. On Mon, Jul 1, 2019 at 10:54 AM Mikhail Gryzykhin wrote: > Hello everybody, > > This question is regarding user post on StackOverflow >

Re: BQ IO GC thrashing when specifying .withMethod(STREAMING_INSERTS)

I think the BQ streaming writes buffers data into batches and sends them when used with STREAMING_INSERTS. Have you been able to ask the user to get a heap dump to see what was using the majority of memory? On Mon, Jul 1, 2019 at 2:34 PM Lukasz Cwik wrote: > I think the Bq > > On Mon, Jul 1, 20

Re: BQ IO GC thrashing when specifying .withMethod(STREAMING_INSERTS)

I think the Bq On Mon, Jul 1, 2019 at 10:54 AM Mikhail Gryzykhin wrote: > Hello everybody, > > This question is regarding user post on StackOverflow > > . > > My understanding of prob

BQ IO GC thrashing when specifying .withMethod(STREAMING_INSERTS)

Hello everybody, This question is regarding user post on StackOverflow . My understanding of problem is that setting .withMethod(STREAMING_INSERTS) on BigQueryIO sink causes GC thrashi