Re: Backpressure configuration

2021-04-29 Thread Kurtis Walker
. From: Roman Khachatryan Date: Thursday, April 29, 2021 at 2:49 PM To: Kurtis Walker Cc: user@flink.apache.org Subject: Re: Backpressure configuration EXTERNAL EMAIL Hello Kurt, Assuming that your sink is blocking, I would first make sure that it is not chained with the preceding operators

Re: Backpressure configuration

2021-04-29 Thread Roman Khachatryan
Hello Kurt, Assuming that your sink is blocking, I would first make sure that it is not chained with the preceding operators. Otherwise, the same thread will output data and perform windowing/triggering. You can add disableChaining after addSink to prevent this [1]. Besides that, you probably cou

Backpressure configuration

2021-04-29 Thread Kurtis Walker
Hello, I’m building a POC for a Flink app that loads data from Kafka in to a Greenplum data warehouse. I’ve built a custom sink operation that will bulk load a number of rows. I’m using a global window, triggering on number of records, to collect the rows for each sink. I’m finding that whi