> On 27 Apr 2021, at 08:39, Thomas Fredriksen(External)
> wrote:
>
> Thank you, this is very informative.
>
> We tried reducing the JdbcIO batch size from 1 to 1000, then to 100. In
> our runs, we no longer see the explicit OOM-error, but we are seeing executor
> heartbeat timeouts.
Hello all,
we are trying to run an Apache Beam pipeline which reads from Pub/Sub and
writes to multiple S3 buckets. The problem comes from the fact that we do
not know the region of destination buckets in Amazon. The items are mixed
on the Pub/Sub side for multiple s3 buckets in multiple regions.
Thank you, this is very informative.
We tried reducing the JdbcIO batch size from 1 to 1000, then to 100. In
our runs, we no longer see the explicit OOM-error, but we are seeing
executor heartbeat timeouts. From what we understand, this is typically
caused by OOM-errors also. However, the
