Re: General guidance

2021-04-15 Thread Almeida, Julius
utilization. And fails with java.io.IOException: Cannot allocate memory Configs used : Replicas: 15 CPUs: 2 Mem: 8Gi Offset: 4 Thanks, Julius From: "Almeida, Julius" Reply-To: "user@beam.apache.org" Date: Monday, March 29, 2021 at 8:11 AM To: "user@beam.apache.org"

Re: General guidance

2021-04-15 Thread Almeida, Julius
ulius" Reply-To: "user@beam.apache.org" Date: Monday, March 29, 2021 at 8:11 AM To: "user@beam.apache.org" Subject: Re: General guidance This email is from an external sender. Hi Jan, I am using Beam 2.23, but let me upgrade to 2.25 7 verify. Thanks, Julius From:

Re: General guidance

2021-03-29 Thread Almeida, Julius
Hi Jan, I am using Beam 2.23, but let me upgrade to 2.25 7 verify. Thanks, Julius From: Jan Lukavský Reply-To: "user@beam.apache.org" Date: Monday, March 29, 2021 at 12:56 AM To: "user@beam.apache.org" Subject: Re: General guidance This email is from an external sender.

Re: General guidance

2021-03-29 Thread Jan Lukavský
Hi Julius, which version of Beam do you run? There has been a fix for 2.25.0 [1] which could address what you see.  Jan [1] https://issues.apache.org/jira/browse/BEAM-10760 On 3/25/21 8:11 PM, Kenneth Knowles wrote: This is a Beam issue indeed, though it is an issue with the FlinkRunner. So

Re: General guidance

2021-03-25 Thread Kenneth Knowles
This is a Beam issue indeed, though it is an issue with the FlinkRunner. So I think I will BCC the Flink list. You may be in one of the following situations: - These timers should not be viewed as distinct by the runner, but deduped, per https://issues.apache.org/jira/browse/BEAM-8212#comment-169

General guidance

2021-03-25 Thread Almeida, Julius
Hi Team, My streaming pipeline is based on beam & running using flink runner with rocksdb as state backend. Over time I am seeing memory spike & after giving a look at heap dump, I am seeing records in ‘__StatefulParDoGcTimerId’ which seems to be never cleaned. Found this jira https://issues