Akshay Iyangar created BEAM-8212:
------------------------------------
Summary: StatefulParDoFn creates GC timers for every record
Key: BEAM-8212
URL: https://issues.apache.org/jira/browse/BEAM-8212
Project: Beam
Issue Type: Bug
Components: beam-community
Reporter: Akshay Iyangar
Assignee: Aizhamal Nurmamat kyzy
Hi
So currently the StatefulParDoFn create timers for all the records.
[https://github.com/apache/beam/blob/master/runners/core-java/src/main/java/org/apache/beam/runners/core/StatefulDoFnRunner.java#L211]
This becomes a problem if you are using GlobalWindows for streaming where these
timers get created and never get closed since the window will never close.
This is a problem especially if your memory bound in rocksDB where these timers
take up potential space and sloe the pipelines considerably.
Was wondering that if the pipeline runs in global windows we should avoid
adding timers to it at all?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
