Yes, to be extra clear: User state cleanup works correctly in the
FlinkRunner.
The portable Flink Runner needs a fix to behave the same way. Thanks for
opening the issue Thomas.
-Max
On 21.03.19 14:25, Thomas Weise wrote:
Created https://issues.apache.org/jira/browse/BEAM-6876 for the portable
Flink runner issue.
On Wed, Mar 20, 2019 at 11:10 AM Kenneth Knowles <[email protected]
<mailto:[email protected]>> wrote:
On Wed, Mar 20, 2019 at 6:23 AM Maximilian Michels <[email protected]
<mailto:[email protected]>> wrote:
Hi,
I just realized that user state acquired via StateInternals in
the Flink
Runner is not released automatically even when it falls out of the
Window scope. There are ways to work around this, i.e. setting a
cleanup
timer that fires when the Window expires.
Do we expect Runners to perform the cleanup? I would think so since
users do not have control over state once the window expires.
Just to be super clear for anyone not digging in the referenced
code: yes, we do. And the code Reuven referenced is utility code
that a runner can use to facilitate this, or the runner can do its
own thing.
Kenn
Thanks,
Max
