Hello Edward, please find my answers within your message below: On Wed, Nov 4, 2020 at 1:35 PM Colletta, Edward <edward.colle...@fmr.com> wrote:
> Using Flink 1.9.2 with FsStateBackend, Session cluster. > > > > 1. Does heap state get cleaned up when a job is cancelled? > > We have jobs that we run on a daily basis. We start each morning and > cancel each evening. We noticed that the process size does not seem to > shrink. We are looking at the resident size of the process with ps and > also the USED column for Heap on the taskmanager page of the flink > dashboard. > There is no explicit cleanup happening on the Flink side. The heap should be cleaned up when GC kicks in. > > 1. How can I examine the usage of Flink Managed Memory? > > The configuration documentation seems to indicate this is used for batch > jobs, and we are only using the Streaming API. I reduced > taskmanager.memory.fraction > to 0.3, but I think this is still reserving too much memory to an area we > will not be using. > Unfortunately, I don't know of any way to monitor the managed memory for Flink 1.9.2 as is. We're going to introduce new metrics for managed memory [1], network memory [2] and metaspace [3] in the upcoming release of Flink 1.12.0. This should make it easier to monitor these memory pools. I hope that helps a bit. Best, Matthias [1] https://issues.apache.org/jira/browse/FLINK-14406 [2] https://issues.apache.org/jira/browse/FLINK-14422 [3] https://issues.apache.org/jira/browse/FLINK-19617
