Thank you. Your suggestion is good and I benefit a lot. For my case, I want to know the state memory size for other reasons. When the the gc pressure is bigger, I need to limit the source or discard some data from the source to ensure job’s running. If the state size is bigger, I need to discard data. If the state size is not bigger, I need to limit the source. The state size shows the resident memory. For event time, discarding data can reduce memory usage. Could you please give me some suggestions?
> 在 2019年11月20日,下午3:16,sysukelee <sysuke...@gmail.com> 写道: > > Hi Liu, > We monitor the jvm used/max heap memory to determine whether to rescale the > job. > To avoid problems caused by oom, you don't need to know exactly how much > memory exactly used by state. > Focusing on jvm memory use is more reasonable. > > <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=sysukelee&uid=sysukelee%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22sysukelee%40gmail.com%22%5D> > On 11/20/2019 15:08,刘建刚<liujiangangp...@gmail.com> > <mailto:liujiangangp...@gmail.com> wrote: > We are using flink 1.6.2. For filesystem backend, we want to monitor > the state size in memory. Once the state size becomes bigger, we can get > noticed and take measures such as rescaling the job, or the job may fail > because of the memory. > We have tried to get the memory usage for the jvm, like gc throughput. > For our case, state can vary greatly at the peak. So maybe I can refer to > the state memory size. > I checked the metrics and code, but didn't find any information about > the state memory size. I can get the checkpoint size, but they are > serialized result that can not reflect the running state in memory. Can > anyone give me some suggestions? Thank you very much.