[
https://issues.apache.org/jira/browse/FLINK-17928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-17928:
-----------------------------------
Labels: pull-request-available (was: )
> Incorrect state size reported when using unaligned checkpoints
> ---------------------------------------------------------------
>
> Key: FLINK-17928
> URL: https://issues.apache.org/jira/browse/FLINK-17928
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
> Affects Versions: 1.11.0
> Reporter: Piotr Nowojski
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> Even when checkpoints on HDFS are between 100-300MBs, the reported state size
> is in orders of magnitude larger with values like:
> {noformat}
> 1GiB 1.5TiB 2.0TiB 2.1TiB 2.1TiB
> 148GiB 148GiB 148GiB 148GiB 148GiB 148GiB
> {noformat}
> it's probably because we have multiple
> {{Collection<InputChannelStateHandle>}}, and each of the individual handle
> returns the same value from {{AbstractChannelStateHandle#getStateSize}} - the
> full size of the spilled data, ignoring that only small portion of those data
> belong to a single input channel/result subpartition. In other words {{
> org.apache.flink.runtime.state.AbstractChannelStateHandle#getStateSize}}
> should be taking the offsets into account and return only the size of the
> data that belong exclusively to this handle.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
