[
https://issues.apache.org/jira/browse/FLINK-23461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Khachatryan updated FLINK-23461:
--------------------------------------
Fix Version/s: (was: 1.14.0)
1.15.0
> Consider disallowing in-memory state handles for materialized state
> -------------------------------------------------------------------
>
> Key: FLINK-23461
> URL: https://issues.apache.org/jira/browse/FLINK-23461
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / State Backends
> Reporter: Roman Khachatryan
> Priority: Major
> Fix For: 1.15.0
>
>
> *For non-mateialized part*, FLINK-21353 doesn't use nor
> FsCheckpointStreamFactory neither PlaceholderStreamStateHandle so it's not an
> issue. Adding it in the future doesn't make sense as for such small changes
> incremental checkpoint might work better.
>
> *For materialized part, ByteStreamStateHandle* can be currently used. This
> can bring back to life issues like FLINK-21351 - if checkpoint subsumption on
> TM *will* be decoupled from the state backends state. Removing those
> assumptions is one of the goals of changing the ownership.
> An easy way to solve it is to just enforce zero threshold for writing to DFS
> instead of memory.
>
> *PlaceholderStreamStateHandle can be used for the materialized state*
> (regardless of ByteStreamStateHandle; unless SnapshotStrategy is changed).
> However, it shouldn't cause any issues:
> - if the file is shared (i.e. after recovery) then by definition it should
> be managed by JM
> - otherwise, JM should still replace placeholders (FLINK-23137); and it
> should have received the original state objects before; no re-upload should
> happen (FLINK-23344) - so JM and TM will always refer to the same file
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
