[
https://issues.apache.org/jira/browse/FLINK-25261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463808#comment-17463808
]
Roman Khachatryan edited comment on FLINK-25261 at 12/22/21, 2:43 PM:
----------------------------------------------------------------------
# Correct (as I wrote above in context of this ticket I mean in-memory data
only)
# As I wrote above in context of this ticket I mean in-memory data only
# In-memory means any in-memory object not needed for checkpointing anymore,
depending on DSTL implementation (reference to file or an in-memory byte array
to-be -flushed); To discard already flushed but not included into any
checkpoint changes, we have three options: a) shared/private state ownership
and TM-side registry (FLINK-23139); b) TM-side registry only for changelog; c)
rely on FLINK-24852 (which will likely be needed by FLINK-25395). I propose to
postpone this decision until we decide on FLINK-25395 and state ownership. Note
that this only happens if pre-emptive upload is enabled (otherwise, state
changes are always associated with some checkpoint)
# Materialization is independent, but handling its result is "synchronized"
with checkpointing by using Task mailbox; so it's only the writer.truncate()
method that should take ongoing checkpoints into account; and this is the with
the current FS writer.
was (Author: roman_khachatryan):
# Correct (as I wrote above in context of this ticket I mean in-memory data
only)
# As I wrote above in context of this ticket I mean in-memory data only
# In-memory means any in-memory object not needed for checkpointing anymore,
depending on DSTL implementation (reference to file or an in-memory byte array
to-be -flushed); To discard already flushed but not included into any
checkpoint changes, we have three options: a) shared/private state ownership
and TM-side registry (FLINK-23139); b) TM-side registry only for changelog; c)
rely on FLINK-24852 (which will likely be needed by FLINK-25395). I propose to
postpone this decision until we decide on FLINK-25395 and state ownership.
# Materialization is independent, but handling its result is "synchronized"
with checkpointing by using Task mailbox; so it's only the writer.truncate()
method that should take ongoing checkpoints into account; and this is the with
the current FS writer.
> Changelog not truncated on materialization
> ------------------------------------------
>
> Key: FLINK-25261
> URL: https://issues.apache.org/jira/browse/FLINK-25261
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.15.0
> Reporter: Roman Khachatryan
> Assignee: Roman Khachatryan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> [https://github.com/apache/flink/blob/dcc4d43e413b20f70036e73c61d52e2e1c5afee7/flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/ChangelogKeyedStateBackend.java#L640]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
