[
https://issues.apache.org/jira/browse/FLINK-23170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Khachatryan updated FLINK-23170:
--------------------------------------
Description:
Currently, changelog state backend writes state metadata on first state access.
It is written to the changelog
On materialization, the changelog can be truncated, so the metadata needs to be
written again.
This can be achieved by resetting AbstractStateChangeLogger.metaDataWritten
flag.
It can be further optimized by storing the SQN at which the metadata was
written and only resetting the flag if materializedSqn >= metadataSqn; but
materialization is relatively rare so it probably doesn't worth it.
Note: with TM-side state ownership, actual log truncation may be delayed after
materialization (until all the checkpoints using the log are subsumed). This
should not affect the above logic.
was:
Currently, changelog state backend writes state metadata on first state access.
It is written to the changelog
On materialization, the changelog can be truncated, so the metadata needs to be
written again.
This can be achieved by resetting AbstractStateChangeLogger.metaDataWritten
flag.
It can be further optimized by storing the SQN at which the metadata was
written and only resetting the flag if materializedSqn >= metadataSqn; but
materialization is relatively rare so it probably doesn't worth it.
> Write metadata after materialization
> ------------------------------------
>
> Key: FLINK-23170
> URL: https://issues.apache.org/jira/browse/FLINK-23170
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / State Backends
> Reporter: Roman Khachatryan
> Priority: Major
> Fix For: 1.14.0
>
>
> Currently, changelog state backend writes state metadata on first state
> access. It is written to the changelog
> On materialization, the changelog can be truncated, so the metadata needs to
> be written again.
> This can be achieved by resetting AbstractStateChangeLogger.metaDataWritten
> flag.
> It can be further optimized by storing the SQN at which the metadata was
> written and only resetting the flag if materializedSqn >= metadataSqn; but
> materialization is relatively rare so it probably doesn't worth it.
> Note: with TM-side state ownership, actual log truncation may be delayed
> after materialization (until all the checkpoints using the log are subsumed).
> This should not affect the above logic.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
