[
https://issues.apache.org/jira/browse/FLINK-25479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Khachatryan resolved FLINK-25479.
---------------------------------------
Assignee: Roman Khachatryan
Resolution: Fixed
Merged as 53c3525d0f4d4b3117edf1f8fcce0ff2ecfdda19.
> Changlog materialization with incremental checkpoint cannot work well in
> local tests
> ------------------------------------------------------------------------------------
>
> Key: FLINK-25479
> URL: https://issues.apache.org/jira/browse/FLINK-25479
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / State Backends, Tests
> Reporter: Yun Tang
> Assignee: Roman Khachatryan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> Currently, changelog materialization would call RocksDB state backend's
> snapshot method to generate {{IncrementalRemoteKeyedStateHandle}} as
> ChangelogStateBackendHandleImpl's materialized artifacts. And before next
> materialization, it will always report the same
> {{IncrementalRemoteKeyedStateHandle}} as before.
> For local tests, TM would report the {{IncrementalRemoteKeyedStateHandle}} to
> JM via local {{LocalRpcInvocation}}. However, as {{LocalRpcInvocation}} would
> not de/serialize message, which leads once we register the
> {{IncrementalRemoteKeyedStateHandle}} on JM side, it will also add a
> {{sharedStateRegistry}} to the one located on TM side. For the 2nd
> checkpoint, TM would reported same {{IncrementalRemoteKeyedStateHandle}} with
> {{sharedStateRegistry}} to JM. And it will then throw exception as it
> already contains a {{sharedStateRegistry}}:
> IncrementalRemoteKeyedStateHandle
> {code:java}
> public void registerSharedStates(SharedStateRegistry stateRegistry, long
> checkpointID) {
> Preconditions.checkState(
> sharedStateRegistry != stateRegistry,
> "The state handle has already registered its shared states to
> the given registry.");
> }
> {code}
> This bug would go in distribution environment as
> {{IncrementalRemoteKeyedStateHandle}} would be serialized and
> {{sharedStateRegistry}} is tagged as {{transient}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
