[
https://issues.apache.org/jira/browse/FLINK-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481165#comment-17481165
]
Roman Khachatryan edited comment on FLINK-15507 at 1/24/22, 3:11 PM:
---------------------------------------------------------------------
[~pnowojski]
Regarding FLINK-24611, RocksDB behavior was reverted in FLINK-25395; and
Changelog currently doesn't support Local recovery, so FLINK-24611 shouldn't be
an issue.
We do need to implement it for Changelog; and probably not re-upload RocksDB
files too. But IMO these efforts should not block this ticket.
Regarding FLINK-8871, I think it did provide cleanup:
[https://github.com/apache/flink/blob/8fc657b899ec740a348c84a6f387c3806584f999/flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskLocalStateStoreImpl.java#L239]
[~Zakelly]
> Or we just activate it for all backends and the FsStateBackend will occupy
> more local disk space.
I think it's even more critical for RocksDB non-incremental, because it'll
likely occupy more local state.
> I prefer the latter. The configuration is more clear but it may bring some
> confusion in default behavior.
So I agree here, different behaviour depending on whether incremental
checkpoints are enabled is preferable.
was (Author: roman_khachatryan):
[~pnowojski]
Regarding FLINK-24611, RocksDB behavior was reverted in FLINK-25395; and
Changelog currently doesn't support Local recovery, so FLINK-24611 shouldn't be
an issue.
We need to approach the latter, and probably not re-upload RocksDB files too.
But IMO these efforts should not block this ticket.
Regarding FLINK-8871, I think it did provide cleanup:
[https://github.com/apache/flink/blob/8fc657b899ec740a348c84a6f387c3806584f999/flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskLocalStateStoreImpl.java#L239]
[~Zakelly]
> Or we just activate it for all backends and the FsStateBackend will occupy
> more local disk space.
I think it's even more critical for RocksDB non-incremental, because it'll
likely occupy more local state.
> I prefer the latter. The configuration is more clear but it may bring some
> confusion in default behavior.
So I agree here, different behaviour depending on whether incremental
checkpoints are enabled is preferable.
> Activate local recovery for RocksDB backends by default
> -------------------------------------------------------
>
> Key: FLINK-15507
> URL: https://issues.apache.org/jira/browse/FLINK-15507
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Reporter: Stephan Ewen
> Assignee: Yuan Mei
> Priority: Major
> Labels: auto-deprioritized-critical, auto-unassigned,
> pull-request-available
>
> For the RocksDB state backend, local recovery has no overhead when
> incremental checkpoints are used.
> It should be activated by default, because it greatly helps with recovery.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
