[
https://issues.apache.org/jira/browse/FLINK-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sihua Zhou updated FLINK-8602:
------------------------------
Description:
Right now, RocksDB backend supports {{fully checkpoint}} & {{incremental
checkpoint}}. This issue try to improve the performance of recovery from
{{fully checkpoint}} as well as from {{incremental checkpoint}}. It contains 2
sub-tasks.
1. improve recovery performance for incremental checkpoint when {{hasExtraKeys
= true}}
2. introduce `parallel recovery` from both {{fully checkpoint}} and
{{incremental checkpoint}} (Base on {{ingestExternalFile()}} and
{{SstFileWriter}} provided by RocksDB).
Any advice would be highly appreciated!
was:
Right now, RocksDB backend supports {{fully checkpoint}} & {{incremental
checkpoint}}. This issue try to improve the performance of recovery from
{{fully checkpoint}} as well as from {{increment checkpoint }}. It contains 2
sub-task.
1. improve recovery performance for incremental checkpoint when {{hasExtraKeys
= true}}
2. introduce `parallel recovery` from both {{fully checkpoint}} and
{{incremental checkpoint}} (Base on {{ingestExternalFile()}} and
{{SstFileWriter}} provided by RocksDB).
Any advice would be highly appreciated!
> Improve recovery performance for rocksdb backend
> ------------------------------------------------
>
> Key: FLINK-8602
> URL: https://issues.apache.org/jira/browse/FLINK-8602
> Project: Flink
> Issue Type: Improvement
> Components: State Backends, Checkpointing
> Affects Versions: 1.5.0
> Reporter: Sihua Zhou
> Assignee: Sihua Zhou
> Priority: Major
>
> Right now, RocksDB backend supports {{fully checkpoint}} & {{incremental
> checkpoint}}. This issue try to improve the performance of recovery from
> {{fully checkpoint}} as well as from {{incremental checkpoint}}. It contains
> 2 sub-tasks.
> 1. improve recovery performance for incremental checkpoint when
> {{hasExtraKeys = true}}
> 2. introduce `parallel recovery` from both {{fully checkpoint}} and
> {{incremental checkpoint}} (Base on {{ingestExternalFile()}} and
> {{SstFileWriter}} provided by RocksDB).
> Any advice would be highly appreciated!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
