[
https://issues.apache.org/jira/browse/FLINK-9114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17323568#comment-17323568
]
Flink Jira Bot commented on FLINK-9114:
---------------------------------------
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Enable user-provided, custom CheckpointRecoveryFactory for non-HA deployments
> -----------------------------------------------------------------------------
>
> Key: FLINK-9114
> URL: https://issues.apache.org/jira/browse/FLINK-9114
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration, Runtime / State Backends
> Reporter: Jacob Park
> Assignee: Jacob Park
> Priority: Major
> Labels: stale-assigned
>
> When you operate a Flink application that uses externalized checkpoints to
> S3, it becomes difficult to determine which checkpoint is the latest to
> recover from. Because S3 provides read-after-write consistency only for PUTS,
> listing a S3 path is not guaranteed to be consistent, so we do not know what
> checkpoint to recover from.
> The goal of this improvement is to allow users to provide a custom
> CheckpointRecoveryFactory for non-HA deployments such that we can use this
> feature to fail checkpoints if we cannot guarantee we will know where a
> checkpoint will be in S3, and co-publish checkpoint metadata to a strongly
> consistent data store.
> I propose the following changes:
> # Modify AbstractNonHaServices and StandaloneHaServices to accept an
> Executor for the custom CheckpointRecoveryFactory.
> # Create a CheckpointRecoveryFactoryLoader to provide the custom
> CheckpointRecoveryFactory from configurations.
> # Add new configurations for this feature.
> We considered the pluggable StateBackend and the potentially pluggable
> HighAvailabilityServices. These were too convoluted to solve our problem, so
> we would like to implement a custom CheckpointRecoveryFactory mechanism.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
