[
https://issues.apache.org/jira/browse/FLINK-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219294#comment-16219294
]
ASF GitHub Bot commented on FLINK-5823:
---------------------------------------
GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/4907
[FLINK-5823] [checkpoints] State Backends also handle Checkpoint Metadata
(part 1)
This is an incremental (first part) rebuild of #3522 on the latest master.
For ease of review, broken down into small chunks.
## Part 1: Application-defined State Backends pick up additional values
from the configuration
We need to keep supporting the scenario of setting a state backends in the
user program, but configuring parameters like checkpoint directory in the
cluster config. To support that, state backends may implement an additional
interface which lets them pick up configuration values from the cluster
configuration.
This also makes testing the checkpoint / savepoint configuration much
easier.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink backend
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4907.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4907
----
commit 8055cc9d84e206abd00cbc339c6cc390c53f2afe
Author: Stephan Ewen <[email protected]>
Date: 2017-10-24T17:46:07Z
[hotfix] [hdfs] Avoid reparsing URIs in Hadoop File Status conversion
commit 96068b93d4c332a9b8cfba06ecc9d42804a79f92
Author: Stephan Ewen <[email protected]>
Date: 2017-10-24T19:20:44Z
[hotfix] [streaming] Move SerializedCheckpointData to proper scope for
MessageAcknowledgingSourceBase
commit e098cc6ac8d77a027aefb6c1bcee41f6214ed74c
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T09:56:59Z
[hotfix] [runtime] Minor optimization in CheckpointMetrics
commit 8a6b78a8dacadb5476c6bf452fbdab352d0ab908
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T11:30:51Z
[hotfix] [checkpoints] fix warnings and make minor improvements to
CheckpointCoordinatorTest and SharedStateRegistry
commit aa09d82fd57d2636f54ec2a56e457f19d8ec9490
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T11:46:34Z
[hotfix] [core] Fix FileUtils.deletePathIfEmpty
commit 15be2602d4d44e29e756cd558130f2303d82fe8b
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T17:04:58Z
[hotfix] [checkpoints] Remove incorrect 'Serializable' from
StateBackendFactory
commit 1438df297bd820c07787563ddcfd2fda8773387f
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T12:16:37Z
[FLINK-7924] [checkpoints] Fix incorrect names of checkpoint options
Checkpoint options are incorrectly always called 'FULL_CHECKPOINT' when
actually,
the checkpoints may always be incremental and only savepoints have to be
full
and self contained.
Initially, we planned to add options for multiple checkpoints, like
checkpoints
that were foreced to be full, and checkpoints that were incremental. That
is not necessary at this point.
commit 6b792bf72fdb0c17625b1fe97c52791d550a74e4
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T15:30:14Z
[FLINK-7925] [checkpoints] Add CheckpointingOptions
The CheckpointingOptions consolidate all checkpointing and state
backend-related
settings that were previously split across different classes.
commit c4ce5522d8052ae8af8134bac4ea74bb5a929027
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T11:23:46Z
[FLINK-5823] [checkpoints] Pass state backend to checkpoint coordinator
commit 337e354bbe198ce6ab68d23cec93bc0c81bbbfaf
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T15:32:17Z
[hotfix] [core] Fix broken JavaDoc links in ConfigConstants
commit 0522ac34146aecfad2dbd9c806e6f7be182d00fd
Author: Stephan Ewen <[email protected]>
Date: 2017-10-25T17:04:10Z
[FLINK-5823] [checkpoints] State backends define checkpoint and savepoint
directories, improved configuration
----
> Store Checkpoint Root Metadata in StateBackend (not in HA custom store)
> -----------------------------------------------------------------------
>
> Key: FLINK-5823
> URL: https://issues.apache.org/jira/browse/FLINK-5823
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Priority: Blocker
> Fix For: 1.4.0
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
