[ 
https://issues.apache.org/jira/browse/FLINK-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615136#comment-15615136
 ] 

ASF GitHub Bot commented on FLINK-4445:
---------------------------------------

GitHub user uce opened a pull request:

    https://github.com/apache/flink/pull/2712

    [FLINK-4445] Add option to ignore unmapped checkpoint state

    When restoring from a checkpoint/savepoint, state for each operator has to 
be restored. For savepoints, this means that the user cannot remove an operator 
from her topology and still use the savepoint.
    
    With this change, we will allow to ignore state that cannot be mapped back 
to the job being restored. The default behaviour does not change.
    
    ## Changes
    
    - I've removed the `allOrNothingState` flag as it was only effecting 
non-partitioned operator state and never set to `true` anyways (except tests). 
The flag controlled whether each non-partitioned operator state  was restored.
    - Moved the savepoint path from the `JobSnapshottingSettings` to the 
`JobGraph`
    - Added the `--ignoreUnmappedState` (short `-i`) flag to the run command: 
`bin/flink run -s <savepointPath> -i ...`
    
    
    I've tested this manually by triggering a savepoint for a job, adjusting 
the job (removing an operator), and then trying to resume from the savepoint. 
By default, restoring fails, but with the flag everything works.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/uce/flink 4445-unmatched_state

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2712.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2712
    
----
commit dc278c51b2bf1f580a6b4cb1670fb70ac871515f
Author: Ufuk Celebi <u...@apache.org>
Date:   2016-10-26T16:00:21Z

    [FLINK-4445] [client] Add ignoreUnmappedState flag to CLI
    
    Allow to specify whether a checkpoint restore should ignore
    checkpoint state that it cannot map to the program. This is
    exposed via the CLI in the run command:
    
    bin/flink run -s <savepointPath> -i ...
    
    Furthermore, the savepoint restore settings are moved out of
    the snapshotting settings.

commit 57621d30dfc4360c786d557a1a00fb57e2ade372
Author: Ufuk Celebi <u...@apache.org>
Date:   2016-10-26T16:05:26Z

    [FLINK-4445] [checkpointing] Add option to ignore unmapped checkpoint state
    
    Allows to ignore checkpoint state that cannot be mapped to a job vertex when
    restoring.

----


> Ignore unmatched state when restoring from savepoint
> ----------------------------------------------------
>
>                 Key: FLINK-4445
>                 URL: https://issues.apache.org/jira/browse/FLINK-4445
>             Project: Flink
>          Issue Type: Improvement
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.1.1
>            Reporter: Ufuk Celebi
>            Assignee: Ufuk Celebi
>
> When currently submitting a job with a savepoint, we require that all state 
> is matched to the new job. Many users have noted that this is overly strict. 
> I would like to loosen this and allow savepoints to be restored without 
> matching all state.
> The following options come to mind:
> (1) Keep the current behaviour, but add a flag to allow ignoring state when 
> restoring, e.g. {{bin/flink -s <savepoint> --ignoreUnmatchedState}}. This 
> would be non-API breaking.
> (2) Ignore unmatched state and continue. Additionally add a flag to be strict 
> about checking the state, e.g. {{bin/flink -s <savepoint> --strict}}. This 
> would be API-breaking as the default behaviour would change. Users might be 
> confused by this because there is no straight forward way to notice that 
> nothing has been restored.
> I'm not sure what's the best thing here. [~gyfora], [~aljoscha] What do you 
> think?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to