[ 
https://issues.apache.org/jira/browse/FLINK-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509298#comment-15509298
 ] 

ASF GitHub Bot commented on FLINK-4379:
---------------------------------------

Github user StefanRRichter commented on the issue:

    https://github.com/apache/flink/pull/2512
  
    Hi,
    
    I have some suggestions for renaming some of the interfaces and their 
methods in this pull request to come up with some clearer, more consistent 
naming schemes. I suggest the following changes:
    
    ## 1) Renaming the state handle that points to operator state: 
PartitionableStateHandle -> OperatorStateHandle
    
    ## 2) Rename: PartitionableStateBackend -> OperatorStateStore
    ```
        /**
         * User-side interface for storing (partitionable) operator state.
         */
        public interface OperatorStateStore {
    
                /**
                 * Creates (or restores) the partitionable state in this 
backend. Each state is registered under a unique name.
                 * The provided serializer is used to de/serialize the state in 
case of checkpointing (snapshot/restore).
                 */
                <S> ListState<S> getListState(String name, TypeSerializer<S> 
partitionStateSerializer) throws Exception;
        }
    ```
    
    ## 3) Rename: PartitionableSnapshotStateBackend -> OperatorStateBackend. I 
propose that the term backend now refers to some (i) store with the ability to 
(ii) snapshot.
    ```
        /**
         * Interface that combines both, the user facing {@link 
OperatorStateStore} interface and the system interface
         * {@link SnapshotProvider}
         */
        public interface OperatorStateBackend
                        extends OperatorStateStore, 
SnapshotProvider<PartitionableOperatorStateHandle> {
        }
    ```
    
    ## 4) Rename: PartitionableCheckpointed -> CheckpointedOperator
        - `storeOperatorState` -> `snapshotState`
        - `restoreOperatorState` -> `restoreState`
    
    ```
        public interface CheckpointedOperator {
    
                /**
                 * This method is called when state should be stored for a 
checkpoint. The state can be registered and written to
                 * the provided state store.
                 */
                void snapshotState(long checkpointId, OperatorStateStore 
stateStore) throws Exception;
    
                /**
                 * This method is called when state should be restored from a 
checkpoint. The state can be obtained from the
                 * provided state store.
                 */
                void restoreState(OperatorStateStore stateStore) throws 
Exception;
        }
    ```
    
    ## 5) Rename: StateRepartitioner -> OperatorStateRepartitioner
    ```
        /**
         * Interface that allows to implement different strategies for 
repartitioning of operator state as parallelism changes.
         */
        public interface OperatorStateRepartitioner {
    
                List<Collection<OperatorStateHandle>> repartitionOperatorState(
                                List<OperatorStateHandle> 
previousParallelSubtaskStates,
                                int parallelism);
        }
    ```
    
    ## 6) Add new interface that allows user-friendly checkpointing code for 
simple cases that to not require custorm serializer
    ```
        /**
         * Simplified interface as adapter to the more complex 
CheckpointedOperator
         */
        public interface ListCheckpointed<T extends Serializable> {
    
                List<T> snapshotState(long checkpointId) throws Exception;
    
                void restoreState(List<T> state) throws Exception;
        }
    ```
    
    ## 7) OperatorStateBackend lifecycle
    
    Another point that we might want to discuss is the life cycle of 
`OperatorStateBackend`. Currently, a new backend is created (+restored) for 
each invocation of the methods in `CheckpointedOperator`. This always provides 
a clean backend to take the operator state for a snapshot. I wonder if it could 
make sense to create `OperatorStateBackend` just once for each 
`AbstractStreamOperator`, similar to the KeyedStateBackend. This would give 
users the option to actually keep operator state only in the 
`OperatorStateBackend`. However, we need a way to signal that all state must be 
passed to the backend before a snapshot. For example, large operator states 
could be managed in RocksDB this way, and we could provide more proxy 
collections (currently we only support a list of substates) over time.
    
    What do you think @aljoscha @StephanEwen ?


> Add Rescalable Non-Partitioned State
> ------------------------------------
>
>                 Key: FLINK-4379
>                 URL: https://issues.apache.org/jira/browse/FLINK-4379
>             Project: Flink
>          Issue Type: New Feature
>          Components: State Backends, Checkpointing
>            Reporter: Ufuk Celebi
>            Assignee: Stefan Richter
>
> This issue is associated with [FLIP-8| 
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-8%3A+Rescalable+Non-Partitioned+State].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to