[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/5239 ---

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170274734 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FileBasedStateOutputStream.java --- @@ -0,0 +1,155 @@ +/* + * Lice

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170271924 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FileBasedStateOutputStream.java --- @@ -0,0 +1,155 @@ +/* + *

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170270552 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskLocalStateStore.java --- @@ -166,20 +190,38 @@ public TaskStateSnapshot retr

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170269557 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FileBasedStateOutputStream.java --- @@ -0,0 +1,155 @@ +/* + * Lice

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170268121 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerServices.java --- @@ -141,7 +147,7 @@ public JobLeaderService

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170266913 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DuplicatingCheckpointOutputStream.java --- @@ -0,0 +1,287 @@ +/* + * Lic

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170266486 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FileBasedStateOutputStream.java --- @@ -0,0 +1,155 @@ +/* + *

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170264017 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DuplicatingCheckpointOutputStream.java --- @@ -0,0 +1,287 @@ +/* + * Lic

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170263293 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskExecutorLocalStateStoresManager.java --- @@ -186,6 +210,34 @@ private void d

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170261966 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -270,6

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170260996 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/LocalRecoveryConfig.java --- @@ -0,0 +1,120 @@ +/* + * Licensed to the A

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170253853 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/TaskLocalStateStoreImplTest.java --- @@ -59,12 +59,18 @@ public void before() th

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170253246 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/TaskExecutorLocalStateStoresManagerTest.java --- @@ -46,9 +48,15 @@ public void

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170253088 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -1629,

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170251605 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -270,6 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170250183 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskLocalStateStoreImpl.java --- @@ -0,0 +1,298 @@ +/* + * Licensed to the

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170243957 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerServices.java --- @@ -141,7 +147,7 @@ public JobLeaderService getJo

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170242003 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FileBasedStateOutputStream.java --- @@ -0,0 +1,155 @@ +/* + * Lice

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170249880 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskExecutorLocalStateStoresManager.java --- @@ -186,6 +210,34 @@ private void doR

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170239200 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DuplicatingCheckpointOutputStream.java --- @@ -0,0 +1,287 @@ +/* + * Licensed

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170238910 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DuplicatingCheckpointOutputStream.java --- @@ -0,0 +1,287 @@ +/* + * Licensed

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170229683 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/LocalRecoveryConfig.java --- @@ -0,0 +1,120 @@ +/* + * Licensed to the A

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-23 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170228635 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskExecutorLocalStateStoresManager.java --- @@ -89,6 +90,22 @@ public TaskExecu

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170039184 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/TestLocalDirectoryProvider.java --- @@ -0,0 +1,40 @@ +/* + * Licensed to t

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170038982 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/TaskLocalStateStoreImplTest.java --- @@ -59,12 +59,18 @@ public void before() thro

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170038223 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/TaskExecutorLocalStateStoresManagerTest.java --- @@ -46,9 +48,15 @@ public void te

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170037177 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/LocalRecoveryConfig.java --- @@ -0,0 +1,120 @@ +/* + * Licensed to the Apa

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170035520 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/LocalRecoveryConfig.java --- @@ -0,0 +1,120 @@ +/* + * Licensed to the Apa

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170034046 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -1629,9

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170033424 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskManagerServicesConfiguration.java --- @@ -193,7 +193,12 @@ public stati

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r170026181 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskExecutorLocalStateStoresManager.java --- @@ -89,6 +90,22 @@ public TaskExecuto

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169909152 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotDirectory.java --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apa

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169905650 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotDirectory.java --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apach

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169899631 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotDirectory.java --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apa

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169899133 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotDirectory.java --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apa

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169898758 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DirectoryStateHandle.java --- @@ -0,0 +1,101 @@ +/* + * Licensed to the

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169898668 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotDirectory.java --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apa

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169897306 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DirectoryStateHandle.java --- @@ -0,0 +1,101 @@ +/* + * Licensed to the

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169896224 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/CheckpointStreamWithResultProvider.java --- @@ -172,4 +167,30 @@ public Checkpoi

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169896622 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DirectoryKeyedStateHandle.java --- @@ -0,0 +1,105 @@ +/* + * Licensed to

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169895777 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -2123,

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169895663 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -1552,

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169728016 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotDirectory.java --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apach

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169726507 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DirectoryStateHandle.java --- @@ -0,0 +1,101 @@ +/* + * Licensed to the Ap

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169727620 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotDirectory.java --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apach

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169723987 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -1552,10

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169725430 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/CheckpointStreamWithResultProvider.java --- @@ -172,4 +167,30 @@ public Checkpoint

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169724637 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -2123,9

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169727315 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SnapshotDirectory.java --- @@ -0,0 +1,172 @@ +/* + * Licensed to the Apach

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169726050 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/DirectoryKeyedStateHandle.java --- @@ -0,0 +1,105 @@ +/* + * Licensed to t

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169651017 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169649287 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-21 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169638326 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-19 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169062887 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-19 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169062508 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-19 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169061774 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-19 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169047804 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-19 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169047628 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-19 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r169021546 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-17 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168923652 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-17 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168923464 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-17 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168923188 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread bowenli86
Github user bowenli86 commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168891314 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** The co

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168826365 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,11 +30

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168779711 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PrioritizedOperatorSubtaskState.java --- @@ -0,0 +1,265 @@ +/* + * Licen

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168779365 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PrioritizedOperatorSubtaskState.java --- @@ -0,0 +1,265 @@ +/* + *

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168778447 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PrioritizedOperatorSubtaskState.java --- @@ -0,0 +1,265 @@ +/* + * Licen

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168778376 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168778178 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r16849 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PrioritizedOperatorSubtaskState.java --- @@ -0,0 +1,265 @@ +/* + *

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r16837 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,11 +30

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168777497 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PrioritizedOperatorSubtaskState.java --- @@ -0,0 +1,265 @@ +/* + *

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168776176 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,11 +30

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168775173 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PrioritizedOperatorSubtaskState.java --- @@ -0,0 +1,265 @@ +/* + * Licen

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168774627 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PrioritizedOperatorSubtaskState.java --- @@ -0,0 +1,265 @@ +/* + * Licen

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168773814 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168773128 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,11 +30

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168762457 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,1

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168760098 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -266,11 +30

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168753623 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168753503 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168753488 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168753362 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168753314 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168753015 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752606 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168750923 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168751139 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java --- @@ -336,1697 +

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752498 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskExecutorLocalStateStoresManager.java --- @@ -18,81 +18,267 @@ package

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752518 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752445 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752196 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752319 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752264 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752042 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168752064 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168751784 --- Diff: docs/ops/state/large_state_tuning.md --- @@ -234,4 +234,97 @@ Compression can be activated through the `ExecutionConfig`: **Notice:** T

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168747439 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskLocalStateStoreImpl.java --- @@ -0,0 +1,298 @@ +/* + * Licensed to t

[GitHub] flink pull request #5239: [FLINK-8360] Implement task-local state recovery

2018-02-16 Thread sihuazhou
Github user sihuazhou commented on a diff in the pull request: https://github.com/apache/flink/pull/5239#discussion_r168747007 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskLocalStateStoreImpl.java --- @@ -0,0 +1,298 @@ +/* + * Licensed to the Ap

  1   2   3   4   >