[GitHub] flink issue #4360: [FLINK-7220] [checkpoints] Update RocksDB dependency to 5...

2017-07-19 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4360 Yes, that is correct, I kindly asked them to also update their license in the releases on their issue tracker https://github.com/facebook/rocksdb/issues/2605. I will update the PR

[GitHub] flink issue #4360: [FLINK-7220] [checkpoints] Update RocksDB dependency to 5...

2017-07-18 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4360 Just noticed that the license is not updated in the 5.5.1 branch, but only in master. So I guess this means we have to wait for a release that has the updated Apache compatible license

[GitHub] flink issue #4360: [FLINK-7220] [checkpoints] Update RocksDB dependency to 5...

2017-07-18 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4360 CC @aljoscha - suggesting to add this change also to the 1.3.2 release. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request #4360: [FLINK-7220] [checkpoints] Update RocksDB dependen...

2017-07-18 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/4360 [FLINK-7220] [checkpoints] Update RocksDB dependency to 5.5.1 This PR updates the RocksDB dependency to 5.5.1. Further changes: - the more performant merge operator is used via

[GitHub] flink issue #4353: [FLINK-7213] Introduce state management by OperatorID in ...

2017-07-17 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4353 CC @StephanEwen @zentol --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #4353: [FLINK-7213] Introduce state management by Operato...

2017-07-17 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/4353 [FLINK-7213] Introduce state management by OperatorID in TaskManager Flink-5892 introduced the job manager / checkpoint coordinator part of managing state on the operator level instead

[GitHub] flink pull request #4192: [FLINK-6964] [checkpoint] Fix externalized increme...

2017-07-14 Thread StefanRRichter
Github user StefanRRichter closed the pull request at: https://github.com/apache/flink/pull/4192 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #4192: [FLINK-6964] [checkpoint] Fix externalized increme...

2017-07-14 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4192#discussion_r127453343 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/IncrementalKeyedStateHandleTest.java --- @@ -187,7 +188,7 @@ public void

[GitHub] flink issue #4192: [FLINK-6964] [checkpoint] Fix externalized incremental ch...

2017-07-14 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4192 Thanks @aljoscha and @StephanEwen for the thorough review :-) I will merge this once Travis is green. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request #4192: [FLINK-6964] [checkpoint] Fix externalized increme...

2017-07-14 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4192#discussion_r127431267 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/IncrementalKeyedStateHandleTest.java --- @@ -187,7 +188,7 @@ public void

[GitHub] flink issue #4192: [FLINK-6964] [checkpoint] Fix externalized incremental ch...

2017-07-13 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4192 @aljoscha @StephanEwen I have added an IT case for this problem. It is testing a sequence externalized checkpoint recoveries, using full and incremental checkpoints on standalone

[GitHub] flink pull request #4192: [FLINK-6964] [checkpoint] Fix externalized increme...

2017-07-11 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4192#discussion_r126725696 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/savepoint/SavepointV2Serializer.java --- @@ -391,7 +392,7 @@ private

[GitHub] flink pull request #4192: [FLINK-6964] [checkpoint] Fix externalized increme...

2017-07-11 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4192#discussion_r126725439 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZooKeeperCompletedCheckpointStoreTest.java --- @@ -160,9 +162,12 @@ public

[GitHub] flink issue #4246: [FLINK-7063] [checkpoint] Call super.cancel(...) before c...

2017-07-10 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4246 Thanks for the review @NicoK ! Merged in 74adb84742. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request #4246: [FLINK-7063] [checkpoint] Call super.cancel(...) b...

2017-07-10 Thread StefanRRichter
Github user StefanRRichter closed the pull request at: https://github.com/apache/flink/pull/4246 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4130: [FLINK-6773] [checkpoint] Introduce compression (snappy) ...

2017-07-04 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4130 Merged in 5171513a3c. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #4130: [FLINK-6773] [checkpoint] Introduce compression (s...

2017-07-04 Thread StefanRRichter
Github user StefanRRichter closed the pull request at: https://github.com/apache/flink/pull/4130 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4130: [FLINK-6773] [checkpoint] Introduce compression (snappy) ...

2017-07-04 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4130 Thanks for the review @tillrohrmann! I will address your comments and proceed with merging. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink issue #4213: [FLINK-7032] Overwrite inherited properties from parent p...

2017-07-03 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4213 I observed similar problems as @pnowojski and the fix helped for me, so +1 from my side. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink issue #4246: [FLINK-7063] [checkpoint] Call super.cancel(...) before c...

2017-07-03 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4246 CC @NicoK --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #4246: [FLINK-7063] [checkpoint] Call super.cancel(...) b...

2017-07-03 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/4246 [FLINK-7063] [checkpoint] Call super.cancel(...) before closing strea… Minor fix that has only consequences in exception reporting. With this change, the future is first set to canceled

[GitHub] flink pull request #4192: [FLINK-6964] [checkpoint] Fix externalized increme...

2017-06-27 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/4192 [FLINK-6964] [checkpoint] Fix externalized incremental checkpoints fo… This PR fixes a problem that came up with incremental checkpoints and the `StandaloneCompletedCheckpointStore`. After

[GitHub] flink issue #4130: [FLINK-6773] [checkpoint] Introduce compression (snappy) ...

2017-06-15 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4130 CC @tillrohrmann --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] flink pull request #4130: [FLINK-6773] [checkpoint] Introduce compression (s...

2017-06-15 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/4130 [FLINK-6773] [checkpoint] Introduce compression (snappy) for keyed st… This PR introduce optional snappy compression for the keyed state in full checkpoints and savepoints. This feature

[GitHub] flink pull request #:

2017-06-15 Thread StefanRRichter
Github user StefanRRichter commented on the pull request: https://github.com/apache/flink/commit/9141379f6d2654886d48154b453170cc23b89a87#commitcomment-22558757 In flink-runtime/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java: In flink-runtime/src/main

[GitHub] flink pull request #4019: [FLINK-6776] [runtime] Use skip instead of seek fo...

2017-06-15 Thread StefanRRichter
Github user StefanRRichter closed the pull request at: https://github.com/apache/flink/pull/4019 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #4019: [FLINK-6776] [runtime] Use skip instead of seek for small...

2017-06-15 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4019 Merged in 9141379f6d. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink issue #4019: [FLINK-6776] [runtime] Use skip instead of seek for small...

2017-06-13 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4019 Updated with comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #4019: [FLINK-6776] [runtime] Use skip instead of seek fo...

2017-06-13 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4019#discussion_r121632647 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java --- @@ -89,4 +99,14 @@ public long skip(long n

[GitHub] flink issue #4108: [FLINK-6685] Adjust scopes of SafetyNetCloseableRegistry ...

2017-06-13 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4108 @tillrohrmann thanks for the review and the merge! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request #4019: [FLINK-6776] [runtime] Use skip instead of seek fo...

2017-06-13 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4019#discussion_r121630151 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java --- @@ -31,11 +31,15 @@ */ public final

[GitHub] flink issue #4073: [FLINK-6804] [state] Consistent state migration behaviour...

2017-06-12 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4073 Very good work. I think the properly addresses the mentioned issues. +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink issue #4107: [FLINK-6899] [state] Create correctly sized state array i...

2017-06-12 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4107 LGTM +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #4108: [FLINK-6685] Adjust scopes of SafetyNetCloseableRe...

2017-06-12 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/4108 [FLINK-6685] Adjust scopes of SafetyNetCloseableRegistry usage This PR addresses remaining issues with the `SafetyNetCloseableregistry`: - reset the registry to `null` after usage

[GitHub] flink issue #4090: [FLINK-6869] [scala] Specify serialVersionUID for all Sca...

2017-06-08 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4090 Overall, I think this is ok as a best effort until we have some eager registration that helps with the remaining problems in the heap backend. --- If your project is set up for it, you can

[GitHub] flink issue #4058: [FLINK-6833] [task] Fail StreamTask only due to async exc...

2017-06-08 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4058 LGTM +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #4058: [FLINK-6833] [task] Fail StreamTask only due to as...

2017-06-06 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4058#discussion_r120291185 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java --- @@ -824,11 +824,14 @@ public

[GitHub] flink issue #4011: [FLINK-6766] Update documentation about async backends an...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4011 Thanks for the input @alpinegizmo ! I updated the text accordingly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink pull request #4026: [FLINK-6764] Deduplicate stateless serializers in ...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4026#discussion_r119364516 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/CompositeTypeSerializerConfigSnapshot.java --- @@ -110,4 +123,20 @@ public

[GitHub] flink issue #4026: [FLINK-6764] Deduplicate stateless serializers in checkpo...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4026 Overall, very nice work! I just had few minor comments. After they are addressed, this is good to merge. +1 --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink issue #4026: [FLINK-6764] Deduplicate stateless serializers in checkpo...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4026 Overall, very nice work! I just had few minor comments. After they are addressed, this is good to merge. +1 --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink pull request #4026: [FLINK-6764] Deduplicate stateless serializers in ...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4026#discussion_r119359879 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/CompositeTypeSerializerConfigSnapshot.java --- @@ -110,4 +123,20 @@ public

[GitHub] flink pull request #4026: [FLINK-6764] Deduplicate stateless serializers in ...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4026#discussion_r119358003 --- Diff: flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/PojoSerializer.java --- @@ -795,13 +782,15 @@ public

[GitHub] flink pull request #4026: [FLINK-6764] Deduplicate stateless serializers in ...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4026#discussion_r119357811 --- Diff: flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/PojoSerializer.java --- @@ -795,13 +782,15 @@ public

[GitHub] flink pull request #4026: [FLINK-6764] Deduplicate stateless serializers in ...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4026#discussion_r119348235 --- Diff: flink-core/src/test/java/org/apache/flink/api/java/typeutils/runtime/PojoSerializerTest.java --- @@ -606,6 +665,45 @@ public void

[GitHub] flink pull request #4026: [FLINK-6764] Deduplicate stateless serializers in ...

2017-05-31 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/4026#discussion_r119348073 --- Diff: flink-core/src/test/java/org/apache/flink/api/java/typeutils/runtime/PojoSerializerTest.java --- @@ -606,6 +665,45 @@ public void

[GitHub] flink pull request #4019: [FLINK-6776] [runtime] Use skip instead of seek fo...

2017-05-30 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/4019 [FLINK-6776] [runtime] Use skip instead of seek for small forward rep… …ositioning in DFS streams Reading checkpoint meta data and finding key-groups in restores sometimes

[GitHub] flink issue #4011: [FLINK-6766] Update documentation about async backends an...

2017-05-30 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/4011 CC @rmetzger --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #4011: [FLINK-6766] Update documentation about async back...

2017-05-29 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/4011 [FLINK-6766] Update documentation about async backends and incrementa… …l checkpoints This PR introduces some documentation about async heap backends and incremental snapshots

[GitHub] flink issue #3937: [FLINK-6482] [core] Add nested serializers to config snap...

2017-05-22 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3937 Overall, LGTM. +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] flink pull request #3937: [FLINK-6482] [core] Add nested serializers to conf...

2017-05-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3937#discussion_r117710523 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/CompositeTypeSerializerConfigSnapshot.java --- @@ -19,47 +19,65

[GitHub] flink pull request #3937: [FLINK-6482] [core] Add nested serializers to conf...

2017-05-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3937#discussion_r117710279 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/CompositeTypeSerializerConfigSnapshot.java --- @@ -19,47 +19,65

[GitHub] flink pull request #3937: [FLINK-6482] [core] Add nested serializers to conf...

2017-05-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3937#discussion_r117697041 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/CompositeTypeSerializerConfigSnapshot.java --- @@ -73,13 +91,11 @@ public

[GitHub] flink pull request #3937: [FLINK-6482] [core] Add nested serializers to conf...

2017-05-22 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3937#discussion_r117696993 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/CompositeTypeSerializerConfigSnapshot.java --- @@ -73,13 +91,11 @@ public

[GitHub] flink issue #3939: [FLINK-6612] Allow ZooKeeperStateHandleStore to lock crea...

2017-05-19 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3939 Thanks for your work @tillrohrmann ! I merged this manually. Please close this PR and the jira. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink issue #3939: [FLINK-6612] Allow ZooKeeperStateHandleStore to lock crea...

2017-05-18 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3939 LGMT +1. Tested that this is running with incremental checkpoints, however the test did not (yet) cover a "split brain" scenario with to JobManagers running in parallel. --- If yo

[GitHub] flink issue #3925: [FLINK-6600] Add key serializer config snapshot to keyed ...

2017-05-17 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3925 Thanks for adding this so quickly @tzulitai ! The changes look good to me +1. @tillrohrmann maybe you can already include this for your testing. --- If your project is set up

[GitHub] flink pull request #3925: [FLINK-6600] Add key serializer config snapshot to...

2017-05-17 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3925#discussion_r116977867 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapKeyedStateBackend.java --- @@ -385,6 +386,19 @@ private void

[GitHub] flink pull request #3925: [FLINK-6600] Add key serializer config snapshot to...

2017-05-17 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3925#discussion_r116977261 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

[GitHub] flink pull request #3925: [FLINK-6600] Add key serializer config snapshot to...

2017-05-17 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3925#discussion_r116977075 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

[GitHub] flink issue #3912: [FLINK-6589] [core] Deserialize ArrayList with capacity o...

2017-05-17 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3912 Yes, there is a tradeoff, and probably the nicest way would be to restore the internal array size as it was when we serialized the original. However, I guess that is too much effort

[GitHub] flink issue #3912: [FLINK-6589] [core] Deserialize ArrayList with capacity o...

2017-05-17 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3912 No concern, just curious if there is any reason for the 1 additional. The problem also still exists now for a second element. So is it, in general, unlikely that new elements are added

[GitHub] flink pull request #3888: [FLINK-6020] Introduce BlobServer#readWriteLock to...

2017-05-15 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3888#discussion_r116446914 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobServerConnection.java --- @@ -235,8 +235,54 @@ else if (contentAddressable

[GitHub] flink issue #3884: [FLINK-6284] Correct sorting of completed checkpoints in ...

2017-05-15 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3884 LGTM +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #3870: [Flink 6537] Fixes and improvements for incrementa...

2017-05-14 Thread StefanRRichter
Github user StefanRRichter closed the pull request at: https://github.com/apache/flink/pull/3870 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #3870: [Flink 6537] Fixes and improvements for incremental check...

2017-05-14 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3870 Thanks @rmetzger ! Merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink issue #3882: [FLINK-6565] Fail memory-backed state restores with meani...

2017-05-12 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3882 LGTM +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #3886: [FLINK-6554] [core] Make CompatibilityResult options more...

2017-05-12 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3886 LGTM +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #3883: [FLINK-6566] [core] More restricted interface for ...

2017-05-12 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3883#discussion_r116249086 --- Diff: flink-core/src/main/java/org/apache/flink/core/io/VersionedIOReadableWritable.java --- @@ -18,55 +18,66 @@ package

[GitHub] flink issue #3870: [Flink 6537] Fixes and improvements for incremental check...

2017-05-12 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3870 Updated and enhanced. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #3870: [Flink 6537] Fixes and improvements for incrementa...

2017-05-12 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3870#discussion_r116179467 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/SharedStateRegistry.java --- @@ -18,91 +18,137 @@ package

[GitHub] flink pull request #3870: [Flink 6537] Fixes and improvements for incrementa...

2017-05-12 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3870#discussion_r116175297 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBIncrementalKeyedStateHandle.java

[GitHub] flink pull request #3870: [Flink 6537] Fixes and improvements for incrementa...

2017-05-12 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3870#discussion_r116174101 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBIncrementalKeyedStateHandle.java

[GitHub] flink issue #3870: [Flink 6537] Fixes and improvements for incremental check...

2017-05-11 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3870 @gyfora Yes, this is still a limitation, because the incremental checkpoints are currently not (yet) externalizable. The missing piece is, exactly as you said, familiarizing

[GitHub] flink issue #3870: [Flink 6537] Fixes and improvements for incremental check...

2017-05-11 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3870 CC @shixiaogang @StephanEwen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request #3870: [Flink 6537] Fixes and improvements for incrementa...

2017-05-11 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/3870 [Flink 6537] Fixes and improvements for incremental checkpoints in RocksDB This PR bundles several fixes and improvements for incremental checkpoints in RocksDB. In particular

[GitHub] flink pull request #3859: [FLINK-6504] [FLINK-6467] [checkpoints] Add needed...

2017-05-10 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3859#discussion_r115721158 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

[GitHub] flink pull request #3859: [FLINK-6504] [FLINK-6467] [checkpoints] Add needed...

2017-05-10 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3859#discussion_r115719033 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

[GitHub] flink pull request #3859: [FLINK-6504] [FLINK-6467] [checkpoints] Add needed...

2017-05-10 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3859#discussion_r115714053 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

[GitHub] flink issue #3853: [FLINK-6509] [tests] Perform TestingListener#waitForNewLe...

2017-05-09 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3853 LGMT +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #3837: [FLINK-6471] [checkpoint] Fix RocksDBStateBackendTest

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3837 Merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink pull request #3837: [FLINK-6471] [checkpoint] Fix RocksDBStateBackendT...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter closed the pull request at: https://github.com/apache/flink/pull/3837 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] flink issue #3839: FLINK-6474 Potential loss of precision in 32 bit integer ...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3839 Merged manually in 38003c2829. Please close this PR and the jira. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] flink issue #3839: FLINK-6474 Potential loss of precision in 32 bit integer ...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3839 LGTM +1, will merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] flink pull request #3837: [FLINK-6471] [checkpoint] Fix RocksDBStateBackendT...

2017-05-07 Thread StefanRRichter
GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/3837 [FLINK-6471] [checkpoint] Fix RocksDBStateBackendTest `RocksDBStateBackendTest::testCancelRunningSnapshot` sometimes failes. This is cause by a problem that I fixed

[GitHub] flink issue #3834: [FLINK-6425] [runtime] Activate serializer upgrades in st...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3834 I was already familiar with the high level design of this PR from my discussions with @tzulitai. As the feature freeze is already tomorrow, I had to focus my review on the functionality

[GitHub] flink issue #3522: [FLINK-5823] [checkpoints] State Backends also handle Che...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3522 I agree that we can change the Strings to some proper types later. Besides what I mentioned before, I just thought that even if a string is the underlying data type for the moment, having

[GitHub] flink pull request #3834: [FLINK-6425] [runtime] Activate serializer upgrade...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3834#discussion_r115139530 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/TypeSerializer.java --- @@ -161,7 +162,93 @@ public abstract

[GitHub] flink pull request #3834: [FLINK-6425] [runtime] Activate serializer upgrade...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3834#discussion_r115139440 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/TypeSerializer.java --- @@ -161,7 +162,93 @@ public abstract

[GitHub] flink pull request #3834: [FLINK-6425] [runtime] Activate serializer upgrade...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3834#discussion_r115139379 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/MigrationStrategy.java --- @@ -0,0 +1,87 @@ +/* + * Licensed

[GitHub] flink pull request #3834: [FLINK-6425] [runtime] Activate serializer upgrade...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3834#discussion_r115139339 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

[GitHub] flink pull request #3834: [FLINK-6425] [runtime] Activate serializer upgrade...

2017-05-07 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3834#discussion_r115139044 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

[GitHub] flink issue #3801: [FLINK-6364] [checkpoints] Implement incremental checkpoi...

2017-05-05 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3801 I merged your changes. Please close this PR and the JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink issue #3801: [FLINK-6364] [checkpoints] Implement incremental checkpoi...

2017-05-05 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3801 I am sorry, but before merging I noticed that some tests (e.g. `RocksDBStateBackendTest.testCancelRunningSnapshot`) fail sporadically (only on Travis). I tracked the problem and I think

[GitHub] flink issue #3801: [FLINK-6364] [checkpoints] Implement incremental checkpoi...

2017-05-04 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3801 @shixiaogang This looks good for me, I will go ahead and merge this. Thanks for your work! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink pull request #3801: [FLINK-6364] [checkpoints] Implement incremental c...

2017-05-03 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3801#discussion_r114580123 --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java --- @@ -769,9 +769,10 @@ public

[GitHub] flink pull request #3801: [FLINK-6364] [checkpoints] Implement incremental c...

2017-05-03 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3801#discussion_r114579898 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateHandle.java --- @@ -0,0

[GitHub] flink issue #3813: [FLINK-6427] Ensure file length is flushed in StreamWrite...

2017-05-03 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3813 No objections from my side. LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request #3801: [FLINK-6364] [checkpoints] Implement incremental c...

2017-05-03 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3801#discussion_r114504710 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

[GitHub] flink pull request #3801: [FLINK-6364] [checkpoints] Implement incremental c...

2017-05-02 Thread StefanRRichter
Github user StefanRRichter commented on a diff in the pull request: https://github.com/apache/flink/pull/3801#discussion_r114380645 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java

<    5   6   7   8   9   10   11   12   13   14   >