[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-08 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r159400371 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/SequenceNumberingViewReader.java --- @@ -77,6 +83,37 @@ void

[GitHub] flink pull request #5250: [FLINK-8207][network-tests] Unify TestInfiniteBuff...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5250 [FLINK-8207][network-tests] Unify TestInfiniteBufferProvider and TestPooledBufferProvider ## What is the purpose of the change `TestPooledBufferProvider` and `TestInfiniteBufferProvider

[GitHub] flink issue #5250: [FLINK-8207][network-tests] Unify TestInfiniteBufferProvi...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5250 test failures are unrelated ---

[GitHub] flink pull request #5251: [FLINK-8208][network-tests] Reduce mockito usage i...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5251 [FLINK-8208][network-tests] Reduce mockito usage in RecordWriterTest ## What is the purpose of the change Reduce mockito usage in `RecordWriterTest`. This PR is based on #5250

[GitHub] flink pull request #5252: [FLINK-8209][network-tests] Make LocalBufferPoolDe...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5252 [FLINK-8209][network-tests] Make LocalBufferPoolDestroyTest less implementation dependent ## What is the purpose of the change Make `LocalBufferPoolDestroyTest` less implementation dependent

[GitHub] flink pull request #5253: [FLINK-8210][network-tests] Collect results into p...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5253 [FLINK-8210][network-tests] Collect results into proper mock in MockEnvironment ## What is the purpose of the change Collect results into a proper mock in `MockEnvironment`. This PR is based

[GitHub] flink pull request #5254: [FLINK-8214][streaming-tests] Collect results into...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5254 [FLINK-8214][streaming-tests] Collect results into proper mock in StreamMockEnvironment ## What is the purpose of the change Collect results into proper mock in `StreamMockEnvironment`. This

[GitHub] flink pull request #5255: [FLINK-8221][network-benchmarks] Define latency ne...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5255 [FLINK-8221][network-benchmarks] Define latency network benchmarks in Flink project ## What is the purpose of the change This PR is based on #5134 and adds tools to setup simple micro

[GitHub] flink issue #4509: [FLINK-7406][network] Implement Netty receiver incoming p...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4509 merged in 268867ce620a2c12879749db2ecb68bbe129cad5 @zhijiangW can you close this PR? ---

[GitHub] flink issue #4533: [FLINK-7416][network] Implement Netty receiver outgoing p...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4533 merged in 1752fdb339df4e4d0a5063b24c460abdc0a44264 @zhijiangW can you close this PR? ---

[GitHub] flink issue #5250: [FLINK-8207][network-tests] Unify TestInfiniteBufferProvi...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5250 merged in 91c72b9dc611e73790147256adf859c177afe862 ---

[GitHub] flink pull request #5250: [FLINK-8207][network-tests] Unify TestInfiniteBuff...

2018-01-08 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/5250 ---

[GitHub] flink pull request #5251: [FLINK-8208][network-tests] Reduce mockito usage i...

2018-01-08 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/5251 ---

[GitHub] flink issue #5251: [FLINK-8208][network-tests] Reduce mockito usage in Recor...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5251 merged in 97db0bf9c1448a7e672f5d0235e301d03e1cf7d2 ---

[GitHub] flink pull request #5252: [FLINK-8209][network-tests] Make LocalBufferPoolDe...

2018-01-08 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/5252 ---

[GitHub] flink issue #5252: [FLINK-8209][network-tests] Make LocalBufferPoolDestroyTe...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5252 merged in 5cf37782f482f5dd51ad599a083eaa62dfcd805a ---

[GitHub] flink pull request #5253: [FLINK-8210][network-tests] Collect results into p...

2018-01-08 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/5253 ---

[GitHub] flink issue #5253: [FLINK-8210][network-tests] Collect results into proper m...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5253 merged in af6bdb606e825d0d66ba532bcb9d8335f9f4c54b ---

[GitHub] flink issue #5254: [FLINK-8214][streaming-tests] Collect results into proper...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5254 merged in 0888bb622e275ac6ff2408c2ae5014fd787b5dbd ---

[GitHub] flink pull request #5254: [FLINK-8214][streaming-tests] Collect results into...

2018-01-08 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/5254 ---

[GitHub] flink issue #5105: [FLINK-8178][network] Introduce not threadsafe write only...

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5105 merged in c6945c2ef48d4c2cad3fc935435c1ab83e834969 @pnowojski can you close this PR? ---

[GitHub] flink issue #5134: [FLINK-8220] Implement set of network benchmarks

2018-01-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5134 merged in c816191113d813156467f3e33856636ef0bcce38 @pnowojski can you close this PR? ---

[GitHub] flink pull request #5259: [FLINK-8252][benchmarks] convert network benchmark...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5259 [FLINK-8252][benchmarks] convert network benchmarks to streaming benchmarks ## What is the purpose of the change This converts the benchmarks created with #5134 and #5255 into proper

[GitHub] flink pull request #5260: [FLINK-8375][network] Remove unnecessary synchroni...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5260 [FLINK-8375][network] Remove unnecessary synchronization ## What is the purpose of the change Synchronized blocks in ResultPartition could affect only: 1. totalNumberOfBuffers and

[GitHub] flink pull request #5261: [FLINK-8371][network] always recycle Buffers when ...

2018-01-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5261 [FLINK-8371][network] always recycle Buffers when releasing SpillableSubpartition ## What is the purpose of the change There were places where `Buffer` instances were not released upon

[GitHub] flink pull request #5261: [FLINK-8371][network] always recycle Buffers when ...

2018-01-09 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5261#discussion_r160349800 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartition.java --- @@ -52,6 +55,10 @@ /** Flag

[GitHub] flink pull request #5176: [FLINK-8279][blob] fall back to TaskManager temp d...

2018-01-09 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5176#discussion_r160353839 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobUtils.java --- @@ -127,21 +132,28 @@ private static BlobStoreService

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-09 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r160361957 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -88,6 +93,35 @@ public void run

[GitHub] flink pull request #5186: [FLINK-8288] [runtime] register job master rest en...

2018-01-09 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5186#discussion_r160368484 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/JobClusterEntrypoint.java --- @@ -132,6 +122,16 @@ protected void

[GitHub] flink pull request #5275: [FLINK-7499][io] fix double buffer release in Spil...

2018-01-10 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5275 [FLINK-7499][io] fix double buffer release in SpillableSubpartitionView ## What is the purpose of the change This is a rebase of #4581 for the release-1.4 branch. Please refer to the

[GitHub] flink pull request #5276: [FLINK-8371][network] always recycle Buffers when ...

2018-01-10 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5276 [FLINK-8371][network] always recycle Buffers when releasing SpillableSubpartition ## What is the purpose of the change This is a rebase of #5261 for the release-1.4 branch. Please refer to

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r160714424 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -250,10 +304,12 @@ private void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r160694722 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -88,6 +94,37 @@ public void run

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r160695373 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -98,20 +135,35 @@ public void close

[GitHub] flink pull request #5275: [FLINK-7499][io] fix double buffer release in Spil...

2018-01-11 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/5275 ---

[GitHub] flink issue #5276: [FLINK-8371][network] always recycle Buffers when releasi...

2018-01-11 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5276 closed via 71ede3992afe8f6907dd3c6c1e232c5b745048b4 ---

[GitHub] flink issue #5275: [FLINK-7499][io] fix double buffer release in SpillableSu...

2018-01-11 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5275 closed via a316989e5dfbb1dc0d555193425a4d6bd5f42d8d ---

[GitHub] flink pull request #5276: [FLINK-8371][network] always recycle Buffers when ...

2018-01-11 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/5276 ---

[GitHub] flink pull request #5288: [FLINK-8395][network] add a read-only sliced ByteB...

2018-01-12 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5288 [FLINK-8395][network] add a read-only sliced ByteBuf implementation based on NetworkBuffer ## What is the purpose of the change To insert the same buffer (or sub-regions from it) multiple

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161242036 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/SequenceNumberingViewReader.java --- @@ -49,10 +50,24

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161295951 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionView.java --- @@ -199,6 +199,19 @@ public boolean

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161287607 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/SequenceNumberingViewReader.java --- @@ -77,6 +92,46 @@ void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161249082 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionView.java --- @@ -199,6 +199,19 @@ public boolean

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161295428 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpilledSubpartitionView.java --- @@ -163,6 +190,15 @@ public boolean

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161282684 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/SequenceNumberingViewReader.java --- @@ -77,6 +92,46 @@ void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161248402 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultSubpartitionView.java --- @@ -52,4 +52,9 @@ boolean

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161247589 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/SequenceNumberingViewReader.java --- @@ -77,6 +92,46 @@ void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161241148 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/SequenceNumberingViewReader.java --- @@ -49,10 +50,24

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161296711 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpilledSubpartitionView.java --- @@ -73,6 +73,9 @@ /** Flag

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161545040 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -277,23 +277,31 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161546199 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -372,16 +379,18 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161485403 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -434,6 +443,29 @@ private

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161559913 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +77,95 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161565896 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +78,142 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161547041 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +77,95 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161559690 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +77,95 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161567331 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +78,142 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161559642 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +77,95 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161573976 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionTest.java --- @@ -238,6 +238,7 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161569135 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartitionTest.java --- @@ -134,13 +135,22 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161578518 --- Diff: flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java --- @@ -59,7 +59,7 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161559926 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +77,95 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161546145 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -372,16 +379,18 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161567305 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +78,142 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161570121 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartitionTest.java --- @@ -119,6 +119,7 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161565565 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueueTest.java --- @@ -71,4 +78,142 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161576253 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/taskmanager/TaskCancelAsyncProducerConsumerITCase.java --- @@ -84,7 +84,7 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r161568012 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartitionTest.java --- @@ -119,6 +119,7 @@ public void

[GitHub] flink issue #4552: [FLINK-7456][network] Implement Netty sender incoming pip...

2018-01-15 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4552 one thing which we talked about offline: as a precaution, we should keep the old implementation around and allow the users to basically turn the credit-based flow control algorithm on/off (the

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-17 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r162047076 --- Diff: flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java --- @@ -59,7 +59,7 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-17 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r162106371 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestServerHandler.java --- @@ -82,10 +83,17 @@ protected void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-17 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r162104968 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -125,11 +126,11 @@ private void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-17 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r162103769 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java --- @@ -164,11 +165,13 @@ private boolean

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-17 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r162107348 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpilledSubpartitionView.java --- @@ -31,7 +31,6 @@ import

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r162323707 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpilledSubpartitionView.java --- @@ -31,7 +31,6 @@ import

[GitHub] flink pull request #5314: [FLINK-8425][network] fix SpilledSubpartitionView ...

2018-01-18 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5314 [FLINK-8425][network] fix SpilledSubpartitionView not protected against concurrent release calls ## What is the purpose of the change It seems like `SpilledSubpartitionView` is not protected

[GitHub] flink issue #4552: [FLINK-7456][network] Implement Netty sender incoming pip...

2018-01-19 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4552 looks good - Let's start some cluster tests and then we're ready to merge ---

[GitHub] flink issue #5317: [FLINK-8458] Add the switch for keeping both the old mode...

2018-02-20 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5317 @zhijiangW can you rebase the new changes? The previous implementation got merged and from the comments I see here, there might have been some changes since then which should be applied as well. ---

[GitHub] flink pull request #5549: [FLINK-8733][network] fix SpillableSubpartition#sp...

2018-02-21 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5549 [FLINK-8733][network] fix SpillableSubpartition#spillFinishedBufferConsumers() not counting spilled bytes ## What is the purpose of the change With [FLINK-8583], `SpillableSubpartition` lost

[GitHub] flink pull request #5550: [FLINK-8734][network] fix partition bytes counting...

2018-02-21 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5550 [FLINK-8734][network] fix partition bytes counting and re-enable in tests ## What is the purpose of the change With [FLINK-8583], `SpillableSubpartition` does not update byte statistics

[GitHub] flink pull request #5551: [FLINK-8736][network] fix memory segment offsets f...

2018-02-21 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5551 [FLINK-8736][network] fix memory segment offsets for slices of slices being wrong ## What is the purpose of the change [FLINK-8588] introduced memory segment offsets to the `Buffer` classes

[GitHub] flink issue #5557: [FLINK-8694][runtime] Workaround notifyDataAvailable race...

2018-02-22 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5557 While this may certainly "fix" the problem with the flush-always setting, I don't like the approach of ignoring the problem that some `notifyDataAvailable` calls are ignored and we h

[GitHub] flink pull request #5558: [FLINK-8747][bugfix] The tag of waiting for floati...

2018-02-22 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5558#discussion_r170028724 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java --- @@ -396,32 +399,49 @@ public

[GitHub] flink pull request #5558: [FLINK-8747][bugfix] The tag of waiting for floati...

2018-02-22 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5558#discussion_r170027206 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -337,6 +337,11 @@ public int

[GitHub] flink issue #5557: [FLINK-8694][runtime] Workaround notifyDataAvailable race...

2018-02-22 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5557 We can still merge bug fixes after feature freeze. The original cause is also bug but only influencing latency except for the always-flush case. ---

[GitHub] flink issue #5549: [FLINK-8733][network] fix SpillableSubpartition#spillFini...

2018-02-22 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5549 I would actually like to not implement a test since the value itself is only printed to the log anyway and not exposed in any other way. ---

[GitHub] flink issue #5550: [FLINK-8734][network] fix partition bytes counting and re...

2018-02-22 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5550 Since the tests go through various different scenarios, that's the natural place to also verify statistics which should be aligned with the real world despite the overhead during changes.

[GitHub] flink issue #5551: [FLINK-8736][network] fix memory segment offsets for slic...

2018-02-22 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5551 Honestly, there is no way, the tests' deadlocks are related to the changes of this PR. ---

[GitHub] flink issue #4529: [FLINK-7428][network] avoid buffer copies when receiving ...

2018-02-23 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4529 With the recent update of the netty version in flink-shaded (see https://github.com/apache/flink-shaded/commit/1233f1bb0e2b9fafa4260603aa130b7eb9995a7a), most of this PR is indirectly included

[GitHub] flink issue #5549: [FLINK-8733][network] fix SpillableSubpartition#spillFini...

2018-02-23 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5549 alright, there you go... but with your definition of dead code, I could easily remove some hundred lines of code :P ---

[GitHub] flink pull request #5570: [FLINK-8768][network] Let NettyMessageDecoder inhe...

2018-02-23 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5570 [FLINK-8768][network] Let NettyMessageDecoder inherit from LengthFieldBasedFrameDecoder ## What is the purpose of the change Instead of being two steps in the channel pipeline

[GitHub] flink pull request #5571: [FLINK-8759][network] preparations for the update ...

2018-02-23 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5571 [FLINK-8759][network] preparations for the update of netty to version 4.0.56 ## What is the purpose of the change Based on the changes from #5570, this PR prepares a netty version bump by

[GitHub] flink pull request #5572: [FLINK-8694][runtime] Fix notifyDataAvailable race...

2018-02-26 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5572#discussion_r170562983 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartition.java --- @@ -208,7 +208,7 @@ public

[GitHub] flink pull request #5572: [FLINK-8694][runtime] Fix notifyDataAvailable race...

2018-02-26 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5572#discussion_r170560655 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartition.java --- @@ -48,6 +48,8 @@ /** Flag

[GitHub] flink pull request #5572: [FLINK-8694][runtime] Fix notifyDataAvailable race...

2018-02-26 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5572#discussion_r170590385 --- Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkThroughputBenchmarkTests.java --- @@ -52,6 +52,14

[GitHub] flink pull request #5572: [FLINK-8694][runtime] Fix notifyDataAvailable race...

2018-02-26 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5572#discussion_r170589453 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionTest.java --- @@ -319,7 +319,7 @@ public void

[GitHub] flink pull request #5572: [FLINK-8694][runtime] Fix notifyDataAvailable race...

2018-02-26 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5572#discussion_r170585362 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/iterative/task/IterationHeadTask.java --- @@ -447,6 +447,8 @@ private void

[GitHub] flink pull request #5572: [FLINK-8694][runtime] Fix notifyDataAvailable race...

2018-02-26 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5572#discussion_r170560833 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartition.java --- @@ -65,9 +67,8 @@ public boolean add

[GitHub] flink pull request #5572: [FLINK-8694][runtime] Fix notifyDataAvailable race...

2018-02-26 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5572#discussion_r170557786 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/SingleInputGateTest.java --- @@ -428,13 +455,15 @@ private

[GitHub] flink pull request #5572: [FLINK-8694][runtime] Fix notifyDataAvailable race...

2018-02-26 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5572#discussion_r170574788 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionView.java --- @@ -242,6 +239,22 @@ public boolean

<    5   6   7   8   9   10   11   >