[
https://issues.apache.org/jira/browse/FLINK-7406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196772#comment-16196772
]
ASF GitHub Bot commented on FLINK-7406:
---------------------------------------
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4509#discussion_r143425927
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java
---
@@ -434,6 +435,52 @@ public void testWaitForFloatingBuffersOnBuffer()
throws Exception {
verify(bufferPool, times(1)).requestBuffer();
}
+ /**
+ * Tests to verify that there is no race condition with two things
running in parallel:
+ * requesting floating buffers and some other thread recycling them.
+ */
+ @Test
+ public void testConcurrentRequestBufferAndNotifyBufferAvailable()
throws Exception {
+ // Setup
+ final ExecutorService executor =
Executors.newFixedThreadPool(1);
+ final Buffer buffer = TestBufferFactory.createBuffer();
+ final BufferPool bufferPool = mock(BufferPool.class);
+ when(bufferPool.requestBuffer()).thenReturn(null);
+
when(bufferPool.addBufferListener(any(BufferListener.class))).thenReturn(true);
+
+ final SingleInputGate inputGate = mock(SingleInputGate.class);
+ when(inputGate.getBufferPool()).thenReturn(bufferPool);
+ when(inputGate.getBufferProvider()).thenReturn(bufferPool);
+ try {
+ final RemoteInputChannel inputChannel =
createRemoteInputChannel(inputGate);
+ // Trigger to request one floating buffer on sender
backlog
+ inputChannel.onBuffer(buffer, 0, 1);
+
+ final CountDownLatch sync = new CountDownLatch(1);
+
+ // Submit task and wait to finish
+ Future<Void> result = executor.submit(new
Callable<Void>() {
+ @Override
+ public Void call() throws Exception {
+
inputChannel.notifyBufferAvailable(buffer);
+ sync.countDown();
+
+ return null;
+ }
+ });
--- End diff --
This will not run the code in parallel - `onBuffer()` is always executed
before `inputChannel.notifyBufferAvailable(buffer);`.
Among the tests I recently stumbled upon,
`BlobServerGetTest#testConcurrentGetOperations()` may be a good base to start a
new concurrency test with.
> Implement Netty receiver incoming pipeline for credit-based
> -----------------------------------------------------------
>
> Key: FLINK-7406
> URL: https://issues.apache.org/jira/browse/FLINK-7406
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Reporter: zhijiang
> Assignee: zhijiang
> Fix For: 1.4.0
>
>
> This is a part of work for credit-based network flow control.
> Currently {{PartitionRequestClientHandler}} receives and reads
> {{BufferResponse}} from producer. It will request buffer from {{BufferPool}}
> for holding the message. If not got, the message is staged temporarily and
> {{autoread}} for channel is set false.
> For credit-based mode, {{PartitionRequestClientHandler}} can always get
> buffer from {{RemoteInputChannel}} for reading messages from producer.
> The related works are:
> * Add the backlog of producer in {{BufferResponse}} message structure
> * {{PartitionRequestClientHandler}} requests buffer from
> {{RemoteInputChannel}} directly
> * {{PartitionRequestClientHandler}} updates backlog for
> {{RemoteInputChannel}}, and it may trigger requests of floating buffers from
> {{BufferPool}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
