StefanRRichter commented on a change in pull request #8361:
[FLINK-12434][network] Replace listeners with CompletableFuture in InputGates
URL: https://github.com/apache/flink/pull/8361#discussion_r282061361
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/UnionInputGate.java
##########
@@ -197,32 +197,49 @@ public void requestPartitions() throws IOException,
InterruptedException {
return Optional.of(bufferOrEvent);
}
- @Override
- public Optional<BufferOrEvent> pollNextBufferOrEvent() throws
UnsupportedOperationException {
- throw new UnsupportedOperationException();
- }
-
- private InputGateWithData waitAndGetNextInputGate() throws IOException,
InterruptedException {
+ private Optional<InputGateWithData> waitAndGetNextInputGate(boolean
blocking) throws IOException, InterruptedException {
while (true) {
- InputGate inputGate;
- boolean moreInputGatesAvailable;
synchronized (inputGatesWithData) {
while (inputGatesWithData.size() == 0) {
- inputGatesWithData.wait();
+ if (blocking) {
+ inputGatesWithData.wait();
+ } else {
+ resetIsAvailable();
+ return Optional.empty();
+ }
+ }
+ final InputGate inputGate =
inputGatesWithData.remove();
+
+ // In case of inputGatesWithData being
inaccurate do not block on an empty inputGate, but just poll the data.
+ Optional<BufferOrEvent> bufferOrEvent =
inputGate.pollNextBufferOrEvent();
+
+ if (bufferOrEvent.isPresent() &&
bufferOrEvent.get().moreAvailable()) {
+ // enqueue the inputGate at the end to
avoid starvation
+ inputGatesWithData.add(inputGate);
+ } else {
+ inputGate.isAvailable().thenRun(() ->
queueInputGate(inputGate));
Review comment:
This cliend code might be able to squeeze out the fact that it can do `if
(inputGate.isAvailable() == AVAILABLE) {...run...}` and only has to do volatile
reads via `thenRun` in the other cases. Clearly makes the code a bit more ugly
and might go into a separate method or just be premature optimization :) If we
go for it, that should be part of the test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
