pnowojski commented on a change in pull request #8443: [FLINK-12510][network]
Fix deadlock in InputGates
URL: https://github.com/apache/flink/pull/8443#discussion_r284185496
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/UnionInputGate.java
##########
@@ -210,12 +211,16 @@ public void requestPartitions() throws IOException,
InterruptedException {
}
Iterator<InputGate> inputGateIterator =
inputGatesWithData.iterator();
- final InputGate inputGate =
inputGateIterator.next();
+ inputGate = inputGateIterator.next();
inputGateIterator.remove();
+ }
- // In case of inputGatesWithData being
inaccurate do not block on an empty inputGate, but just poll the data.
- Optional<BufferOrEvent> bufferOrEvent =
inputGate.pollNextBufferOrEvent();
+ // In case of inputGatesWithData being inaccurate do
not block on an empty inputGate, but just poll the data.
+ // Do not poll the gate under inputGatesWithData lock,
since this can trigger notifications
+ // that could deadlock because of wrong locks taking
order.
+ Optional<BufferOrEvent> bufferOrEvent =
inputGate.pollNextBufferOrEvent();
+ synchronized (inputGatesWithData) {
Review comment:
This was the old behaviour. I have concatenated those two critical sections
together in a hotfix/refactoring commit 😳
There should be no issue with this, since there are only two actors locking
on {{inputGatesWithData}}: one reader of the input gate, and the guy/guys
notifying that new data has arrived.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
