QiLuo-BD commented on a change in pull request #7186: [FLINK-10941] Keep slots
which contain unconsumed result partitions
URL: https://github.com/apache/flink/pull/7186#discussion_r239314285
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java
##########
@@ -134,10 +138,17 @@ public void cancel(InputChannelID receiverId) {
ctx.pipeline().fireUserEventTriggered(receiverId);
}
- public void close() {
+ public void close() throws IOException {
if (ctx != null) {
ctx.channel().close();
}
+
+ LOG.info("Close all {} readers pending for close.",
readersToClose.size());
Review comment:
Thanks Zhijiang. Agree with you that TM should only exit after all
connections are closed.
Regarding the side effect that 1 tail tasks out of 10 tasks will delay
releasing partitions, I think this may not be bad since the resource is managed
at TM level. If a TM cannot be released, it doesn't make too much difference
whether it has 1 partition or 10 partitions. Please correct me if anything
wrong.
Considering all existing limitations, I think the resource management could
be further improved via external shuffle service that you've proposed. This PR
may focus on solving current issue. Looking forward to your further comments. :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
