zhijiangW commented on a change in pull request #8654: [FLINK-12647][network]
Add feature flag to disable release of consumed blocking partitions
URL: https://github.com/apache/flink/pull/8654#discussion_r293329355
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultPartitionManager.java
##########
@@ -110,14 +124,20 @@ void onConsumedPartition(ResultPartition partition) {
LOG.debug("Received consume notification from {}.", partition);
synchronized (registeredPartitions) {
- final ResultPartition previous =
registeredPartitions.remove(partition.getPartitionId());
- // Release the partition if it was successfully removed
- if (partition == previous) {
- partition.release();
- ResultPartitionID partitionId =
partition.getPartitionId();
- LOG.debug("Released partition {} produced by
{}.",
- partitionId.getPartitionId(),
partitionId.getProducerId());
- }
+ registeredPartitions.computeIfPresent(
+ partition.getPartitionId(),
+ (resultPartitionID, resultPartition) -> {
+ if (partition == resultPartition) {
+ if
(partition.isManagedExternally() &&
!isReleaseExternallyManagedPartitionsOnConsumption) {
Review comment:
I have another option for this. ATM both `ResultPartition` and
`ResultPartitionManager` need check the conditions of releasing partitions
consumed, then the logic seems difficult to trace.
We could make `ResultPartitionManager` not aware of this. The
`isReleaseExternallyManagedPartitionsOnConsumption` could be merged with
`partition.isManagedExternally()` to generate one final boolean tag in
`ResultPartition`. Then the `ResultPartition` checks this tag to decide whether
to call `partitionManager.onConsumedPartition` while one sub partition is
consumed. The logic in `ResultPartitionManager` would be simple:
- Based on consumption release: The behavior is the same as before after
calling `ResultPartitionManger#onConsumedPartition`, remove from map and
release directly.
- Based on external release: via `ResultPartitionManager#releasePartition`.
And we only have one boolean tag in `ResultPartition` to handle the external
check. WDYT?
This suggestion is not relevant with pin issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
