azagrebin commented on a change in pull request #9693: [FLINK-13984] Separate
on-heap and off-heap managed memory pools
URL: https://github.com/apache/flink/pull/9693#discussion_r326675307
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/memory/MemoryManager.java
##########
@@ -394,74 +318,79 @@ public void release(Collection<MemorySegment> segments) {
return;
}
- // -------------------- BEGIN CRITICAL SECTION
-------------------
- synchronized (lock) {
- if (isShutDown) {
- throw new IllegalStateException("Memory manager
has been shut down.");
- }
+ Preconditions.checkState(!isShutDown, "Memory manager has been
shut down.");
- // since concurrent modifications to the collection
- // can disturb the release, we need to try potentially
multiple times
- boolean successfullyReleased = false;
- do {
- final Iterator<MemorySegment> segmentsIterator
= segments.iterator();
+ EnumMap<MemoryType, Long> releasedMemory = new
EnumMap<>(MemoryType.class);
- Object lastOwner = null;
- Set<MemorySegment> segsForOwner = null;
+ // since concurrent modifications to the collection
+ // can disturb the release, we need to try potentially multiple
times
+ boolean successfullyReleased = false;
+ do {
+ Iterator<MemorySegment> segmentsIterator =
segments.iterator();
- try {
- // go over all segments
- while (segmentsIterator.hasNext()) {
-
- final MemorySegment seg =
segmentsIterator.next();
- if (seg == null ||
seg.isFreed()) {
- continue;
- }
-
- final Object owner =
seg.getOwner();
-
- try {
- // get the list of
segments by this owner only if it is a different owner than for
- // the previous one (or
it is the first segment)
- if (lastOwner != owner)
{
- lastOwner =
owner;
- segsForOwner =
this.allocatedSegments.get(owner);
- }
-
- // remove the segment
from the list
- if (segsForOwner !=
null) {
-
segsForOwner.remove(seg);
- if
(segsForOwner.isEmpty()) {
-
this.allocatedSegments.remove(owner);
- }
- }
-
- if (isPreAllocated) {
-
memoryPool.returnSegmentToPool(seg);
- }
- else {
- seg.free();
-
numNonAllocatedPages++;
- }
- }
- catch (Throwable t) {
- throw new
RuntimeException(
- "Error
removing book-keeping reference to allocated memory segment.", t);
- }
+ //noinspection ProhibitedExceptionCaught
+ try {
+ MemorySegment segment = null;
+ while (segment == null &&
segmentsIterator.hasNext()) {
+ segment = segmentsIterator.next();
+ if (segment.isFreed()) {
+ segment = null;
}
+ }
+ while (segment != null) {
+ segment =
releaseSegmentsForOwnerUntilNextOwner(segment, segmentsIterator,
releasedMemory);
Review comment:
True, it would simplify the things. I thought about it. The original code
was doing only one traversal over the segments to release, I think to optimise
the code and spare memory. I decided to keep this approach to avoid any
possible degradation. Although, we can later try whether there is actual
performance degradation and change it to the simpler approach for better
maintainability.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
