cloud-fan commented on code in PR #53578:
URL: https://github.com/apache/spark/pull/53578#discussion_r2644758390
##########
core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java:
##########
@@ -390,16 +397,21 @@ public MemoryBlock allocatePage(long size, MemoryConsumer
consumer) {
try {
page = memoryManager.tungstenMemoryAllocator().allocate(acquired);
} catch (OutOfMemoryError e) {
- logger.warn("Failed to allocate a page ({} bytes), try again.",
- MDC.of(LogKeys.PAGE_SIZE, acquired));
+ if (!isRetry) {
+ logger.warn("Failed to allocate a page ({} bytes), try again.", e,
+ MDC.of(LogKeys.PAGE_SIZE, acquired));
+ } else {
+ logger.warn("Failed to allocate a page ({} bytes), try again.",
+ MDC.of(LogKeys.PAGE_SIZE, acquired));
+ }
// there is no enough memory actually, it means the actual free memory
is smaller than
// MemoryManager thought, we should keep the acquired memory.
synchronized (this) {
acquiredButNotUsed += acquired;
allocatedPages.clear(pageNumber);
}
// this could trigger spilling to free some pages.
- return allocatePage(size, consumer);
+ return allocatePage(size, consumer, true);
Review Comment:
not related to this PR, but the retry is infinite?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
