[
https://issues.apache.org/jira/browse/SPARK-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-3018:
-------------------------------
Description: BlockFetcherIterator retains ManagedBuffers returned by
BlockClient.fetchBlocks. Those buffers are released when the iterators are
traversed fully. In the case of task failures or completion without exhausting
the iterator, this could lead to memory leak. (was: BlockFetcherIterator
retains ReferenceCountedBuffers returned by client.fetchBlocks. Those buffers
are released when the iterators are traversed fully. In the case of task
failures or completion without exhausting the iterator, this could lead to
memory leak.)
> Release all ManagedBuffers upon task completion/failure
> -------------------------------------------------------
>
> Key: SPARK-3018
> URL: https://issues.apache.org/jira/browse/SPARK-3018
> Project: Spark
> Issue Type: Sub-task
> Components: Shuffle, Spark Core
> Reporter: Reynold Xin
> Assignee: Reynold Xin
>
> BlockFetcherIterator retains ManagedBuffers returned by
> BlockClient.fetchBlocks. Those buffers are released when the iterators are
> traversed fully. In the case of task failures or completion without
> exhausting the iterator, this could lead to memory leak.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
