[
https://issues.apache.org/jira/browse/SPARK-8454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-8454.
---------------------------------
Resolution: Incomplete
> Update ShuffleBlockFetcherIterator.next to close previous stream when
> returning a new one
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-8454
> URL: https://issues.apache.org/jira/browse/SPARK-8454
> Project: Spark
> Issue Type: Improvement
> Components: Block Manager
> Reporter: Matt Massie
> Priority: Major
> Labels: bulk-closed
>
> For context, see the PR comments at
> https://github.com/apache/spark/pull/6423#discussion_r31370774
> The current HashShuffleReader closes the input stream when all records have
> been read by using a NextIterator. However, we want the
> ShuffleBlockFetcherIterator to be more defensive about closing files and
> releasing memory to protect against other ShuffleReaders which may not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
