Github user szhem commented on a diff in the pull request:
https://github.com/apache/spark/pull/23083#discussion_r235759169
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -72,7 +73,8 @@ final class ShuffleBlockFetcherIterator(
maxBlocksInFlightPerAddress: Int,
maxReqSizeShuffleToMem: Long,
detectCorrupt: Boolean)
- extends Iterator[(BlockId, InputStream)] with DownloadFileManager with
Logging {
+ extends Iterator[(BlockId, InputStream)] with DownloadFileManager with
TaskCompletionListener
--- End diff --
Introduced [the corresponding
field](https://github.com/apache/spark/pull/23083/files#diff-27109eb30a77542d377c936e0d134420R156).
```scala
/**
* Task completion callback to be called in both success as well as
failure cases to cleanup.
* It may not be called at all in case the `cleanup` method has already
been called before
* task completion.
*/
private[this] val cleanupTaskCompletionListener = (_: TaskContext) =>
cleanup()
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
