Github user szhem commented on a diff in the pull request:
https://github.com/apache/spark/pull/23083#discussion_r235319165
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -72,7 +73,8 @@ final class ShuffleBlockFetcherIterator(
maxBlocksInFlightPerAddress: Int,
maxReqSizeShuffleToMem: Long,
detectCorrupt: Boolean)
- extends Iterator[(BlockId, InputStream)] with DownloadFileManager with
Logging {
+ extends Iterator[(BlockId, InputStream)] with DownloadFileManager with
TaskCompletionListener
--- End diff --
The main reason is that TaskCompletionListener is added in one place (in
`initialize` method) and needs to be removed in another one (in `cleanup`
method).
Will introduce a field for TaskCompletionListener instead. Thank you!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
