Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19836#discussion_r155230290
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala
---
@@ -159,11 +160,18 @@ class BlockManagerMasterEndpoint(
// Ask the slaves to remove the RDD, and put the result in a sequence
of Futures.
// The dispatcher is used as an implicit argument into the Future
sequence construction.
val removeMsg = RemoveRdd(rddId)
- Future.sequence(
- blockManagerInfo.values.map { bm =>
- bm.slaveEndpoint.ask[Int](removeMsg)
- }.toSeq
- )
+
+ val handleRemoveRddException: PartialFunction[Throwable, Int] = {
+ case e: IOException =>
+ logWarning(s"Error trying to remove RDD $rddId", e)
+ 0 // zero blocks were removed
+ }
+
+ val futures = blockManagerInfo.values.map { bm =>
+
bm.slaveEndpoint.ask[Int](removeMsg).recover(handleRemoveRddException)
--- End diff --
personally I think
```
bm.slaveEndpoint.ask[Int](removeMsg).recover {
case e: IOException =>
logWarning(s"Error trying to remove RDD $rddId", e)
0 // zero blocks were removed
}
```
is more readable
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
