Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21322
Introducing a new concept which is similar to broadcast seems like an
overkill. We can just update broadcast, to allow it to be memory-only.
However, there might be simpler solutions to fit your case, without
broadcast. e.g.
```
val myObj = ...
rdd.mapPartitions { it =>
try {
// process data
} finally {
myObj.close()
}
}
```
I think we need to clear define the use case and think about whether we
need a new API or not.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
