Github user bkrieger commented on a diff in the pull request:
https://github.com/apache/spark/pull/22995#discussion_r236391043
--- Diff:
core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala ---
@@ -93,7 +96,14 @@ private[spark] class TorrentBroadcast[T: ClassTag](obj:
T, id: Long)
private var checksums: Array[Int] = _
override protected def getValue() = {
- _value
+ val memoized: T = if (_value == null) null.asInstanceOf[T] else
_value.get
--- End diff --
Good catch. I'll make it synchronized, so it only loads one at a time.
Re: WeakReference, sure, I can change it to SoftReference. That'll be
closer to the original behavior, and should still give the improvement we want.
When I try with `.asInstanceOf[T]` it fails to compile with:
```
[error]
/Users/bkrieger/Documents/git/spark/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala:98:
type mismatch;
[error] found : Null(null)
[error] required: T
[error] val memoized: T = if (_value == null) null else _value.get
[error] ^
[info] Null(null) <: T?
[info] false
[error] one error found
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
