Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/21114#discussion_r183770468
--- Diff: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala ---
@@ -258,14 +258,8 @@ private[spark] object AccumulatorContext {
* Returns the [[AccumulatorV2]] registered with the given ID, if any.
*/
def get(id: Long): Option[AccumulatorV2[_, _]] = {
- Option(originals.get(id)).map { ref =>
- // Since we are storing weak references, we must check whether the
underlying data is valid.
- val acc = ref.get
- if (acc eq null) {
- throw new IllegalStateException(s"Attempted to access garbage
collected accumulator $id")
- }
- acc
- }
+ val ref = originals.get(id)
+ Option(if (ref != null) ref.get else null)
--- End diff --
As the discussion in
[JIRA](https://issues.apache.org/jira/browse/SPARK-22371), here should not
raise an Exception, but we may also need some warning logs?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
