[
https://issues.apache.org/jira/browse/SPARK-18568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822397#comment-15822397
]
Andrew Ray commented on SPARK-18568:
------------------------------------
RDD's have the same problem for cached collections of mutable objects
{code}
import scala.collection.mutable.{Map => MMap}
val rdd = sc.parallelize(MMap(1->1)::Nil)
rdd.cache()
val rdd2 = rdd.map(_ += 2 ->2)
scala> rdd2.collect()
res21: Array[scala.collection.mutable.Map[Int,Int]] = Array(Map(2 -> 2, 1 -> 1))
scala> rdd.collect()
res22: Array[scala.collection.mutable.Map[Int,Int]] = Array(Map(2 -> 2, 1 -> 1))
{code}
So I think the moral is it use immutable objects.
> vertex attributes in the edge triplet not getting updated in super steps for
> Pregel API
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-18568
> URL: https://issues.apache.org/jira/browse/SPARK-18568
> Project: Spark
> Issue Type: Bug
> Components: GraphX
> Affects Versions: 2.0.2
> Reporter: Rohit
>
> When running the Pregel API with vertex attribute as complex objects. The
> vertex attributes are not getting updated in the triplet view. For example if
> the vertex attributes changes in first superstep for vertex"a" the triplet
> src attributes in the send msg program for the first super step gets the
> latest attributes of the vertex "a" but on 2nd super step if the vertex
> attributes changes in the vprog the edge triplets are not updated with this
> new state of the vertex for all the edge triplets having the vertex "a" as
> src or destination. if I re-create the graph using g = Graph(g.vertices,
> g.edges) in the while loop before the next super step then its getting
> updated. But this fix is not good performance wise. A detailed description of
> the bug along with the code to recreate it is in the attached URL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
