After further debugging into this issue, I find that this bug is related to
the triplets view of a graph in GraphX.
If a graph is generated by outer join two other graphs via outerJoinVertices
operation, the graph's triplets view and the vertices view may be
inconsistent. In the following example, dataGraph's triplets view will be
inconsistent with its vertices view.
> // purGraph is a toy graph with edges: 2->1, 3->1, 2->3.
> val purGraph = Graph(dataVertex, dataEdge).persist()
> purGraph.triplets.count()
> val inNeighborGraph = purGraph.collectNeighbors(EdgeDirection.In)
> val dataGraph = purGraph.outerJoinVertices(inNeighborGraph)((vid,
> property, inNeighborList) => {...
> })
>
> //
*
> dataGraph's vertices view and triplets view will be inconsistent.
*
> dataGraph.vertices.foreach {...}
> dataGraph.triplets.foreach {...}
I find a similar issue on JIRA SPARK-6378
<https://issues.apache.org/jira/browse/SPARK-6378> . I have gave more
details and a code demo in the issue's comment. If you are interested in
this issue, you can follow it there. Thanks a lot.
Regards,
Zhaokang
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Strange-behavior-of-collectNeighbors-API-in-GraphX-tp26459p26468.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
