Re: Non-Deterministic Graph Building

2015-04-24 Thread hokiegeek2
Hi Everyone, Here's the Scala code for generating the EdgeRDD, VertexRDD, and Graph: //Generate a mapping of vertex (edge) names to VertexIds val vertexNameToIdRDD = rawEdgeRDD.flatMap(x = Seq(x._1.src,x._1.dst)).distinct.zipWithUniqueId.cache //Generate VertexRDD with vertex data (in my case,

Non-Deterministic Graph Building

2015-04-23 Thread hokiegeek2
Hi Everyone, I am running into a really weird problem that only one other person has reported to the best of my knowledge (and the thread never yielded a resolution). I build a GraphX Graph from an input EdgeRDD and VertexRDD via the Graph(VertexRDD,EdgeRDD) constructor. When I execute

RDD.filter vs. RDD.join--advice please

2015-04-22 Thread hokiegeek2
Hi Everyone, I have two options of filtering the RDD resulting from the Graph.vertices method as illustrated with the following pseudo code: 1. Filter val vertexSet = Set(vertexOne,vertexTwo...); val filteredVertices = Graph.vertices.filter(x = vertexSet.contains(x._2.vertexName)) 2. Join

Quick GraphX gutcheck

2015-04-01 Thread hokiegeek2
Hi Everyone, Quick (hopefully) and silly (likely) question--the VertexId can be used to join the VertexRDD generated from Graph.vertices with a transformed RDD where the keys are vertexIds from the original graph, correct? --John -- View this message in context: