After parsing the unstructured contents from a file, I now have the following HashMap of key, value pairs where the key represents vertices and the value represents the edge. Note how using the edges, the vertices can be connected by joining the values. I am now trying to build Spark vertex and edge RDD from this hashmap. Can you please give me some pointers?
tempLHM.foreach(x=>println(x)) (A.L0,A) (B.L0,B) (C.L0,C) (D.L0,D) (E.L0,E) (a.L0M1,A) (b.L0M1,B) (c.L0M1,n4) (a.L0M2,n4) (b.L0M2,D) (c.L0M2,n5) (a.L0M3,n5) (b.L0M3,C) (c.L0M3,E) Is there a direct way to map this hashmap to vertex and edge RDD? tempLHM is a mutable LinkedHashMap[String,String] In the above hashmap, in elements (A.L0,A) and (a.L0M1,A), A.L0 and a.L0M1 are keys(vertices) that are joined by the common value A (edge) Here is what I want to derive val vertex:RDD(vertexId, VertexName) i.e ((A.L0).Long, A.L0), ((a.L0M1).Long, a.L0M1) etc val edge:RDD((vertexId1, vertexId2), EdgeName) i.e ((A.L0).Long, (a.L0M1).Long), A) But I am unable to achieve this with my limited Spark programmatic skills. Any help is greatly appreciated. Thanks. Aarthi -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-map-a-HashMap-containing-vertex-as-key-and-edge-as-values-into-Spark-RDD-tp25743.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
