I answered this on StackOverflow:
I did an experiment with [AnormCypher][1]
I think AnormCypher could be a bit more performant though, not sure what it
is, would have to profile it.
Like this:
implicit val connection = Neo4jREST.setServer("localhost", 7474,
"/db/data/")
val conf = new SparkConf().setAppName("Simple Application")
val sc = new SparkContext(conf)
val logData = sc.textFile(FILE, 4).cache()
val count = logData
.flatMap( _.split(" "))
.map( w =>
Cypher("CREATE(:Word {text:{text}})")
.on( "text" -> w ).execute()
).filter( _ ).count()
Neo4j 2.2.x has great concurrent write performance that you can use from
Spark. So the more concurrent threads you can have to write to Neo4j the
better. If you can batch statements in batches of 100 to 1000 each per
request then even better.
[1]: http://AnormCypher.org
On Thu, Jun 25, 2015 at 10:19 PM, Naren <[email protected]> wrote:
> I need to establish a connection from Spark Streaming to Neo4j graph
> database.The RDDs are of type((is,I),(am,Hello)(sam,happy)....). I need to
> establish a edge between each pair of words in Neo4j.
>
>
> In Spark Streaming documentation I found
>
>
> dstream.foreachRDD { rdd =>
> rdd.foreachPartition { partitionOfRecords =>
> // ConnectionPool is a static, lazily initialized pool of connections
> val connection = ConnectionPool.getConnection()
> partitionOfRecords.foreach(record => connection.send(record))
> ConnectionPool.returnConnection(connection) // return to the pool for
> future reuse
> }}
> to the push to the data to an external database.
>
> I am doing this in Scala. I am little confused about how to go about? I
> found AnormCypher and Neo4jScala wrapper. Can I use these to get work done?
> If so, how can I do that? If not, all there any better alternatives?
>
> Thank you all....
>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
