FWIW, you might be better off "pipelining" these writes through a single worker thread/queue. It helps with a few performance issues: 1) you can avoid synchronization concerns 2) you can manage "block writes" (e.g. a number of writes in a single transaction) more easily and 3) you can (if you choose) implement throttling/queueing to support burst mode scenarios where the # requests to write data exceed your ability to process them (and to provide responsiveness for queries).
This is what we do with our activity stream engine in ThingWorx. Note that the same applies for deleting entries - we try to do "bulk deletes" as well. ________________________________________ From: [email protected] [[email protected]] On Behalf Of Mattias Persson [[email protected]] Sent: Saturday, November 05, 2011 9:14 AM To: Neo4j user discussions Subject: Re: [Neo4j] Node Id generation deadlock You'd have to go with another solution then. Is your application this critical to write throughput or are you just thinking ahead and making sure that it some day might need to support an amount of write throughput? 2011/11/3 Yaniv Ben Yosef <[email protected]> > Hi, > > I've also been wondering about this subject. > According to the Neo4J design guide ( > http://wiki.neo4j.org/content/Design_Guide) the factory node and id > generator patterns are the way to implement sequential ID generation. > However, according to this thread, it sounds like in a multi-threaded > environment I have one of two choices: > 1. lock the factory node. However this will force transaction > serialization. That's not practical in my app. > 2. reducing the granularity of the transactions - in the sense that each > transaction should only contain one node creation (of the same type, i.e. > uses the same factory node). That's not practical for me either, because in > several cases I would like to create more than one node in the same > transaction. > Since the design guide recommends the factory node pattern, I'm wondering > if there's anything I'm missing here, or that I should just avoid this > pattern and use some other ID generation mechanism. > > Thanks, > Yaniv > > > On Thu, Nov 3, 2011 at 11:13 AM, Mattias Persson > <[email protected]>wrote: > > > 2011/11/3 Cres <[email protected]> > > > > > This solution would have been ok if I had only one node created from > that > > > factory in each transaction. > > > > > > It doesn't matter... after factory.setProperty is run that transaction > > has > > got a write lock on that factory node which is held until the transaction > > committs. The benefit you get from my proposal would be that you make > sure > > you read the correct value. > > > > > > > however, as shown in the sample code I posted in the original message, > I > > > have multiple nodes created in one transaction, and multiple such > > > transactions in multiple threads. > > > So while the creation of the actual nodes will of course be serialized, > > one > > > thread's transaction will have to wait for the other thread's > transaction > > > to > > > finish completely, and so if the first thread has some processing to do > > > between the creation of the first and second nodes, the other third > won't > > > be > > > able to create its two nodes in the meanwhile, because the first thread > > > will > > > have the lock on the factory node until the entire transaction > completes. > > > > > > I'm looking for a way to do this having the nodes creation serialized > but > > > without having the entire transactions serialized, possibly by somehow > > > releasing the lock on the factory node in mid-transaction, or by any > > other > > > method. > > > > > > Thanks again, > > > Ran. > > > > > > -- > > > View this message in context: > > > > > > http://neo4j-community-discussions.438527.n3.nabble.com/Node-Id-generation-deadlock-tp3473118p3476498.html > > > Sent from the Neo4j Community Discussions mailing list archive at > > > Nabble.com. > > > _______________________________________________ > > > Neo4j mailing list > > > [email protected] > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > -- > > Mattias Persson, [[email protected]] > > Hacker, Neo Technology > > www.neotechnology.com > > _______________________________________________ > > Neo4j mailing list > > [email protected] > > https://lists.neo4j.org/mailman/listinfo/user > > > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [[email protected]] Hacker, Neo Technology www.neotechnology.com _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
