Depending on just how many are to be deleted, a common pattern is to collect all 'to-be-deleted' during the main iteration, and then delete in a second iteration over the collection. Especially if you only keep the relationship ids, the memory load might be manageable. You said 'several hundred thousand', so if you deleted half of one million, that would be 500000 ids, or 500000*4/(1024*1024) =~ 2MB. This is not a large amount of memory to use for a temporary collection. Not sure how big it would be if you collected the relationship objects themselves, perhaps still manageable in memory.
On Fri, Jul 30, 2010 at 7:34 PM, Rick Bullotta < rick.bullo...@burningskysoftware.com> wrote: > Very interesting question, Alex. Since you've potentially "mutated" the > collection you're Iterating, the "correct" thing to do is to invalidate the > iterator, but I do see the need to meet your functional requirements of > incremental updates. > > While I'm not sure it would be ideal, one approach you could possibly take > is to do the deletes on another "worker" thread. You could a variety of > techniques to get the "work" to those worker thread(s) (queues, etc.), and > in theory, the iterator would continue on with its work on the "old" view > of > the graph. While this could be a big consistency problem for many > applications, if you knew that in your application it would not be, it > might > work. Not sure what happens to iterators in the case of other-thread > commits. > > > -----Original Message----- > From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] > On > Behalf Of Alex D'Amour > Sent: Friday, July 30, 2010 1:21 PM > To: Neo user discussions > Subject: [Neo4j] Stability of Iterators Across Transactions? > > Hello all, > > I have an application where I have a node that has several hundred thousand > relationships (this probably needs to be changed). In the application I > iterate over these relationships, and delete a large subset of them. > Because > there are so many writes, I want to commit the transaction every few > thousand deletions. The problem is that the getAllRelationships iterator > seems to halt after the first transaction commit. > > Clearly, I should reduce the number of relationships that are connected to > this node, but is this the expected behavior? Should iterators be made > stable across transactions, or are they only supposed to be guaranteed > within a transaction? > > Thanks, > Alex > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
