After doing insertion with the following query:
MERGE (headNode:HEAD {list:"mylist"})
REMOVE headNode._lock_
WITH headNode
MATCH (headNode)-[old:LINK]->after
DELETE old
CREATE headNode-[:LINK]->(newNode:LINKNODE { number : {nodeNumber}
})-[:LINK]->after
RETURN headNode, newNode
I still go the same relationship errors and my linked list was messed up (
see https://dl.dropboxusercontent.com/u/83907628/messedUpLinkedList.png for
a screen shot of it )
I thought maybe I should place a lock on the next node to stop that
happening so I changed my query to be
MERGE (headNode:HEAD {list:"mylist"})
REMOVE headNode._lock_
WITH headNode
MATCH (headNode)-[old:LINK]->after
REMOVE after._lock_
DELETE old
CREATE headNode-[:LINK]->(newNode:LINKNODE { number : {nodeNumber}
})-[:LINK]->after
RETURN headNode, newNode
I get the same error and the results are no different (I still get the
messed up data structure)
The full error is:
Neo.DatabaseError.Statement.ExecutionFailure
Relationship 391279 not found
So I changed the query to be:
MATCH (headNode:HEAD {list:"mylist"})-[old:LINK]->after
REMOVE headNode._lock_
REMOVE after._lock_
DELETE old
CREATE headNode-[:LINK]->(newNode:LINKNODE { number : {nodeNumber}
})-[:LINK]->after
RETURN headNode, newNode
And now I only get the Relationship Error if the insert is not possible.
That means I can re-run the query if I get this specific error and all is
well. My data structure does not get messed up.
You will notice I removed the MERGE as that was not helping and placed the
two REMOVE statements in the same context as the insert, they are not
divided from the CREATE via a WITH.
It seems that placing a WITH clause in the statement removes the lock. Does
this sound correct? The WITH clause does divide the query into section as
it is a pseudo RETURN, so the idea that the lock is removed at the WITH
clause would make some logical sense.
On Tue, Feb 11, 2014 at 11:54 PM, Michael Hunger <
[email protected]> wrote:
> Aran,
>
> it seems that MERGE doesn't take a write lock by default, only when it
> didn't find anything in the first place and has to create it.
>
> Can you try the following instead.
>
> "Removal of a non-existing property causes a write lock on the node". So
> if we do that on the head-node it should actually work.
>
>
> MERGE (headNode:HEAD {list:"mylist"})
>>>
>> REMOVE headNode._lock_
>
> WITH headNode
>>> MATCH (headNode)-[old:LINK]->after
>>> DELETE old
>>> CREATE headNode-[:LINK]->(newNode:LINKNODE { number : {nodeNumber}
>>> })-[:LINK]->after
>>>
>>
> Am 11.02.2014 um 11:15 schrieb Aran Mulholland <[email protected]>:
>
> Anyone got any more ideas on how to do this?
>
>
> On Tue, Feb 11, 2014 at 3:01 PM, Aran Mulholland <[email protected]
> > wrote:
>
>> I have placed this question on stack overflow here:
>>
>>
>> http://stackoverflow.com/questions/21692829/neo4j-how-do-you-do-parallel-insertion-into-a-linked-list-via-cypher-over-rest
>>
>>
>>
>> On Mon, Feb 10, 2014 at 11:40 PM, Aran Mulholland <
>> [email protected]> wrote:
>>
>>> thanks for your help, sorry for not being clearer.
>>>
>>> I create the constraint then if I run
>>>
>>> MERGE (headNode:HEAD {list:"mylist"})-[:LINK]->headNode
>>>
>>> I get Error: org.neo4j.cypher.PatternException: MERGE needs at least
>>> some part of the pattern to already be known. Please provide values for one
>>> of: headNode, headNode
>>>
>>> So I change it to:
>>>
>>> MERGE (headNode:HEAD {list:"mylist"})
>>> WITH headNode
>>> MERGE headNode-[:LINK]->(headNode)
>>>
>>> Then run the query:
>>>
>>> MERGE (headNode:HEAD {list:"mylist"})
>>> WITH headNode
>>> MATCH (headNode)-[old:LINK]->after
>>> DELETE old
>>> CREATE headNode-[:LINK]->(newNode:LINKNODE { number : {nodeNumber}
>>> })-[:LINK]->after
>>>
>>> Then I get Error: Relationship 391112 not found
>>>
>>> I have been doing this test work in a node.js project. (so I can test
>>> adding multiple nodes in parallel)
>>> If it helps, the source code is here
>>> https://github.com/aranm/neo4j-linked-list
>>> The relevant file is here
>>> https://github.com/aranm/neo4j-linked-list/blob/master/tests/parallelInsertionTest.js
>>>
>>> When I change line 40 from
>>>
>>> async.map(nodesToInsert, function(item, callback) --inserting in parallel
>>>
>>> to
>>>
>>> async.mapSeries(nodesToInsert, function(item, callback) --inserting in
>>> series
>>>
>>> then all is well, it works when we are not hitting the db at the same
>>> time. But as soon as we introduce the parallel stuff all goes south.
>>>
>>>
>>> On Mon, Feb 10, 2014 at 11:13 PM, Michael Hunger <
>>> [email protected]> wrote:
>>>
>>>> You would also use MERGE for the initial creation
>>>>
>>>> What doesn't work? Did you create the constraint?
>>>>
>>>> Am 10.02.2014 um 12:49 schrieb Aran Mulholland <
>>>> [email protected]>:
>>>>
>>>> Just to make it absolutely clear, are you saying when creating the
>>>> initial node I do:
>>>>
>>>> MERGE (headNode:HEAD {list:"mylist"})
>>>>
>>>> CREATE (headNode)-[:LINK]->(headNode)
>>>>
>>>>
>>>> But when inserting I do:
>>>>
>>>> MERGE (headNode:HEAD {list:"mylist"})
>>>> WITH headNode //had to add this
>>>> line, cause I got an error WITHout it :)
>>>> MATCH (headNode)-[old:LINK]->after
>>>> DELETE old
>>>> CREATE headNode-[:LINK]->(newNode:LINKNODE { number : {nodeNumber}
>>>> })-[:LINK]->after
>>>>
>>>> Sorry for the re-clarification (I'm feeling a little dense here). The
>>>> reason I am asking is that it doesn't work, so I must have mis-understood
>>>> you somewhere.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Feb 10, 2014 at 10:30 PM, Michael Hunger <
>>>> [email protected]> wrote:
>>>>
>>>>> Only with unique constraints, merge takes a global lock on the label +
>>>>> property.
>>>>>
>>>>> Michael
>>>>>
>>>>> Am 10.02.2014 um 12:17 schrieb Aran Mulholland <
>>>>> [email protected]>:
>>>>>
>>>>> Why do you need the unique constraint?
>>>>>
>>>>>
>>>>> On Mon, Feb 10, 2014 at 9:52 PM, Michael Hunger <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> I think you have to change it to use merge which takes a schema index
>>>>>> lock.
>>>>>>
>>>>>> ie.
>>>>>>
>>>>>> create constraint on (h:HEAD) assert h.list is unique;
>>>>>>
>>>>>> MERGE (headNode:HEAD {list:"mylist"})
>>>>>>
>>>>>> MATCH (headNode)-[old:LINK]->after
>>>>>>
>>>>>> DELETE old
>>>>>> CREATE headNode-[:LINK]->(newNode:LINKNODE { number : {nodeNumber}
>>>>>> })-[:LINK]->after
>>>>>>
>>>>>>
>>>>>>
>>>>>> HTH
>>>>>>
>>>>>> Michael
>>>>>>
>>>>>> Am 10.02.2014 um 11:30 schrieb Aran Mulholland <
>>>>>> [email protected]>:
>>>>>>
>>>>>> I've got some real headaches inserting into linked lists.
>>>>>>
>>>>>> If I create a node as such:
>>>>>>
>>>>>> CREATE (headNode:HEAD)-[:LINK]->(headNode)
>>>>>>
>>>>>> then insert like so
>>>>>>
>>>>>> MATCH (headNode:HEAD)-[old:LINK]->after
>>>>>> DELETE old
>>>>>> CREATE headNode-[:LINK]->(newNode:LINKNODE { number : {nodeNumber}
>>>>>> })-[:LINK]->after
>>>>>>
>>>>>> then everything is fine and dandy. But when I try to run the
>>>>>> insertion code in parallel, then I get some real issues. I either get an
>>>>>> error like "Relationship 325457 not found" or I get two or more lists
>>>>>> coming off the headNode. The first error I can deal with as I can rerun
>>>>>> the
>>>>>> query, the second is totally unacceptable as my other queries get very
>>>>>> messed up results.
>>>>>>
>>>>>> I am writing for a twitter like service and am using the linked lists
>>>>>> so I can have a news feed. There is a high possibility that the list will
>>>>>> need to be added to by more than one user at once, and this chance will
>>>>>> grow the more users the system has.
>>>>>>
>>>>>> What I want to be able to do is put a lock around the insertion so
>>>>>> that this does not happen.
>>>>>>
>>>>>> How can I better craft my queries so that parallel insertion via
>>>>>> cypher over REST is sustainable?
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Neo4j" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Neo4j" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Neo4j" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Neo4j" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Neo4j" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Neo4j" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>
>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
