Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
I think for such a scenario you'd need read locks which gets upgraded to write locks when modifying. Consider this simple scenario w/o read locks: Thread ONE: get relationship R Thread TWO: get relationship R Thread TWO: delete relationship R and commit transaction Thread ONE: do any modification on relationship R... BOOM Exception If read locks were to be taken when getting relationships: Thread ONE: get relationship R Thread TWO: get relationship R Thread TWO: would like to delete relationship R, but will have to wait until Thread ONE releases its read lock on it Thread ONE: do some modification on relationship R and commit transaction Thread TWO: lock from Thread ONE was released so delete relationship R and commit transaction Unfortunately you cannot provide different isolation levels in neo4j at the moment, but you could mimic that behavior yourself using LockManager/LockReleaser (from graphDb.getConfig()) 2011/11/21 Aseem Kishore aseem.kish...@gmail.com Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
I'm definitely modifying the graph at the same time; that's why this is happening. But my assumption was that that shouldn't be breaking Neo4j -- I thought requests are transactional/serialized. I don't have a repro test at the moment, but the behavior is really simple. Here's an example analogy. We have a subgraph like User A [LIKES] Object 1 and also [LIKES] Object 2: (Object 1) --LIKES-- (User A) --LIKES-- (Object 2) One operation will delete the [LIKE] to Object 1, while another operation queries whether User A [LIKES] Object 2. The query can be a Cypher query, e.g.: START user=(123), obj=(456) MATCH (user)-[rel]-(obj) WHERE rel~TYPE = 'LIKES' RETURN rel But we've also seen this bug where we used a simple traverse (specifying just max depth 1 and the LIKES relationship type). When you say looks like a normal case, do you mean that the query should indeed fail like this? I wouldn't expect this -- what can I do to prevent an error? Should I be retrying? Thanks Peter! Aseem On Sun, Nov 20, 2011 at 10:08 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, What query are you running, and are you modifying your graph on any way at the same time? Do you have a small example test reproducing this? Looks like a normal case where this relationship is deleted for some reason, no failure. Maybe a better Cypher error should point that out... /peter Sent from my phone, please excuse typos and autocorrection. On Nov 21, 2011 3:22 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
On Sun, Nov 20, 2011 at 10:43 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Cypher in 1.4 was very experimental. If you are using Cypher, I would suggest you move to 1.5 ASAP - a lot has happened there. If you still have the same problems with 1.5, then we can start looking for the problem. Hmm, unfortunately we're in the middle of a milestone, so we haven't prioritized looking into upgrading yet, but to clarify, this bug happens with queries that use the traverse REST API as well, not just Cypher. But if this really is too difficult to pin down, I can ask again when we've upgraded to 1.5. Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
Mattias, thanks for the suggestion, but to clarify, we're using the server as-is, with the REST API; we're not using the Java API. Is there any way we can specify locking like you suggest via the server config or similar? Aseem On Mon, Nov 21, 2011 at 12:26 AM, Mattias Persson matt...@neotechnology.com wrote: I think for such a scenario you'd need read locks which gets upgraded to write locks when modifying. Consider this simple scenario w/o read locks: Thread ONE: get relationship R Thread TWO: get relationship R Thread TWO: delete relationship R and commit transaction Thread ONE: do any modification on relationship R... BOOM Exception If read locks were to be taken when getting relationships: Thread ONE: get relationship R Thread TWO: get relationship R Thread TWO: would like to delete relationship R, but will have to wait until Thread ONE releases its read lock on it Thread ONE: do some modification on relationship R and commit transaction Thread TWO: lock from Thread ONE was released so delete relationship R and commit transaction Unfortunately you cannot provide different isolation levels in neo4j at the moment, but you could mimic that behavior yourself using LockManager/LockReleaser (from graphDb.getConfig()) 2011/11/21 Aseem Kishore aseem.kish...@gmail.com Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
One more point of clarification: these are concurrent REST API requests. One request is deleting a relationship, another request is querying for a(nother) relationship. A simple analogy is a user unliking one thing on Facebook then immediately liking something else. I wouldn't expect the database to throw an error in this case, right? Am I wrong in assuming that the requests should be serialized / handled gracefully? If I'm wrong, should I just be optimistically retrying failed requests? Thanks again! Aseem On Mon, Nov 21, 2011 at 12:47 AM, Aseem Kishore aseem.kish...@gmail.comwrote: I'm definitely modifying the graph at the same time; that's why this is happening. But my assumption was that that shouldn't be breaking Neo4j -- I thought requests are transactional/serialized. I don't have a repro test at the moment, but the behavior is really simple. Here's an example analogy. We have a subgraph like User A [LIKES] Object 1 and also [LIKES] Object 2: (Object 1) --LIKES-- (User A) --LIKES-- (Object 2) One operation will delete the [LIKE] to Object 1, while another operation queries whether User A [LIKES] Object 2. The query can be a Cypher query, e.g.: START user=(123), obj=(456) MATCH (user)-[rel]-(obj) WHERE rel~TYPE = 'LIKES' RETURN rel But we've also seen this bug where we used a simple traverse (specifying just max depth 1 and the LIKES relationship type). When you say looks like a normal case, do you mean that the query should indeed fail like this? I wouldn't expect this -- what can I do to prevent an error? Should I be retrying? Thanks Peter! Aseem On Sun, Nov 20, 2011 at 10:08 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, What query are you running, and are you modifying your graph on any way at the same time? Do you have a small example test reproducing this? Looks like a normal case where this relationship is deleted for some reason, no failure. Maybe a better Cypher error should point that out... /peter Sent from my phone, please excuse typos and autocorrection. On Nov 21, 2011 3:22 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
2011/11/21 Aseem Kishore aseem.kish...@gmail.com One more point of clarification: these are concurrent REST API requests. One request is deleting a relationship, another request is querying for a(nother) relationship. A simple analogy is a user unliking one thing on Facebook then immediately liking something else. I wouldn't expect the database to throw an error in this case, right? Am I wrong in assuming that the requests should be serialized / handled gracefully? If I'm wrong, should I just be optimistically retrying failed requests? That isn't possible through pure REST a.t.m. A retry would do the trick but feels like jut a workaround, really. It would be nice with a way to control isolation using an http header or similar. And no, you shouldn't need to worry about these things. Thanks again! Aseem On Mon, Nov 21, 2011 at 12:47 AM, Aseem Kishore aseem.kish...@gmail.com wrote: I'm definitely modifying the graph at the same time; that's why this is happening. But my assumption was that that shouldn't be breaking Neo4j -- I thought requests are transactional/serialized. I don't have a repro test at the moment, but the behavior is really simple. Here's an example analogy. We have a subgraph like User A [LIKES] Object 1 and also [LIKES] Object 2: (Object 1) --LIKES-- (User A) --LIKES-- (Object 2) One operation will delete the [LIKE] to Object 1, while another operation queries whether User A [LIKES] Object 2. The query can be a Cypher query, e.g.: START user=(123), obj=(456) MATCH (user)-[rel]-(obj) WHERE rel~TYPE = 'LIKES' RETURN rel But we've also seen this bug where we used a simple traverse (specifying just max depth 1 and the LIKES relationship type). When you say looks like a normal case, do you mean that the query should indeed fail like this? I wouldn't expect this -- what can I do to prevent an error? Should I be retrying? Thanks Peter! Aseem On Sun, Nov 20, 2011 at 10:08 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, What query are you running, and are you modifying your graph on any way at the same time? Do you have a small example test reproducing this? Looks like a normal case where this relationship is deleted for some reason, no failure. Maybe a better Cypher error should point that out... /peter Sent from my phone, please excuse typos and autocorrection. On Nov 21, 2011 3:22 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Relationship not found error while traversing/querying (REST API)
Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
On Mon, Nov 21, 2011 at 3:22 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? No, not really. It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Cypher in 1.4 was very experimental. If you are using Cypher, I would suggest you move to 1.5 ASAP - a lot has happened there. If you still have the same problems with 1.5, then we can start looking for the problem. Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
