Re: [Neo4j] REST API (optimistic or transactional) concurrency?
of HTTP-capable client applications/platforms/languages. That said, I'd implement a pessimistic transaction recovery scheme with a configurable time to live. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org ] On Behalf Of Jacob Hansson Sent: Wednesday, May 11, 2011 10:58 AM To: Neo4j user discussions Subject: Re: [Neo4j] REST API (optimistic or transactional) concurrency? We started discussing how an implementation would look, but bumped into problems. The plan was to look at potentially adding a restful transaction API, something like create a transaction context resource via a POST, and then work with the database via that context resource, commiting/aborting it when done. The problem with this approach, and with any other HTTP-based approach that we could come up with that allows real over-the-wire transactions, is that we have no easy way of knowing if a client has crashed. We could potentially end up with a ton of open transactions holding locks on the DB, and wouldn't know if the clients were still around. With an approach that's closer to the network, we can roll back transactions if the connection dies, but doing that with HTTP is a lot harder. The idea of transactional support over the wire is still very much alive, but it will probably not be done over HTTP.. /Jake On Wed, May 11, 2011 at 12:21 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Just wanted to ask: how'd the lab day go last Friday? Any updates on this? =) Cheers, Aseem On Thu, May 5, 2011 at 7:42 AM, Mattias Persson matt...@neotechnology.comwrote: 2011/5/5 Aseem Kishore aseem.kish...@gmail.com Yes, great points. I've worked a good deal with Microsoft's cloud NoSQL DB, Windows Azure Table Storage, and for what it's worth, I thought they solved this problem very elegantly. That API is entirely REST-based, and they use HTTP If-Match / If-Modified-Since headers to achieve conditional requests, which gets them optimistic concurrency like I described below. E.g. if you want to read and update a row, your update request (e.g. PUT) is conditional based on what you expect the value to be (specified via the If-* header or headers). If it's no longer that value, the request will fail, so you can re-read and try again. This allows it to be nicely scalable since it doesn't require transactions AKA locks. Part of the problem for Neo4j though is that (a) the index is separate from the graph, which requires separate manual operations, and (b) the index supports multiple nodes under a single key/value pair, which doesn't lend itself to a fail if a node already exists metaphor. I know you guys are working on solving (a) in the next release, but I would also love an alternative to (b), e.g. being able to specify (whether at an index level, a key/value pair level, or a query/request level) that I *don't* want multiple nodes under the same key/value pair in an index. I have also think that such an index type is good to have, for just this and for performance reasons in that you can do low-level optimizations if you know that there will be only zero or one node/relationship for a given key/value pair in an index. Hopefully there will be time for that soon :) Thanks much for your consideration! And good luck. =) Aseem On Thu, May 5, 2011 at 1:00 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Assem, I agree. The problem is not the database API IMHO, but the mismatching semantics of HTTP REST and transactional DB APIs, and the balancing of shoving over work to the database server via server side processing scripting or plugins and client side operations via HTTP. Let's see if we can come up with a first prototype. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, May 5, 2011 at 9:26 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Thanks Peter. Looking forward to seeing what you guys come up with! Dima, thanks for the tip. It would be nice to not to have to write a custom plugin for what's arguably a pretty fundamental need from database APIs. =) Aseem On Wed, May 4, 2011 at 2:05 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Guys, Friday is labday, so we there are tentative plans to sketch on some transactional semantics in the REST API and circle back to the list
Re: [Neo4j] REST API (optimistic or transactional) concurrency?
I think you have to still do it over HTTP *in addition* to any other transport, if for no other reason than the ubiquity of HTTP-capable client applications/platforms/languages. That said, I'd implement a pessimistic transaction recovery scheme with a configurable time to live. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jacob Hansson Sent: Wednesday, May 11, 2011 10:58 AM To: Neo4j user discussions Subject: Re: [Neo4j] REST API (optimistic or transactional) concurrency? We started discussing how an implementation would look, but bumped into problems. The plan was to look at potentially adding a restful transaction API, something like create a transaction context resource via a POST, and then work with the database via that context resource, commiting/aborting it when done. The problem with this approach, and with any other HTTP-based approach that we could come up with that allows real over-the-wire transactions, is that we have no easy way of knowing if a client has crashed. We could potentially end up with a ton of open transactions holding locks on the DB, and wouldn't know if the clients were still around. With an approach that's closer to the network, we can roll back transactions if the connection dies, but doing that with HTTP is a lot harder. The idea of transactional support over the wire is still very much alive, but it will probably not be done over HTTP.. /Jake On Wed, May 11, 2011 at 12:21 PM, Aseem Kishore aseem.kish...@gmail.comwrote: Just wanted to ask: how'd the lab day go last Friday? Any updates on this? =) Cheers, Aseem On Thu, May 5, 2011 at 7:42 AM, Mattias Persson matt...@neotechnology.comwrote: 2011/5/5 Aseem Kishore aseem.kish...@gmail.com Yes, great points. I've worked a good deal with Microsoft's cloud NoSQL DB, Windows Azure Table Storage, and for what it's worth, I thought they solved this problem very elegantly. That API is entirely REST-based, and they use HTTP If-Match / If-Modified-Since headers to achieve conditional requests, which gets them optimistic concurrency like I described below. E.g. if you want to read and update a row, your update request (e.g. PUT) is conditional based on what you expect the value to be (specified via the If-* header or headers). If it's no longer that value, the request will fail, so you can re-read and try again. This allows it to be nicely scalable since it doesn't require transactions AKA locks. Part of the problem for Neo4j though is that (a) the index is separate from the graph, which requires separate manual operations, and (b) the index supports multiple nodes under a single key/value pair, which doesn't lend itself to a fail if a node already exists metaphor. I know you guys are working on solving (a) in the next release, but I would also love an alternative to (b), e.g. being able to specify (whether at an index level, a key/value pair level, or a query/request level) that I *don't* want multiple nodes under the same key/value pair in an index. I have also think that such an index type is good to have, for just this and for performance reasons in that you can do low-level optimizations if you know that there will be only zero or one node/relationship for a given key/value pair in an index. Hopefully there will be time for that soon :) Thanks much for your consideration! And good luck. =) Aseem On Thu, May 5, 2011 at 1:00 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Assem, I agree. The problem is not the database API IMHO, but the mismatching semantics of HTTP REST and transactional DB APIs, and the balancing of shoving over work to the database server via server side processing scripting or plugins and client side operations via HTTP. Let's see if we can come up with a first prototype. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, May 5, 2011 at 9:26 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Thanks Peter. Looking forward to seeing what you guys come up with! Dima, thanks for the tip. It would be nice to not to have to write a custom plugin for what's arguably a pretty fundamental need from database APIs. =) Aseem On Wed, May 4, 2011 at 2:05 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Guys, Friday is labday, so we
