[Neo4j] 1.5 question: auto indexes over REST API
Hi guys, Sorry if I'm a bit behind the times here ;) -- we're just now upgrading to 1.5. (But awesome that 1.6 has been released! We can go ahead upgrade to it right after this.) We use exclusively the REST API. We were excited about auto-indexing in 1.4, so we used it over the REST API taking advantage of the implementation detail (?) that the index name was node_auto_index. So e.g. we would query the auto index via: GET /db/data/index/node/*node_auto_index*/key/value This seems to still work after we upgrade to 1.5, which is good. But I'm looking through the 1.5 manual, and I saw this new section: http://docs.neo4j.org/chunked/1.5/rest-api-auto-indexes.html Which suggests this new syntax: GET http://localhost:7474/db/data/index/*auto*/node/key/value This seems to be new, but I can't find any issue on GitHub or any email about this. I also can't figure out how to use this myself -- as I query the REST API, I never get JSON that references this kind of URL. And if I try to query with this kind of URL myself, I always get a 404 or 400. What's the deal with this? Thanks! Aseem ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Suggestions on how to order relationships?
Hi there, Take a Facebook-like example, where users can like different movies/music/celebs/etc. Suppose we want to let users drag-and-drop these movies/etc. on their profile pages, to let them e.g. show their favorite movies first. How would you guys recommend achieving that in Neo4j? If possible, I'd like to avoid creating a node for every relationship (redundancy/overhead as we do this more and more). I can't think of anything better than putting properties on the relationship. Maybe index numbers (e.g. rel D might have index: 0, rel B might have index: 1, etc.), but that's essentially reordering an array, which sucks. The other option that thus came to mind was to mimic a linked list: have an after property that contains the ID of the relationship this one comes after (and/or maybe a before property instead or in addition). Just wondering if there are better ideas! Thanks. =) Aseem ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Suggestions on how to order relationships?
Not a bad idea. So something like: if a user drags/drops item X in between item Y and Z, set X's weight to halfway between Y's and Z's? Clever. =) On Thu, Jan 19, 2012 at 1:28 PM, Linan Wang tali.w...@gmail.com wrote: how about weight? On Thu, Jan 19, 2012 at 6:23 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hi there, Take a Facebook-like example, where users can like different movies/music/celebs/etc. Suppose we want to let users drag-and-drop these movies/etc. on their profile pages, to let them e.g. show their favorite movies first. How would you guys recommend achieving that in Neo4j? If possible, I'd like to avoid creating a node for every relationship (redundancy/overhead as we do this more and more). I can't think of anything better than putting properties on the relationship. Maybe index numbers (e.g. rel D might have index: 0, rel B might have index: 1, etc.), but that's essentially reordering an array, which sucks. The other option that thus came to mind was to mimic a linked list: have an after property that contains the ID of the relationship this one comes after (and/or maybe a before property instead or in addition). Just wondering if there are better ideas! Thanks. =) Aseem ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Best wishes, Linan Wang Architect, Programmer, PhD ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Suggestions on how to order relationships?
Can't say I'm a fan of this approach. You lose the ability to quickly determine if the user likes a particular movie, and storing the user (presumably their ID) as a property on the relationship feels like it defeats the point of a graph db, know what I mean? On Thu, Jan 19, 2012 at 1:29 PM, Rick Otten rot...@manta.com wrote: Another approach, probably not better, would be to only have a relationship from the user to the first movie in the list. Between the movies have relationships with the user as a property, so you could traverse from first to last (which would have no outgoing relationships for that user). You'd have to drop and add relationships to change the ordering (instead of changing properties on a set of relationships). -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, January 19, 2012 1:24 PM To: Neo4j user discussions Subject: [Neo4j] Suggestions on how to order relationships? Hi there, Take a Facebook-like example, where users can like different movies/music/celebs/etc. Suppose we want to let users drag-and-drop these movies/etc. on their profile pages, to let them e.g. show their favorite movies first. How would you guys recommend achieving that in Neo4j? If possible, I'd like to avoid creating a node for every relationship (redundancy/overhead as we do this more and more). I can't think of anything better than putting properties on the relationship. Maybe index numbers (e.g. rel D might have index: 0, rel B might have index: 1, etc.), but that's essentially reordering an array, which sucks. The other option that thus came to mind was to mimic a linked list: have an after property that contains the ID of the relationship this one comes after (and/or maybe a before property instead or in addition). Just wondering if there are better ideas! Thanks. =) Aseem ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
I'm definitely modifying the graph at the same time; that's why this is happening. But my assumption was that that shouldn't be breaking Neo4j -- I thought requests are transactional/serialized. I don't have a repro test at the moment, but the behavior is really simple. Here's an example analogy. We have a subgraph like User A [LIKES] Object 1 and also [LIKES] Object 2: (Object 1) --LIKES-- (User A) --LIKES-- (Object 2) One operation will delete the [LIKE] to Object 1, while another operation queries whether User A [LIKES] Object 2. The query can be a Cypher query, e.g.: START user=(123), obj=(456) MATCH (user)-[rel]-(obj) WHERE rel~TYPE = 'LIKES' RETURN rel But we've also seen this bug where we used a simple traverse (specifying just max depth 1 and the LIKES relationship type). When you say looks like a normal case, do you mean that the query should indeed fail like this? I wouldn't expect this -- what can I do to prevent an error? Should I be retrying? Thanks Peter! Aseem On Sun, Nov 20, 2011 at 10:08 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, What query are you running, and are you modifying your graph on any way at the same time? Do you have a small example test reproducing this? Looks like a normal case where this relationship is deleted for some reason, no failure. Maybe a better Cypher error should point that out... /peter Sent from my phone, please excuse typos and autocorrection. On Nov 21, 2011 3:22 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
On Sun, Nov 20, 2011 at 10:43 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Cypher in 1.4 was very experimental. If you are using Cypher, I would suggest you move to 1.5 ASAP - a lot has happened there. If you still have the same problems with 1.5, then we can start looking for the problem. Hmm, unfortunately we're in the middle of a milestone, so we haven't prioritized looking into upgrading yet, but to clarify, this bug happens with queries that use the traverse REST API as well, not just Cypher. But if this really is too difficult to pin down, I can ask again when we've upgraded to 1.5. Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
Mattias, thanks for the suggestion, but to clarify, we're using the server as-is, with the REST API; we're not using the Java API. Is there any way we can specify locking like you suggest via the server config or similar? Aseem On Mon, Nov 21, 2011 at 12:26 AM, Mattias Persson matt...@neotechnology.com wrote: I think for such a scenario you'd need read locks which gets upgraded to write locks when modifying. Consider this simple scenario w/o read locks: Thread ONE: get relationship R Thread TWO: get relationship R Thread TWO: delete relationship R and commit transaction Thread ONE: do any modification on relationship R... BOOM Exception If read locks were to be taken when getting relationships: Thread ONE: get relationship R Thread TWO: get relationship R Thread TWO: would like to delete relationship R, but will have to wait until Thread ONE releases its read lock on it Thread ONE: do some modification on relationship R and commit transaction Thread TWO: lock from Thread ONE was released so delete relationship R and commit transaction Unfortunately you cannot provide different isolation levels in neo4j at the moment, but you could mimic that behavior yourself using LockManager/LockReleaser (from graphDb.getConfig()) 2011/11/21 Aseem Kishore aseem.kish...@gmail.com Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship not found error while traversing/querying (REST API)
One more point of clarification: these are concurrent REST API requests. One request is deleting a relationship, another request is querying for a(nother) relationship. A simple analogy is a user unliking one thing on Facebook then immediately liking something else. I wouldn't expect the database to throw an error in this case, right? Am I wrong in assuming that the requests should be serialized / handled gracefully? If I'm wrong, should I just be optimistically retrying failed requests? Thanks again! Aseem On Mon, Nov 21, 2011 at 12:47 AM, Aseem Kishore aseem.kish...@gmail.comwrote: I'm definitely modifying the graph at the same time; that's why this is happening. But my assumption was that that shouldn't be breaking Neo4j -- I thought requests are transactional/serialized. I don't have a repro test at the moment, but the behavior is really simple. Here's an example analogy. We have a subgraph like User A [LIKES] Object 1 and also [LIKES] Object 2: (Object 1) --LIKES-- (User A) --LIKES-- (Object 2) One operation will delete the [LIKE] to Object 1, while another operation queries whether User A [LIKES] Object 2. The query can be a Cypher query, e.g.: START user=(123), obj=(456) MATCH (user)-[rel]-(obj) WHERE rel~TYPE = 'LIKES' RETURN rel But we've also seen this bug where we used a simple traverse (specifying just max depth 1 and the LIKES relationship type). When you say looks like a normal case, do you mean that the query should indeed fail like this? I wouldn't expect this -- what can I do to prevent an error? Should I be retrying? Thanks Peter! Aseem On Sun, Nov 20, 2011 at 10:08 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, What query are you running, and are you modifying your graph on any way at the same time? Do you have a small example test reproducing this? Looks like a normal case where this relationship is deleted for some reason, no failure. Maybe a better Cypher error should point that out... /peter Sent from my phone, please excuse typos and autocorrection. On Nov 21, 2011 3:22 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Activity Streams and Twitter Sample App
This is very interesting -- thanks Peter for the link, and thanks maxdemarzi for starting this conversation. In our social network -- which has extremely little load, it's just in beta -- we currently use basically (a), and it works just fine. We use Cypher to do the sorting/trimming on the server, e.g. roughly this query: START user=(123) MATCH (user) -[:FOLLOWS]- (friend) -[:EVENT]- (event) RETURN event ORDER BY event.timestamp DESC LIMIT 20 This obviously will not scale, since Neo4j has to read every single potential event from disk and process it in memory. We too have been thinking about this in the back of our minds, and the obvious solution in our minds is indeed to use a linked list timeline for events, and have a custom traverser that merge sorts in realtime. Since this'd be a linked list where you want the newest events at the head of the list, there will indeed be a small write-time cost of having to update two relationships whenever a new event is created (i.e. inserting it at the head of the list). Rene mentions this approach as the first (baseline) approach he tried. It's at 3:08 in the video in that post. He decides that approach isn't good enough, however, because merge sorting depends on the number of friends you have (# users you follow), which he wants to avoid at read-time. IMHO however, that's not worth optimizing for. It's rare that a user follows a very high number of other people. E.g. it's usually around let's say 100-1000 people. The flip side -- the # of *followers* -- can be very high. And interestingly, Rene's solution suffers from *that* at write-time. E.g. a new event has to update every *followers'* Graphity index. I'll provide Rene this feedback, but FWIW, I have the feeling that a simple timeline will easily be good enough, in fact optimal, for a social network on Neo4j: - O(1) writes - O(d) reads in space performance - O(d log d) reads in time performance Where d is the out-degree -- # of users you follow -- which is generally small. Cheers, Aseem On Mon, Nov 21, 2011 at 6:07 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: You might even be interested in Rene Pickards work on a full solution (albeit with some write-time overhead), see http://www.rene-pickhardt.de/graphity-an-efficient-graph-model-for-retrieving-the-top-k-news-feeds-for-users-in-social-networks/ Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 10:36 PM, maxdemarzi maxdema...@gmail.com wrote: I had not considered imperfect solutions, and in some activity stream scenarios a sampling of the last few messages could work. The sample would have to be taken from all Person nodes because if we sample from the Tweets in general and we encounter a chatty person node early on, it would take up all the sample space. Person1 - Person 2 - 10 tweets Person1 - Person3 - 1000 tweets = Sample Size reached, traversal stops Person1 - Person4 - 10 tweets Person1 - PersonX - 10 tweets Person3 would prevent Person4 to PersonX's tweets from ever making it to the sample. Some applications in domains like Financials or Network Monitoring may require the last known status and sampling might not be acceptable. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Activity-Streams-and-Twitter-Sample-App-tp3477669p3478477.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Relationship not found error while traversing/querying (REST API)
Hey guys, If we put our app under a bit of load, creating and removing nodes and relationships concurrently, sometimes we get back a 500 Internal Server Error from the REST API when we do a traverse or Cypher query. Here's an example stack trace: https://gist.github.com/1381423 We're running Neo4j. 1.4 still, but does this stack trace provide any insight/ideas into what might be wrong, what we could do as a workaround, etc.? Can I provide any other info that would help? It goes without saying that our assumption is that Neo4j shouldn't be failing/crashing on queries; our expectation was that operations are transactional, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Can't install server as a service on Amazon EC2 Linux
I'm new to Amazon EC2. I launched a new Amazon Linux (the basic, default image) instance, installed Neo4j on it, and tried to follow the instructions here: http://docs.neo4j.org/chunked/stable/server-installation.html#_linux_service Unfortunately, running `neo4j install` (as well as `neo4j remove`, and regardless of which user I specify or whether that user already exists or not) always results in this error: *update-rc.d: Command not found.* I'm looking into some ways to run startup scripts on EC2, but I'm not sure what the pros/cons are to starting Neo4j manually via a script vs. as a formal service. But just FYI -- can't install Neo4j as a service following the official instructions on EC2 Amazon Linux. Help appreciated. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Can't install server as a service on Amazon EC2 Linux
Sorry, should have mentioned that I made sure to do sudo, and sudo works for other commands just fine. Aseem On Wed, Nov 9, 2011 at 11:06 PM, Aseem Kishore aseem.kish...@gmail.comwrote: I'm new to Amazon EC2. I launched a new Amazon Linux (the basic, default image) instance, installed Neo4j on it, and tried to follow the instructions here: http://docs.neo4j.org/chunked/stable/server-installation.html#_linux_service Unfortunately, running `neo4j install` (as well as `neo4j remove`, and regardless of which user I specify or whether that user already exists or not) always results in this error: *update-rc.d: Command not found.* I'm looking into some ways to run startup scripts on EC2, but I'm not sure what the pros/cons are to starting Neo4j manually via a script vs. as a formal service. But just FYI -- can't install Neo4j as a service following the official instructions on EC2 Amazon Linux. Help appreciated. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Set property on node in Neo4j REST server
On Sun, Oct 16, 2011 at 6:06 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Looking at http://docs.neo4j.org/chunked/snapshot/rest-api-node-properties.html#rest-api-set-property-on-node you can set the property as just a string in the request, which is valid JSON. Hey Peter, I think the confusion may have stemmed from the fact that strings by themselves are *not* actually valid JSON as far as I understand. I believe valid JSON is *technically* only objects and arrays, though many libraries, in practice, tend to support primitive values as valid JSON too. It's certainly convenient, and I prefer that, too. E.g. http://jsonlint.com/ validates {} and [] but not or 1 or true: Parse error on line 1: ^ Expecting '{', '[' Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Recommended way to index and lookup paired properties
Can anyone help w/ this question? =) Thanks! Aseem On Thu, Sep 8, 2011 at 9:58 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Hey guys, quick question on indexing. We track Amazon products in our db, and the way Amazon identifies its products is with an Amazon Standard Identification Number (ASIN). The only catch is that ASINs are only unique within a particular locale, and Amazon has a few different locales (e.g. us, uk, de, jp, etc.). Because of this, when we index and lookup Amazon products, we need to index and lookup with *both* properties together. E.g. we really want to index the pair. As such -- and because we use Neo4j's auto-indexing -- we created a shadow locale+asin property that sits alongside the regular locale and asin properties, and it's a concatenation of the two values, e.g. us+A123456789. We thus index and perform lookups via this shadow property. Recently, though, I saw that you can query even exact indexes with complex Lucene syntax, so in theory, we could ditch the shadow property and query something like (locale:us AND asin:A123456789). Will this be slower though, if there many items w/ the same locale? Generalizing, then, which of these two routes is the recommended way of indexing paired properties? (Or is there a third way?) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
Awesome! Great to hear. Thanks Andrés! Aseem On Wed, Sep 14, 2011 at 12:58 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nice! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Sep 14, 2011 at 9:56 PM, Andres Taylor andres.tay...@neotechnology.com wrote: We finally decided that it should be added. http://docs.neo4j.org/chunked/snapshot/query-function.html#functions-id It's in snapshot, and will be part of our next stable release. Andrés On Thu, Aug 25, 2011 at 2:39 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, Is there any way I can have Cypher return/print node IDs? If I specify for it to return a node, it returns/prints all data about that node, which can be excessive. But the problem is, the node's ID isn't a property on that node. I've tried both node~ID and node~SELF, like rel~TYPE, but neither works. If this isn't possible today, consider it a feature request. =) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: how would you query first- and second-degree friends?
Awesome! Thanks again! Aseem On Wed, Sep 14, 2011 at 12:59 PM, Andres Taylor andres.tay...@neotechnology.com wrote: And now it's in code. http://docs.neo4j.org/chunked/snapshot/query-match.html#match-optional-relationship Hope that helps, Andrés On Thu, Aug 25, 2011 at 10:34 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Good and valid requests, Aseem. Consider them heard. We'll see how soon we can get it to you. Andrés On Thu, Aug 25, 2011 at 12:57 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Size on disk, and number of properties
That is awesomely helpful info. Thank you very much! Aseem On Thu, Sep 8, 2011 at 12:55 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Assem The logical log file is where all changes to be performed to the store are written out before they are actually applied - also referred to as the Write Ahead Log. The file that hosts this is bound to a specific size above which a rotation happens: a new file is created to host the WAL, all pending transactions are moved over, the store files are flushed and the old file marked as unused. Setting the configuration option keep_logical_logs to false deletes the old log files instead of keeping them around when that size limit is hit and a rotation is triggered. Hope that clear things up. cheers, CG On Wed, Sep 7, 2011 at 8:58 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Thanks Johan! Configuring Neo4j with keep_logical_logs=false the logs will automatically be deleted upon rotation. What does upon rotation mean here? Aseem On Wed, Sep 7, 2011 at 5:56 AM, Johan Svensson jo...@neotechnology.com wrote: Removing the log files ending with .vversion number at runtime is perfectly safe to do but will turn off the ability to do incremental backups. You can however still perform live full backups. Configuring Neo4j with keep_logical_logs=false the logs will automatically be deleted upon rotation. -Johan On Sat, Sep 3, 2011 at 1:49 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Thanks for the insights Johan! Regarding the existing disk space then, by far the bulk of it is from the logs. Is there a way to prune or garbage collect them? Is simply deleting the files safe? Should the db be off if I do that? Etc. Thanks much! Aseem On Tue, Aug 30, 2011 at 2:47 AM, Johan Svensson jo...@neotechnology.com wrote: Hi Aseem, This is actually expected behavior when performing file copy of running db and starting up with default configuration. If you remove the files ending with .id in the db directory on the local snapshot and start up setting rebuild_idgenerators_fast=false you should see the accurate amount of nodes, relationships and properties. Regarding the amount of properties not matching this could be due to a non clean shutdown on the production system. We are planing on improving this in the near future by allowing for more aggressive reuse of ids for properties. This will specifically improve things for workloads that perform a lot of property updates. -Johan On Tue, Aug 30, 2011 at 10:05 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, We do offline backups of our db on a semi-regular basis (every few days), where we (1) stop the running db, (2) copy its data directory and (3) restart the db. A few times early on, we did running backups -- but not the proper online way -- where we simply copied the data directory while the db was still running. (We did this during times where we were confident no requests were hitting the db.) We noticed that every time we did the running backup, the number of properties the web admin reported -- and the space on disk of the db -- would jump up quite a bit. We stopped doing that recently. But even now, both these numbers have gotten quite a bit higher than we expect to, and strangely, they seem to differ highly between the running db and the copies. What could be causing all of this? Here are our current numbers: *Production* - 2,338 nodes - 4,473 rels - 114,231 props (higher than we would expect it to be, but not by an order of magnitude) - *1.39 GB!* -- this is way unexpected, particularly since our db used to be in the ~10 KB ballpark, and we certainly haven't experienced hockey stick growth yet ;) The logical log only takes up 57 KB (0%) btw. *Local snapshot* - 2,338 nodes - 4,473 rels - *2,607,892 props!!!* -- ??? - *1.37 GB!* -- equally surprisingly high, but also interesting that it's less than the production db's size. 0 KB logical logs. I looked around the wiki and searched this mailing list but didn't find much clues here. But as requested on another thread, here's the output of `ls -lh data/graph.db/`: total 1474520 -rw-r--r-- 1 aseemk staff11B Aug 30 00:46 active_tx_log drwxr-xr-x 52 aseemk staff 1.7K Aug 30 00:46 index/ -rw-r--r-- 1 aseemk staff 343B Aug 30 00:46 index.db -rw-r--r-- 1 aseemk staff 854K Aug 30 00:46 messages.log -rw-r--r-- 1 aseemk staff36B Aug 30 00:46 neostore -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.id -rw-r--r-- 1 aseemk staff26K Aug 30 00:46 neostore.nodestore.db -rw-r--r-- 1 aseemk staff 9B Aug 30
[Neo4j] Recommended way to index and lookup paired properties
Hey guys, quick question on indexing. We track Amazon products in our db, and the way Amazon identifies its products is with an Amazon Standard Identification Number (ASIN). The only catch is that ASINs are only unique within a particular locale, and Amazon has a few different locales (e.g. us, uk, de, jp, etc.). Because of this, when we index and lookup Amazon products, we need to index and lookup with *both* properties together. E.g. we really want to index the pair. As such -- and because we use Neo4j's auto-indexing -- we created a shadow locale+asin property that sits alongside the regular locale and asin properties, and it's a concatenation of the two values, e.g. us+A123456789. We thus index and perform lookups via this shadow property. Recently, though, I saw that you can query even exact indexes with complex Lucene syntax, so in theory, we could ditch the shadow property and query something like (locale:us AND asin:A123456789). Will this be slower though, if there many items w/ the same locale? Generalizing, then, which of these two routes is the recommended way of indexing paired properties? (Or is there a third way?) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Size on disk, and number of properties
Thanks Johan! Configuring Neo4j with keep_logical_logs=false the logs will automatically be deleted upon rotation. What does upon rotation mean here? Aseem On Wed, Sep 7, 2011 at 5:56 AM, Johan Svensson jo...@neotechnology.comwrote: Removing the log files ending with .vversion number at runtime is perfectly safe to do but will turn off the ability to do incremental backups. You can however still perform live full backups. Configuring Neo4j with keep_logical_logs=false the logs will automatically be deleted upon rotation. -Johan On Sat, Sep 3, 2011 at 1:49 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Thanks for the insights Johan! Regarding the existing disk space then, by far the bulk of it is from the logs. Is there a way to prune or garbage collect them? Is simply deleting the files safe? Should the db be off if I do that? Etc. Thanks much! Aseem On Tue, Aug 30, 2011 at 2:47 AM, Johan Svensson jo...@neotechnology.com wrote: Hi Aseem, This is actually expected behavior when performing file copy of running db and starting up with default configuration. If you remove the files ending with .id in the db directory on the local snapshot and start up setting rebuild_idgenerators_fast=false you should see the accurate amount of nodes, relationships and properties. Regarding the amount of properties not matching this could be due to a non clean shutdown on the production system. We are planing on improving this in the near future by allowing for more aggressive reuse of ids for properties. This will specifically improve things for workloads that perform a lot of property updates. -Johan On Tue, Aug 30, 2011 at 10:05 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, We do offline backups of our db on a semi-regular basis (every few days), where we (1) stop the running db, (2) copy its data directory and (3) restart the db. A few times early on, we did running backups -- but not the proper online way -- where we simply copied the data directory while the db was still running. (We did this during times where we were confident no requests were hitting the db.) We noticed that every time we did the running backup, the number of properties the web admin reported -- and the space on disk of the db -- would jump up quite a bit. We stopped doing that recently. But even now, both these numbers have gotten quite a bit higher than we expect to, and strangely, they seem to differ highly between the running db and the copies. What could be causing all of this? Here are our current numbers: *Production* - 2,338 nodes - 4,473 rels - 114,231 props (higher than we would expect it to be, but not by an order of magnitude) - *1.39 GB!* -- this is way unexpected, particularly since our db used to be in the ~10 KB ballpark, and we certainly haven't experienced hockey stick growth yet ;) The logical log only takes up 57 KB (0%) btw. *Local snapshot* - 2,338 nodes - 4,473 rels - *2,607,892 props!!!* -- ??? - *1.37 GB!* -- equally surprisingly high, but also interesting that it's less than the production db's size. 0 KB logical logs. I looked around the wiki and searched this mailing list but didn't find much clues here. But as requested on another thread, here's the output of `ls -lh data/graph.db/`: total 1474520 -rw-r--r-- 1 aseemk staff11B Aug 30 00:46 active_tx_log drwxr-xr-x 52 aseemk staff 1.7K Aug 30 00:46 index/ -rw-r--r-- 1 aseemk staff 343B Aug 30 00:46 index.db -rw-r--r-- 1 aseemk staff 854K Aug 30 00:46 messages.log -rw-r--r-- 1 aseemk staff36B Aug 30 00:46 neostore -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.id -rw-r--r-- 1 aseemk staff26K Aug 30 00:46 neostore.nodestore.db -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.nodestore.db.id -rw-r--r-- 1 aseemk staff62M Aug 30 00:46 neostore.propertystore.db -rw-r--r-- 1 aseemk staff 133B Aug 30 00:46 neostore.propertystore.db.arrays -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.arrays.id -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.id -rw-r--r-- 1 aseemk staff 1.0K Aug 30 00:46 neostore.propertystore.db.index -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.index.id -rw-r--r-- 1 aseemk staff 4.0K Aug 30 00:46 neostore.propertystore.db.index.keys -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.index.keys.id -rw-r--r-- 1 aseemk staff69M Aug 30 00:46 neostore.propertystore.db.strings -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.strings.id -rw-r--r-- 1 aseemk staff 144K Aug 30 00:46
Re: [Neo4j] REST API: errors should always be JSON, not HTML
Hey Peter, sorry for not being totally clear. It should indeed be an *HTTP* error, e.g. 400 or whatever. The problem is that the *body* is HTML when it should be JSON. That is, you should see a bunch of HTML tags in the body, instead of a simple {error: ..., stacktrace: ...} object. Cheers, Aseem On Wed, Sep 7, 2011 at 7:12 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, is this still relevant? I get the HTTP error with POST /db/data/ext/CypherPlugin/graphdb/execute_query {query: START n=(0) RETURN n.foobar} Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Aug 30, 2011 at 7:59 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey, Just ran into an HTML error response which broke our error handling code since it was expecting the body to be JSON. It seems to only happen with the Cypher plugin. Here's a simple repro -- just paste this into the HTTP console in the web admin: POST /db/data/ext/CypherPlugin/graphdb/execute_query {query: START n=(0) RETURN n.foobar} Assuming a node w/ ID 0 exists, and assuming it has no foobar property, this should return an HTML error. Let me know if my assumption here that errors should always be JSON is an incorrect one! We've worked around this for now, but just reporting as a bug. Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API: errors should always be JSON, not HTML
Sure, Peter. Sorry I haven't done this yet for other issues I've reported! Aseem On Wed, Sep 7, 2011 at 11:42 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, do you care to raise an issue at https://github.com/neo4j/community/issues?sort=createddirection=descstate=openfor this? I understand - this sounds valid. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Sep 7, 2011 at 8:00 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey Peter, sorry for not being totally clear. It should indeed be an *HTTP* error, e.g. 400 or whatever. The problem is that the *body* is HTML when it should be JSON. That is, you should see a bunch of HTML tags in the body, instead of a simple {error: ..., stacktrace: ...} object. Cheers, Aseem On Wed, Sep 7, 2011 at 7:12 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, is this still relevant? I get the HTTP error with POST /db/data/ext/CypherPlugin/graphdb/execute_query {query: START n=(0) RETURN n.foobar} Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Aug 30, 2011 at 7:59 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey, Just ran into an HTML error response which broke our error handling code since it was expecting the body to be JSON. It seems to only happen with the Cypher plugin. Here's a simple repro -- just paste this into the HTTP console in the web admin: POST /db/data/ext/CypherPlugin/graphdb/execute_query {query: START n=(0) RETURN n.foobar} Assuming a node w/ ID 0 exists, and assuming it has no foobar property, this should return an HTML error. Let me know if my assumption here that errors should always be JSON is an incorrect one! We've worked around this for now, but just reporting as a bug. Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API: errors should always be JSON, not HTML
https://github.com/neo4j/community/issues/13 Cheers, Aseem On Wed, Sep 7, 2011 at 11:57 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Sure, Peter. Sorry I haven't done this yet for other issues I've reported! Aseem On Wed, Sep 7, 2011 at 11:42 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, do you care to raise an issue at https://github.com/neo4j/community/issues?sort=createddirection=descstate=openfor this? I understand - this sounds valid. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Sep 7, 2011 at 8:00 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey Peter, sorry for not being totally clear. It should indeed be an *HTTP* error, e.g. 400 or whatever. The problem is that the *body* is HTML when it should be JSON. That is, you should see a bunch of HTML tags in the body, instead of a simple {error: ..., stacktrace: ...} object. Cheers, Aseem On Wed, Sep 7, 2011 at 7:12 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, is this still relevant? I get the HTTP error with POST /db/data/ext/CypherPlugin/graphdb/execute_query {query: START n=(0) RETURN n.foobar} Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Aug 30, 2011 at 7:59 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey, Just ran into an HTML error response which broke our error handling code since it was expecting the body to be JSON. It seems to only happen with the Cypher plugin. Here's a simple repro -- just paste this into the HTTP console in the web admin: POST /db/data/ext/CypherPlugin/graphdb/execute_query {query: START n=(0) RETURN n.foobar} Assuming a node w/ ID 0 exists, and assuming it has no foobar property, this should return an HTML error. Let me know if my assumption here that errors should always be JSON is an incorrect one! We've worked around this for now, but just reporting as a bug. Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] REST API: errors should always be JSON, not HTML
Hey, Just ran into an HTML error response which broke our error handling code since it was expecting the body to be JSON. It seems to only happen with the Cypher plugin. Here's a simple repro -- just paste this into the HTTP console in the web admin: POST /db/data/ext/CypherPlugin/graphdb/execute_query {query: START n=(0) RETURN n.foobar} Assuming a node w/ ID 0 exists, and assuming it has no foobar property, this should return an HTML error. Let me know if my assumption here that errors should always be JSON is an incorrect one! We've worked around this for now, but just reporting as a bug. Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
Yep. Sorry for the delay in responding. For that query, here's the stack trace: java.lang.Character.codePointAt(Character.java:2335) java.util.regex.Pattern$CharProperty.match(Pattern.java:3344) { java.util.regex.Pattern$Branch.match(Pattern.java:4114) java.util.regex.Pattern$GroupHead.match(Pattern.java:4168) java.util.regex.Pattern$Loop.match(Pattern.java:4295) java.util.regex.Pattern$GroupTail.match(Pattern.java:4227) java.util.regex.Pattern$BranchConn.match(Pattern.java:4078) java.util.regex.Pattern$CharProperty.match(Pattern.java:3345) } ... [the part in {} keep repeating; there are 1024 total lines in the stack trace] And it's a java.lang.StackOverflowError exception. The same query works over the index REST API, e.g.: http://localhost:7474/db/data/index/node/node_auto_index?query=uuid_indexed:(ea024173-9740-4ed6-875b-813ba9801571%20OR%20d74b846a-437f-4ff3-88e2-cc6303b1d137%20OR%207ae40c9d-94a0-4c47-8823-833c8e33cfa9%20OR%20c8f40938-2746-4b6b-87aa-0f653671c81f%20OR%20805d0391-a228-4279-b46d-df2d48b0de84%20OR%20231d56ef-5ccc-46ec-bbfa-581115efbb22%20OR%20efaf7c78-3835-433e-86e3-22915c962cca%20OR%203abceafd-e691-4537-a146-49f0af92fabc%20OR%20d80829f4-5671-4344-8dce-162c86cd769b%20OR%203943443c-e780-464f-85fb-8526157c72c9%20OR%20041d7cd5-1d3d-4ec1-943b-1bb59eff39e5%20OR%204fe02747-a7f0-4d48-91be-38544a870934%20OR%203efe3397-36ad-45d1-99e2-97fa04a3fb2d%20OR%20e8d72867-6939-43f4-bc1b-2b4e7e8eb5c9%20OR%208bf755b7-6580-4573-8aee-18a6d6f314e1%20OR%20f122cc05-8937-4175-a713-fee5c3d2a377%20OR%20d05c5824-79ff-42d0-b975-a2da54313d42%20OR%20a71d5af6-089b-47d0-8cfe-16e8059a8306%20OR%2007dc48d6-6256-4ba0-bb26-835ab1c9152a%20OR%20cb8dac45-273f-496b-90e6-3e38eb85f32d%20OR%201735ef28-0b90-443f-b8af-eb77219d5283%20OR%20b705503e-522f-49e2-aaad-571d9b007978%20OR%20f6e43634-b23e-4ff8-895f-6a6c246b95a3%20OR%20d0d9ba5d-ed6e-4493-88ff-93fc75a98a9b%20OR%20688b11fa-03d4-4b6e-8151-733b7e33bd50%20OR%20d6da65c3-e864-4d69-acd0-3ee88eb9cb5c%20OR%20da4bf760-c844-4b36-ae9c-86e65b8be70b%20OR%203c36c953-9757-4874-9620-ab530233a9c2%20OR%20c6ea8316-3eed-4a59-b357-4d6639fa1778%20OR%205231ede6-407b-497c-8081-358c5d03d997%20OR%205f0cf4f2-9225-4eef-85bd-baa31e95634e%20OR%204707f732-7b9c-4703-a269-1f7f0320b495%20OR%20f6136c77-dfd0-4f00-8f03-4508ec8bc234%20OR%2046793cc8-8984-4efe-b958-81a50d126601%20OR%20a1249bda-2c90-4bfd-a2de-e3afd4492a63%20OR%20790372af-6ff4-424d-8668-fa606d585e92%20OR%2018697862-6f8f-40fb-9c3f-9b65daad7b6a%20OR%204467c415-3d2d-4569-a82b-441d8507617c%20OR%2090bc8bcb-8078-4dad-935d-c6feccaa2e41%20OR%2078cf6f7a-250e-4cf7-9ed8-cc790fc23540%20OR%20f65b4132-d646-416f-ba4a-42683afaf821%20OR%20cffa88a9-33b7-411b-81d5-7b65b50736df%20OR%201beaf9b7-086d-4fc3-8ec9-8e1af406a132%20OR%2096249323-ea95-4ab5-9dbd-2bf58965c224%20OR%20cc04cb76-0c94-4c6a-a26d-1fcae979ec92%20OR%2073174682-00ed-4bfd-89d3-0f7efd6736b3%20OR%200cfb807b-45b5-40f7-a418-4ad62f540910%20OR%203f446054-c18a-47c3-a82d-473ed85a50e9%20OR%20a81ffda4-3cae-4e01-8fb1-e0e6c320c9b2%20OR%206c8b5ca8-65c9-4edd-a479-9645cb1ab767%20OR%20d6c22d84-2a6b-4407-bd77-c471fe3662ed%20OR%207d695ebd-b2b2-4611-9b30-39f4ce713a45%20OR%2020486616-db45-4da2-9749-87da3126d1ec%20OR%2051e1107e-9adc-4a2a-ac8b-877a2e2d8b49%20OR%20df5ae476-9b88-4371-91d9-ec8e871ce69e%20OR%20a0540ccd-037e-4247-bbab-8deefd113450%20OR%203a6fd8b4-7a4e-4acb-bdb1-0d7bfe281d3b%20OR%208234cf3b-d0ff-402c-b3ac-7d7ce6838378%20OR%200dda199a-e348-4f90-8fcf-2418c13a5978%20OR%206d752af1-f7e5-488e-94c4-b270f2ef57a9%20OR%2069633e3f-d4ff-4a53-8bd7-77e67fcfc1a5%20OR%2014142a0e-6f61-4bcc-a8b9-b030d20bad6b%20OR%209f55e828-cab0-4bb3-a91a-26bd14529e59%20OR%2031e0e7c7-9aa6-4f44-9c14-2be5d0192b55%20OR%20a17de99f-3348-4cd6-9aa3-30203befc0ac%20OR%20f95af42c-d8c4-43e2-b597-967cf73404f9%20OR%20d9c7c9b4-2e05-4044-a7f0-2b9238d1624a%20OR%2021140f57-c8f2-4746-a47b-abed95c0bed2%20OR%209c3b8c5a-ed1b-4c5c-ac60-7016b5846468%20OR%2038857947-ff47-4da7-8425-761d57bfcfd3%20OR%20422b65a8-59d7-48fb-8853-15f6d372654c%20OR%20597779c5-490b-4e10-b0a6-131315d4198c%20OR%208a5833fe-500a-4c55-8031-713ee093655c%20OR%207c02f1b6-c99b-42f4-9ac6-049c09fa49c7%20OR%20c0bceba8-84d3-4b04-a4f5-1d8d1dcaa233%20OR%2060b0b178-98a9-4819-859b-4d36eed7f3a3) Hope that helps! Aseem On Fri, Aug 26, 2011 at 5:01 AM, Andres Taylor andres.tay...@neotechnology.com wrote: Do you have a stack trace to share? Andrés On Fri, Aug 26, 2011 at 10:55 AM, Aseem Kishore aseem.kish...@gmail.com wrote: I spoke too soon -- this doesn't seem to scale very far with many UUIDs being looked up together. Here's a query looking up 76 things that returns a 500 error -- due to a stack overflow exception. *START*node=(node_auto_index,'uuid_indexed:(f93d416a-40ae-4b5a-87d7-53b6bc4b54ab OR ceaad2cf-1afb-4cdd-b98a-8706dc4c471a OR 7bb157dd-ff76-49d3-9b01-0c506909400c OR 18b06833-b763-419c-b0ba-78e43dd0898d OR fdbaf139-053c-4886-8855-47efc4802f60 OR 8de5a2fb-5a42-4386-9b5a-8d5cbbb3a491 OR a9a54c63-6fb4-489e-bf55-ebca2ef8e2c3 OR 17c2c15e-f116-4092-afbe-336674f7b3ed
[Neo4j] Minor convenience request: Cypher default console =)
What do you guys say to making Cypher the default view in the web admin console? It feels the best as a query language, it's read-only (safe default), and going Cypher, Gremlin, HTTP is even alphabetical. ;) I just ask this selfishly because I find myself repeatedly going to the console now that I know Cypher (so nice!) and it's always an extra click to be able to use it. Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
Doesn't seem to be a character escaping issue, as doing the same query w/ the same syntax w/ just fewer UUIDs works just fine. E.g.: POST /db/data/ext/CypherPlugin/graphdb/execute_query {query: START node=(node_auto_index,'uuid_indexed:(f93d416a-40ae-4b5a-87d7-53b6bc4b54ab OR ceaad2cf-1afb-4cdd-b98a-8706dc4c471a OR 7bb157dd-ff76-49d3-9b01-0c506909400c)') RETURN node} Let me know if I'm misunderstanding you! Aseem On Tue, Aug 30, 2011 at 12:15 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, this sounds like a parsing exception. Maybe there are characters escaped in REST, but not in plain Cypher. Is there a way you could create a graph with only one or two UUID nodes and reproduce the stack trace so we can look at it? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Aug 30, 2011 at 8:13 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Yep. Sorry for the delay in responding. For that query, here's the stack trace: java.lang.Character.codePointAt(Character.java:2335) java.util.regex.Pattern$CharProperty.match(Pattern.java:3344) { java.util.regex.Pattern$Branch.match(Pattern.java:4114) java.util.regex.Pattern$GroupHead.match(Pattern.java:4168) java.util.regex.Pattern$Loop.match(Pattern.java:4295) java.util.regex.Pattern$GroupTail.match(Pattern.java:4227) java.util.regex.Pattern$BranchConn.match(Pattern.java:4078) java.util.regex.Pattern$CharProperty.match(Pattern.java:3345) } ... [the part in {} keep repeating; there are 1024 total lines in the stack trace] And it's a java.lang.StackOverflowError exception. The same query works over the index REST API, e.g.: http://localhost:7474/db/data/index/node/node_auto_index?query=uuid_indexed:(ea024173-9740-4ed6-875b-813ba9801571%20OR%20d74b846a-437f-4ff3-88e2-cc6303b1d137%20OR%207ae40c9d-94a0-4c47-8823-833c8e33cfa9%20OR%20c8f40938-2746-4b6b-87aa-0f653671c81f%20OR%20805d0391-a228-4279-b46d-df2d48b0de84%20OR%20231d56ef-5ccc-46ec-bbfa-581115efbb22%20OR%20efaf7c78-3835-433e-86e3-22915c962cca%20OR%203abceafd-e691-4537-a146-49f0af92fabc%20OR%20d80829f4-5671-4344-8dce-162c86cd769b%20OR%203943443c-e780-464f-85fb-8526157c72c9%20OR%20041d7cd5-1d3d-4ec1-943b-1bb59eff39e5%20OR%204fe02747-a7f0-4d48-91be-38544a870934%20OR%203efe3397-36ad-45d1-99e2-97fa04a3fb2d%20OR%20e8d72867-6939-43f4-bc1b-2b4e7e8eb5c9%20OR%208bf755b7-6580-4573-8aee-18a6d6f314e1%20OR%20f122cc05-8937-4175-a713-fee5c3d2a377%20OR%20d05c5824-79ff-42d0-b975-a2da54313d42%20OR%20a71d5af6-089b-47d0-8cfe-16e8059a8306%20OR%2007dc48d6-6256-4ba0-bb26-835ab1c9152a%20OR%20cb8dac45-273f-496b-90e6-3e38eb85f32d%20OR%201735ef28-0b90-443f-b8af-eb77219d5283%20OR%20b705503e-522f-49e2-aaad-571d9b007978%20OR%20f6e43634-b23e-4ff8-895f-6a6c246b95a3%20OR%20d0d9ba5d-ed6e-4493-88ff-93fc75a98a9b%20OR%20688b11fa-03d4-4b6e-8151-733b7e33bd50%20OR%20d6da65c3-e864-4d69-acd0-3ee88eb9cb5c%20OR%20da4bf760-c844-4b36-ae9c-86e65b8be70b%20OR%203c36c953-9757-4874-9620-ab530233a9c2%20OR%20c6ea8316-3eed-4a59-b357-4d6639fa1778%20OR%205231ede6-407b-497c-8081-358c5d03d997%20OR%205f0cf4f2-9225-4eef-85bd-baa31e95634e%20OR%204707f732-7b9c-4703-a269-1f7f0320b495%20OR%20f6136c77-dfd0-4f00-8f03-4508ec8bc234%20OR%2046793cc8-8984-4efe-b958-81a50d126601%20OR%20a1249bda-2c90-4bfd-a2de-e3afd4492a63%20OR%20790372af-6ff4-424d-8668-fa606d585e92%20OR%2018697862-6f8f-40fb-9c3f-9b65daad7b6a%20OR%204467c415-3d2d-4569-a82b-441d8507617c%20OR%2090bc8bcb-8078-4dad-935d-c6feccaa2e41%20OR%2078cf6f7a-250e-4cf7-9ed8-cc790fc23540%20OR%20f65b4132-d646-416f-ba4a-42683afaf821%20OR%20cffa88a9-33b7-411b-81d5-7b65b50736df%20OR%201beaf9b7-086d-4fc3-8ec9-8e1af406a132%20OR%2096249323-ea95-4ab5-9dbd-2bf58965c224%20OR%20cc04cb76-0c94-4c6a-a26d-1fcae979ec92%20OR%2073174682-00ed-4bfd-89d3-0f7efd6736b3%20OR%200cfb807b-45b5-40f7-a418-4ad62f540910%20OR%203f446054-c18a-47c3-a82d-473ed85a50e9%20OR%20a81ffda4-3cae-4e01-8fb1-e0e6c320c9b2%20OR%206c8b5ca8-65c9-4edd-a479-9645cb1ab767%20OR%20d6c22d84-2a6b-4407-bd77-c471fe3662ed%20OR%207d695ebd-b2b2-4611-9b30-39f4ce713a45%20OR%2020486616-db45-4da2-9749-87da3126d1ec%20OR%2051e1107e-9adc-4a2a-ac8b-877a2e2d8b49%20OR%20df5ae476-9b88-4371-91d9-ec8e871ce69e%20OR%20a0540ccd-037e-4247-bbab-8deefd113450%20OR%203a6fd8b4-7a4e-4acb-bdb1-0d7bfe281d3b%20OR%208234cf3b-d0ff-402c-b3ac-7d7ce6838378%20OR%200dda199a-e348-4f90-8fcf-2418c13a5978%20OR%206d752af1-f7e5-488e-94c4-b270f2ef57a9%20OR%2069633e3f-d4ff-4a53-8bd7-77e67fcfc1a5%20OR%2014142a0e-6f61-4bcc-a8b9-b030d20bad6b%20OR%209f55e828-cab0-4bb3-a91a-26bd14529e59%20OR%2031e0e7c7-9aa6-4f44-9c14-2be5d0192b55%20OR%20a17de99f-3348-4cd6-9aa3-30203befc0ac%20OR%20f95af42c-d8c4
[Neo4j] Size on disk, and number of properties
Hey guys, We do offline backups of our db on a semi-regular basis (every few days), where we (1) stop the running db, (2) copy its data directory and (3) restart the db. A few times early on, we did running backups -- but not the proper online way -- where we simply copied the data directory while the db was still running. (We did this during times where we were confident no requests were hitting the db.) We noticed that every time we did the running backup, the number of properties the web admin reported -- and the space on disk of the db -- would jump up quite a bit. We stopped doing that recently. But even now, both these numbers have gotten quite a bit higher than we expect to, and strangely, they seem to differ highly between the running db and the copies. What could be causing all of this? Here are our current numbers: *Production* - 2,338 nodes - 4,473 rels - 114,231 props (higher than we would expect it to be, but not by an order of magnitude) - *1.39 GB!* -- this is way unexpected, particularly since our db used to be in the ~10 KB ballpark, and we certainly haven't experienced hockey stick growth yet ;) The logical log only takes up 57 KB (0%) btw. *Local snapshot* - 2,338 nodes - 4,473 rels - *2,607,892 props!!!* -- ??? - *1.37 GB!* -- equally surprisingly high, but also interesting that it's less than the production db's size. 0 KB logical logs. I looked around the wiki and searched this mailing list but didn't find much clues here. But as requested on another thread, here's the output of `ls -lh data/graph.db/`: total 1474520 -rw-r--r-- 1 aseemk staff11B Aug 30 00:46 active_tx_log drwxr-xr-x 52 aseemk staff 1.7K Aug 30 00:46 index/ -rw-r--r-- 1 aseemk staff 343B Aug 30 00:46 index.db -rw-r--r-- 1 aseemk staff 854K Aug 30 00:46 messages.log -rw-r--r-- 1 aseemk staff36B Aug 30 00:46 neostore -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.id -rw-r--r-- 1 aseemk staff26K Aug 30 00:46 neostore.nodestore.db -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.nodestore.db.id -rw-r--r-- 1 aseemk staff62M Aug 30 00:46 neostore.propertystore.db -rw-r--r-- 1 aseemk staff 133B Aug 30 00:46 neostore.propertystore.db.arrays -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.arrays.id -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.id -rw-r--r-- 1 aseemk staff 1.0K Aug 30 00:46 neostore.propertystore.db.index -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.index.id -rw-r--r-- 1 aseemk staff 4.0K Aug 30 00:46 neostore.propertystore.db.index.keys -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.index.keys.id -rw-r--r-- 1 aseemk staff69M Aug 30 00:46 neostore.propertystore.db.strings -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.propertystore.db.strings.id -rw-r--r-- 1 aseemk staff 144K Aug 30 00:46 neostore.relationshipstore.db -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.relationshipstore.db.id -rw-r--r-- 1 aseemk staff55B Aug 30 00:46 neostore.relationshiptypestore.db -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.relationshiptypestore.db.id -rw-r--r-- 1 aseemk staff 602B Aug 30 00:46 neostore.relationshiptypestore.db.names -rw-r--r-- 1 aseemk staff 9B Aug 30 00:46 neostore.relationshiptypestore.db.names.id -rw-r--r-- 1 aseemk staff16B Aug 30 00:46 nioneo_logical.log.1 -rw-r--r-- 1 aseemk staff 4B Aug 30 00:46 nioneo_logical.log.active -rw-r--r-- 1 aseemk staff 945K Aug 30 00:46 nioneo_logical.log.v0 -rw-r--r-- 1 aseemk staff16B Aug 30 00:46 nioneo_logical.log.v1 -rw-r--r-- 1 aseemk staff33K Aug 30 00:46 nioneo_logical.log.v10 -rw-r--r-- 1 aseemk staff11K Aug 30 00:46 nioneo_logical.log.v11 -rw-r--r-- 1 aseemk staff32K Aug 30 00:46 nioneo_logical.log.v12 -rw-r--r-- 1 aseemk staff16B Aug 30 00:46 nioneo_logical.log.v13 -rw-r--r-- 1 aseemk staff12M Aug 30 00:46 nioneo_logical.log.v14 -rw-r--r-- 1 aseemk staff 1.4M Aug 30 00:46 nioneo_logical.log.v15 -rw-r--r-- 1 aseemk staff 6.8M Aug 30 00:46 nioneo_logical.log.v16 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v17 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v18 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v19 -rw-r--r-- 1 aseemk staff 1.3M Aug 30 00:46 nioneo_logical.log.v2 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v20 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v21 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v22 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v23 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v24 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46 nioneo_logical.log.v25 -rw-r--r-- 1 aseemk staff25M Aug 30 00:46
Re: [Neo4j] Node.js Bindings / Standalone connection
Hey, just FYI, Daniel Gasienica and I wrote a Node.js REST API library a while back, which we use extensively in our app: http://github.com/gasi/node-neo4j (The README has slightly old installation instructions -- you can just `npm install neo4j`.) Cheers, Aseem On Mon, Aug 29, 2011 at 3:52 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Yes. Yummy. Great. But feel free to poke around the semantics of streaming websockets, would be great to hear experiences there! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Aug 29, 2011 at 12:49 PM, Thomas Fritz fritz...@gmail.com wrote: Great to hear! So when you say binary protocol, you mean a binary interface besides the RESTful API? Yummy. Great! Regards Thomas --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf 2011/8/29 Peter Neubauer peter.neuba...@neotechnology.com: Thomas, as mentioned, the binary protocol is pushed to Neo4j 1.6, but it WILL come. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Aug 29, 2011 at 11:49 AM, Thomas Fritz fritz...@gmail.com wrote: Hi Is there an official node.js binding for Neo4J? Is it possible to access the Neo4J Standalone version directly through a socket or anything with less overhead than the REST API? If not, is there a way to write a Neo4J driver? Is the socket communication documented? Or is it just Java RMI? Do you plan to give users a possibility to access a standalone server other than REST? How do you advise us to use neo4j standalone in a fast way from any language where there is no binding? Kind regards. --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
Hey, FWIW, we ended up needing this for actual queries in our app, not just the console, so we ended up adding a UUID property to every node in our graph, just so we can efficiently query collections of node pairs (where nodes may be duplicated many times across pairs). We (auto-)index this UUID property so we can then do a second lookup for the nodes' data. If Cypher supported IDs or URLs natively, that would have made this a bit more convenient. =) Aseem On Wed, Aug 24, 2011 at 6:41 PM, Aseem Kishore aseem.kish...@gmail.comwrote: I was asking really only for the console. I don't see myself needing to use the ID programmatically -- except it might be a minor optimization to be able to get only node URLs if I already have node data, and I just want to compare the output of this query with the nodes I already have. For the console, the only thing is that nodes w/ many properties (and possibly long values, e.g. descriptions) totally litter the console, making output impossible to read or make sense of. Shortening the results to just IDs (along with selected properties) would be convenient in some cases. In many cases, just the properties aren't identifying enough. Hope this helps, and thanks for the consideration! Aseem On Wed, Aug 24, 2011 at 5:58 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Hi Aseem, For various reasons, we've (so far) decided against allowing the id to be something you can output. Could you tell me a bit more about why you would like to return just the id? It's not hard to do, but it would make future parts of Cypher harder to do. If there are valid use cases we've missed, we'll reconsider. Regards, Andrés On Wed, Aug 24, 2011 at 5:39 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, Is there any way I can have Cypher return/print node IDs? If I specify for it to return a node, it returns/prints all data about that node, which can be excessive. But the problem is, the node's ID isn't a property on that node. I've tried both node~ID and node~SELF, like rel~TYPE, but neither works. If this isn't possible today, consider it a feature request. =) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: how would you query first- and second-degree friends?
I've run into another scenario which I think is related to this feature request. Take the IMDB example: the graph has movie nodes, and each movie has one or more genres/categories. You could express this as a property on each node, or you could have formal genre nodes that movies are connected to. Either way, the scenario is this: we want to show an overview page that shows the most popular movies for each category. E.g. it shows the top 5 comedy movies, the top 5 action movies, drama, horror, and so on. We can't figure out how to do this in one Cypher query. We can ORDER BY and LIMIT, but that's for the whole set, where we want to do this per category. The closest we've come up with is to return all movie+category pairs -- both as just UUIDs to be efficient (see other thread where I've asked for first-class ID support) -- along with the stats for each movie, and then we sort on the client, then we load the movie data for the top 5 per category. This isn't a bad result, but it feels a bit hacky/manual. Would love if Cypher supported this kind of thing directly. It's almost like grouping results and being able to order/limit within groups. But it's also just another first-degree / second-degree scenario. Food for thought. =) Thanks for the consideration! Aseem On Thu, Aug 25, 2011 at 2:00 PM, Aseem Kishore aseem.kish...@gmail.comwrote: Thanks Andrés! Couple of random extensions to the thoughts above: - To be consistent, perhaps for optional matches, the results could always include a row where that optional match was ignored. That way, Elizabeth, null wouldn't be an inconsistency; *every* first-degree friend would have an [friend], null row. - And perhaps *that* row could be the one that has the full node data, and every subsequent row for the second optional match would just reference the URL/ID of the first-degree node. After chewing on this a lot more, though, I've realized that what I really want in this scenario is not tabular data, but a real subgraph. Hence my email just now. =) Aseem On Thu, Aug 25, 2011 at 1:34 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Good and valid requests, Aseem. Consider them heard. We'll see how soon we can get it to you. Andrés On Thu, Aug 25, 2011 at 12:57 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
I spoke too soon -- this doesn't seem to scale very far with many UUIDs being looked up together. Here's a query looking up 76 things that returns a 500 error -- due to a stack overflow exception. *START*node=(node_auto_index,'uuid_indexed:(f93d416a-40ae-4b5a-87d7-53b6bc4b54ab OR ceaad2cf-1afb-4cdd-b98a-8706dc4c471a OR 7bb157dd-ff76-49d3-9b01-0c506909400c OR 18b06833-b763-419c-b0ba-78e43dd0898d OR fdbaf139-053c-4886-8855-47efc4802f60 OR 8de5a2fb-5a42-4386-9b5a-8d5cbbb3a491 OR a9a54c63-6fb4-489e-bf55-ebca2ef8e2c3 OR 17c2c15e-f116-4092-afbe-336674f7b3ed OR 0c241168-8b41-4fc5-b86c-db7dfe4bfc3d OR 352de030-1414-4469-af34-46a968d44409 OR d48769af-0f7a-401d-8e8e-0383d687e87c OR 923bbc53-9012-435f-ac09-a71fd9b27b21 OR 879011a1-e26d-4fd1-ae78-c2d8d1f2ec7d OR 5f0100d5-7be7-48a0-bcd2-4ff706162b3a OR e5cd6233-24c6-4977-8c49-5d13cb386684 OR fb331814-5d8e-44c1-bbca-e9a38cd5b1b7 OR 3fe36ba3-feb2-4e9a-9a83-bd8c42c59960 OR 33f35e99-107f-43cc-9fd8-0e09ec28490a OR c10623a4-082b-455d-aa87-5467a97108d6 OR a2a5cbd6-4cce-4dcb-ba6c-8f4c3fda8349 OR 8a1e21d8-e407-4795-a841-ca891b53eb77 OR c3181928-9f7c-4b90-a658-c19c2513c42c OR ebb867cd-3e40-40ad-8dbd-8e7cd64ef057 OR 62ae80c2-2f61-4318-ae0c-d352c75295b6 OR 103b1113-6afa-48d6-a158-8988d3813589 OR 853fca76-dbfa-4e89-b0db-c059a153101d OR 7a514103-66b9-4dee-9da0-4a02eec1b516 OR f88ab591-6989-4908-8856-83efd8a79df1 OR f4b2aae8-ab7c-4e93-b0db-2dfc7230cc11 OR 074324dc-5194-46b7-a1bc-34f41b719bd0 OR 1d82f823-c76d-4f60-aeed-01f1896fa19f OR 273ea5d5-7ba3-4df1-af1f-e9e12490dc40 OR 6f06879e-a03e-4239-ab1c-6f2301831319 OR 96b77c3a-da52-424a-8a0d-90eaa4563b5f OR f413419c-2269-4dcf-b249-49b5828feadf OR 0e159ace-32e3-48bc-90f9-735528979726 OR 885e6466-d5be-45a6-907c-ff6945a7a147 OR ddbf1468-4714-4001-a695-36d84debfe0e OR df280b6c-f06b-4b06-9581-56434db22d02 OR ddaec953-5194-43c6-b2da-7e168c8c6189 OR f792a366-2654-4702-a095-f546d65005ec OR f45179f5-a4e0-4f0c-bdb0-30f37f7986c5 OR deb11e05-6920-461f-a198-b943caac97e8 OR bca6d42c-df7f-4ccb-bc7a-054223a1c456 OR 65925c96-19c0-47bf-8ae1-30d39918ba0a OR 3909bb89-e361-49fa-950e-d24dc0f4b598 OR d000239e-0e43-4fe2-884a-4baf3a088795 OR fc5481af-6537-4bd7-b717-83c609a50995 OR f897b591-ae71-47c8-8d7f-2088dfb64746 OR fd46e90f-da43-46be-a53f-1f6d301b039d OR 7bda73ad-a80b-4b47-933f-6bf0f3de47f4 OR a2a29161-cda2-4aec-85b8-53ef494a945e OR 8290618b-c6eb-4902-9068-616f0df84040 OR 75b56851-d53a-4069-8d90-cc9a94186579 OR d344990d-96da-43f4-8680-a61b38692841 OR 35898e0b-44e1-4023-855d-a56eb4e07d45 OR 2081ace8-94df-4841-94e7-08e4ea583f2c OR 18d77916-e240-4760-9b7f-9c9ad2687860 OR 5a0d07f9-6635-4544-9f89-7770e2746d37 OR edd3fdee-1545-48e2-8c3b-e8dccfc6d69a OR ff88eade-ffd1-4d98-9170-c43ebef04554 OR bc823ddd-3fd5-448c-926a-414b8742d2bc OR e5cd635e-33be-43bd-bb6f-e9cde367b7ef OR b2f7414d-a751-424a-9786-f6acbb1eae7a OR 50a7ef92-e19d-4bf9-a767-15bc8c9b3481 OR fd64a857-41ac-41bc-8967-5b123042a921 OR 9b0c1dad-e7da-4a94-8a06-41eed34c739c OR 953f6857-30a4-44a2-aa97-b23371a3d3af OR 62d983d9-8790-4105-8370-ac1e2ff76857 OR 4ac9c5de-09e7-4b43-8841-fd4df3ccc472 OR b0cfd10c-ae9c-4efc-96d9-0884c50fc835 OR 68195e2a-a387-4ea9-98a0-c45ba9ffd342 OR 0962c97e-d4e0-4e91-a917-63b3b9148095 OR 0993-3479-4785-b381-44259eeb5bd4 OR cab20da4-54d2-4212-ad2f-cd413246c557 OR 6730e381-000a-4753-bfd7-3482e8e27bd6)') *RETURN* node I guess we'll try a batch API request? Aseem On Fri, Aug 26, 2011 at 12:31 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Hey, FWIW, we ended up needing this for actual queries in our app, not just the console, so we ended up adding a UUID property to every node in our graph, just so we can efficiently query collections of node pairs (where nodes may be duplicated many times across pairs). We (auto-)index this UUID property so we can then do a second lookup for the nodes' data. If Cypher supported IDs or URLs natively, that would have made this a bit more convenient. =) Aseem On Wed, Aug 24, 2011 at 6:41 PM, Aseem Kishore aseem.kish...@gmail.comwrote: I was asking really only for the console. I don't see myself needing to use the ID programmatically -- except it might be a minor optimization to be able to get only node URLs if I already have node data, and I just want to compare the output of this query with the nodes I already have. For the console, the only thing is that nodes w/ many properties (and possibly long values, e.g. descriptions) totally litter the console, making output impossible to read or make sense of. Shortening the results to just IDs (along with selected properties) would be convenient in some cases. In many cases, just the properties aren't identifying enough. Hope this helps, and thanks for the consideration! Aseem On Wed, Aug 24, 2011 at 5:58 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Hi Aseem, For various reasons, we've (so far) decided against allowing the id to be something you can output. Could you tell me a bit more about why you would like to return just the id? It's not hard to do
Re: [Neo4j] Specifying best first order of traverse over REST API
Thanks Peter, we'll look into Gremlin. =) But I'll push back a bit and say it would still ask for a feature like this to be a first-class feature of the REST API. If my app is e.g. in Python, it's not super API-friendly for me to have to write Java-ish code to achieve what I want. I'd get it if what I was asking for was an uncommon edge case request. But for quite a while, this has been the one biggest pain point of Neo4j for us: how to return a meaningful subset of traverse results instead of all results. Having a paged traverser doesn't help at all if you can neither specify the order nor know what the default ordering is based on (I asked a while back and was told it's unspecified). (And again, by order here I mean beyond just breadth first.) I'm all for simple, minimalistic and elegant APIs. I'm also all for being scenario-driven, and solving problems rather than building arbitrary features. So perhaps one solution here would be to introduce a new built-in best first order that's just like Djikstra's shortest-path API: you just specify a cost property (or function) for nodes and/or relationships. This would definitely solve our problem, and, I suspect, most people's problems in this regard. It wouldn't be a bulky feature that doesn't fit in (it would compliment Djikstra's very nicely), and it wouldn't require writing a complex Gremlin script to achieve this (I would think) common requirement. I'll get off the soapbox now. ;) Thanks for your consideration! Aseem On Thu, Aug 25, 2011 at 9:01 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: See the other mail for a pointer :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 25, 2011 at 1:00 AM, Matt Luongo m...@scholr.ly wrote: +1, we could really use that. Client-side sorting sucks. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Aug 24, 2011 at 4:43 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I've just spent a bunch of time reading into how one can control the ordering of a traverse beyond simple breadth first or depth first. More precisely, even when breadth first, how one can control *which* neighbors are traversed first. (It matters less in which order they're traversed vs. which order they're returned if you're returning all results, since you can just sort on the client. But it matters a lot if you want to use the paged traverser, since you're then only returning the first results.) I've learned that this is doable from Java by writing your own BranchSelector implementation: http://components.neo4j.org/neo4j/1.4.1/apidocs/ I've found the built-in implementations, e.g. the pre-order breadth-first and depth-first: https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderBreadthFirstSelector.java https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderDepthFirstSelector.java To achieve a custom best first, Igor Dovgiy for example shared that he modeled his implementation of the breadth-first selector, except just using a PriorityQueue instead of a regular Queue. My question is: is there any way to specify this sort of thing over the REST API instead of having to write a plugin? If not, does that sound like a reasonable feature request? I really just want something simple: nodes ordered by some timestamp property. It's killing us that we can't do this today. We might just have to look into writing this as a plugin... Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Specifying best first order of traverse over REST API
Okay! Will think about this and get back to you soon. Peter, I gotta say -- your open-mindedness and willingness to help developers like me is awesome and much appreciated. Customer satisfaction, etc. =) Cheers, Aseem On Thu, Aug 25, 2011 at 11:12 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Aseem, If you can provide a nice suggestion on how exactly it would look, maybe I can implement it if the others agree? /peter Sent from my phone. On Aug 25, 2011 7:19 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Thanks Peter, we'll look into Gremlin. =) But I'll push back a bit and say it would still ask for a feature like this to be a first-class feature of the REST API. If my app is e.g. in Python, it's not super API-friendly for me to have to write Java-ish code to achieve what I want. I'd get it if what I was asking for was an uncommon edge case request. But for quite a while, this has been the one biggest pain point of Neo4j for us: how to return a meaningful subset of traverse results instead of all results. Having a paged traverser doesn't help at all if you can neither specify the order nor know what the default ordering is based on (I asked a while back and was told it's unspecified). (And again, by order here I mean beyond just breadth first.) I'm all for simple, minimalistic and elegant APIs. I'm also all for being scenario-driven, and solving problems rather than building arbitrary features. So perhaps one solution here would be to introduce a new built-in best first order that's just like Djikstra's shortest-path API: you just specify a cost property (or function) for nodes and/or relationships. This would definitely solve our problem, and, I suspect, most people's problems in this regard. It wouldn't be a bulky feature that doesn't fit in (it would compliment Djikstra's very nicely), and it wouldn't require writing a complex Gremlin script to achieve this (I would think) common requirement. I'll get off the soapbox now. ;) Thanks for your consideration! Aseem On Thu, Aug 25, 2011 at 9:01 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: See the other mail for a pointer :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 25, 2011 at 1:00 AM, Matt Luongo m...@scholr.ly wrote: +1, we could really use that. Client-side sorting sucks. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Aug 24, 2011 at 4:43 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I've just spent a bunch of time reading into how one can control the ordering of a traverse beyond simple breadth first or depth first. More precisely, even when breadth first, how one can control *which* neighbors are traversed first. (It matters less in which order they're traversed vs. which order they're returned if you're returning all results, since you can just sort on the client. But it matters a lot if you want to use the paged traverser, since you're then only returning the first results.) I've learned that this is doable from Java by writing your own BranchSelector implementation: http://components.neo4j.org/neo4j/1.4.1/apidocs/ I've found the built-in implementations, e.g. the pre-order breadth-first and depth-first: https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderBreadthFirstSelector.java https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderDepthFirstSelector.java To achieve a custom best first, Igor Dovgiy for example shared that he modeled his implementation of the breadth-first selector, except just using a PriorityQueue instead of a regular Queue. My question is: is there any way to specify this sort of thing over the REST API instead of having to write a plugin? If not, does that sound like a reasonable feature request? I really just want something simple: nodes ordered by some timestamp property. It's killing us that we can't do this today. We might just have to look into writing this as a plugin... Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Cypher: how would you query first- and second-degree friends?
Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Wishlist: subgraphs over REST API
Sorry if I'm inundating the list w/ too many emails. =D I've been loving Cypher -- way more user-friendly and powerful than the REST API's traverse method -- but I'm finding even Cypher isn't optimized for queries where I really want to fetch a *subgraph*, not tabular data. I can give plenty of scenarios if that helps (I've even drawn diagrams!), but the theme in my queries is repeatedly that I need to understand the relationships between nodes in our graph, which means I want subgraphs. I understand that the traverse method in the REST API gives you the option of returning nodes, relationships, or paths. I'd like to propose a new return type: subgraph (or graph). Here's what I'm envisioning: the JSON contains an adjacency matrix of nodes and relationships -- referenced entirely by URLs -- and an additional map from URL to data (properties) for each referenced node and relationship. Here's an example format: https://gist.github.com/1171908 I'd *love* to be able to query and fetch subgraphs from the graph like this. That would be so awesome. I'd love thoughts and feedback. Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: how would you query first- and second-degree friends?
Thanks Andrés! Couple of random extensions to the thoughts above: - To be consistent, perhaps for optional matches, the results could always include a row where that optional match was ignored. That way, Elizabeth, null wouldn't be an inconsistency; *every* first-degree friend would have an [friend], null row. - And perhaps *that* row could be the one that has the full node data, and every subsequent row for the second optional match would just reference the URL/ID of the first-degree node. After chewing on this a lot more, though, I've realized that what I really want in this scenario is not tabular data, but a real subgraph. Hence my email just now. =) Aseem On Thu, Aug 25, 2011 at 1:34 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Good and valid requests, Aseem. Consider them heard. We'll see how soon we can get it to you. Andrés On Thu, Aug 25, 2011 at 12:57 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Man, I am loving Cypher. Thanks so much guys for introducing it. I'm a bit stuck on one query, though, and I wanted to ask for help. I think the reasons I'm stuck are related to the two feature requests I made yesterday (optional matches, and returning IDs). I want to fetch first- and second-degree friends in one query, in a way that preserves the first-degree friend(s) for each second-degree friends. Assume an asymmetrical friendship model, like Twitter's following, because that makes things a bit easier. Here's a simple transitive example: - Alice follows Bob and Carol. - Bob follows Carol and Dave. - Carol follows Dave and Alice. I want to fetch Alice's first-degree friends (Bob and Carol) and each of their first-degree friends ((Carol and Dave) for Bob, and (Dave and Alice) for Carol). This example is easy enough: START zero=(Alice) MATCH (zero) -[:FOLLOWS]- (first) -[:FOLLOWS]- (second) RETURN first, second I expect to get back results like: - Bob, Carol - Bob, Dave - Carol, Dave - Carol, Alice This is great because I can get Alice's first-degree friends by unique()'ing the first column, for each person, I know their second-degree friends via the second column. But this doesn't work if any of my first-degree friends don't follow anyone. For that, it would be great if I could specify that the second part of that match was optional, and I'd get back a row like this in that case: - Elizabeth, null If I'm returning nodes, it also duplicates a ton of info: each first-degree friend's info is returned in full for each of their friends. Maybe there's no way around that, but this is also where I would find it convenient to be able to return IDs somehow for all but one of the results. Just thinking out loud here, but I'd greatly appreciate any feedback or ideas for this scenario. Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Query/design question: trusted followers scenario
Hi guys, We're building a social network which has an asymmetrical follower model like Twitter's: users follow each other. We have various views where we show a list of people. This could be e.g. all people in the network, or it might be some user's followers, or it might be a list of people that share interests, etc. In these views, it's easy to show how many followers each person has. But we also want to show a message like Followed by 4 people you follow next to each person. This helps show the trustworthiness/relevance of each person. We implemented that by logic like this: 1. Fetch the list of people that *I* follow. 2. Given the list of people we want to show, for each person in parallel... 3. ...Fetch the list of people that follow *that* person... 4. ...And compare this list with the list of people that I follow. Each fetch is a traverse (breadth first, max depth 1). This requires O(n) traverses, where n is the number of people we're showing in this view. (Assume that, generally, the number of people we're showing is smaller than the number of people I potentially follow, but the logic could be reversed if this is not the case: for each person I follow, fetch the list of people that *they* follow.) I wanted to do a sanity check: is this the best way of answering this question? Or is there a better way, e.g. via a single traverse somehow, or via a Cypher or Gremlin query? Thanks much! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: node IDs
I was asking really only for the console. I don't see myself needing to use the ID programmatically -- except it might be a minor optimization to be able to get only node URLs if I already have node data, and I just want to compare the output of this query with the nodes I already have. For the console, the only thing is that nodes w/ many properties (and possibly long values, e.g. descriptions) totally litter the console, making output impossible to read or make sense of. Shortening the results to just IDs (along with selected properties) would be convenient in some cases. In many cases, just the properties aren't identifying enough. Hope this helps, and thanks for the consideration! Aseem On Wed, Aug 24, 2011 at 5:58 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Hi Aseem, For various reasons, we've (so far) decided against allowing the id to be something you can output. Could you tell me a bit more about why you would like to return just the id? It's not hard to do, but it would make future parts of Cypher harder to do. If there are valid use cases we've missed, we'll reconsider. Regards, Andrés On Wed, Aug 24, 2011 at 5:39 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hey guys, Is there any way I can have Cypher return/print node IDs? If I specify for it to return a node, it returns/prints all data about that node, which can be excessive. But the problem is, the node's ID isn't a property on that node. I've tried both node~ID and node~SELF, like rel~TYPE, but neither works. If this isn't possible today, consider it a feature request. =) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: multiple match clauses, but OR, not AND
Yes, this allows specifying multiple relationship types -- as long as the direction is the same. Thanks! It would still be extremely useful to be able to specify this when the directions also differ. (Or in some cases, the shapes of the path also differ.) Cheers, Aseem On Wed, Aug 24, 2011 at 6:04 PM, Andres Taylor andres.tay...@neotechnology.com wrote: Forgot to mention it: We've changed ~TYPE to .TYPE. It's in out snapshot version. Just a heads up. Andrés On Wed, Aug 24, 2011 at 6:02 PM, Andres Taylor andres.tay...@neotechnology.com wrote: On Wed, Aug 24, 2011 at 5:44 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Here's my use case: I want to show my friends' comments and likes. For comments, I can do e.g. (friend) -[:WROTE]- (comment), and for likes, I can do e.g. (friend) -[:LIKES]- (object), but is there a way I can combine both these matches into one Cypher query? I know I can put multiple match clauses, but it's my understanding that that's an AND, not an OR. For this particular case, since the length/shape of the path is the same for both cases, being able to specify multiple choices for relationship type would work here, e.g. (friend) -[:WROTE,:LIKES]- (object), where the returned object could potentially be a comment. What you can do if you want to OR relationship types is this: MATCH friend -[r]- object WHERE r~TYPE = WROTE or r~TYPE = LIKES I've been considering adding something like what you suggest. It's a good idea - it's mostly a matter of picking the most important features first right not. Thanks for your feedback - much appreciated! Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Delete followed by traverse: possible race condition?
I'm deleting a node and then immediately doing a traverse, and the traverse sometimes returns the deleted node. Am I clearly at fault here? Or could this indeed be coming from the Neo4j server? (E.g. a caching issue?) This happens consistently, but not always, maybe around 50%-75% of the time. I'm using the REST API. The node is always truly deleted from the graph whenever I manually check after this happens, and if I re-traverse, the node is gone as it should be, so this suggests it's a caching or timing issue. Any ideas? I'll keep debugging to make sure it's not something I'm doing wrong... Thanks, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Delete followed by traverse: possible race condition?
Related, I had noticed that immediately after adding a node (and connecting it), the same traverse doesn't usually return that node, but the next traverse always does. This feels like the same issue -- a caching or timing one -- just one that never caused an error in our app. The deleting does cause an error, because attempting to then do a second traverse from that deleted node invariably results in an error since the node has been deleted. Aseem On Sun, Aug 14, 2011 at 2:37 AM, Aseem Kishore aseem.kish...@gmail.comwrote: I'm deleting a node and then immediately doing a traverse, and the traverse sometimes returns the deleted node. Am I clearly at fault here? Or could this indeed be coming from the Neo4j server? (E.g. a caching issue?) This happens consistently, but not always, maybe around 50%-75% of the time. I'm using the REST API. The node is always truly deleted from the graph whenever I manually check after this happens, and if I re-traverse, the node is gone as it should be, so this suggests it's a caching or timing issue. Any ideas? I'll keep debugging to make sure it's not something I'm doing wrong... Thanks, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Paged traversal REST API suggestion for improvement
Jim, thanks for the explanation. I understand your constraints, but thinking about it more, I'm actually even more baffled -- how can we actually make use of this paged traversal API in a web scenario? [neo4j db] [web server] [web client, e.g. browser] If I want to show the results of a search, let's say, in a way that supports paging, it just seems like I can't take advantage of the Neo4j API at all here. If the user wants to be able to refresh the page, or click Prev, the browser isn't keeping state across page changes, so it has to rely on the web server to give it the right page. If the server now has to keep state of all results, it defeats the purpose of using the Neo4j paging API altogether: the server necessarily has to fetch everything and handle paging itself. Am I missing something? Or is this (typical in our case) scenario not something you guys are designing for? (And if not, what *is* the use case you guys envisioned for this?) It seems to me that the root cause of this dilemma is the IterableT constraint you mentioned -- it only goes forward. Perhaps a better API/model then would be to use an offset parameter. This is different than page because it doesn't require the server to keep state; it just tells the server, return me the results starting *after* this result (which would generally be the last result I saw on my previous page of results). E.g. the following requests: - Get me the results of this traverse would return everything - Get me the results of this traverse, paged with size 20 would return 20 results - Get me the results of this traverse, paged with size 20, starting with the node/rel after this one [given by an ID] would return the next 20 results - etc. Aseem On Thu, Jul 28, 2011 at 5:09 AM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, When you GET a resource, you're asking for its state at a particular instant in time. GET's don't always return the same content, what's important about them is that clients can't be held responsible for any (unintended) side-effects of a GET operation. For example, if you GET the BBC new page now, and then GET it again in 20 minutes, the bytes you receive will almost certainly be different. I think what's perhaps a bit odd in the paged traverser case is that clients do in fact trigger a state transition on GET which materially changes the response. Adding /next to the paged traverser URI does not change this, it just lengthens the URI. Under the covers, the REST API is just using the traversal API and so gets back an IterableT which is forward-only. We can't (or rather won't) do things like ?page=2 (or similar) because that would involve holding all the traversal state on the server which is one of the things paged traversers were meant to avoid (being kind to both the server and clients). If you want to remember all traversal state on the client, that's OK. You can page through your own local data structures. In fact given the number of clients is likely to be larger than the number of servers, this makes sense since the memory burden is shared amongst the many clients, rather than being placed on the single server. So, I'll buy that we've conflated a little too much into a single GET, but it's motivated by a sincere trade off to not overburden the server. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Paged traversal REST API suggestion for improvement
Sorry, I don't understand. Are you going to tell Google and Bing to send JSON for their search results also, and page on the client side? There is a very valid use case for not wanting to require JavaScript to page. Aseem On Thu, Jul 28, 2011 at 3:25 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: Not really, unless the JSON content is HUGE - which is rare. The small price in bandwidth versus a responsive UI and the need for not managing state is a good tradeoff. The only exception might be mobile apps, but in that case, I'd just fetch a smaller block (maybe 4 or 5 pages worth). Very few users have the patience to page through more than a few pages worth of information. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Daniel Gasienica Sent: Thursday, July 28, 2011 6:20 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Sure but isn’t it a huge waste of bandwidth if you load hundreds of results and the user only looks at the first dozen? On Thu, Jul 28, 2011 at 15:16, Rick Bullotta rick.bullo...@thingworx.com wrote: BTW, paging is a relic of the dial-up modem days, IMNSHO. If a machine is the client of the REST API call, it should get all the data in a single, atomic call. If it is a browser or mobile app, there's a natural limit to the # of items that a human will bother paging through, which is fairly small (20 - 500 typically), so again, it is painless to get all the data in a single call. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Thursday, July 28, 2011 6:11 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement If you don't keep state paging will not work properly if the data changes often. What may have been record #21 when you are viewing the first page of 20 result might not be record #21 when you go to fetch the 2nd page. If you're only concerned with pages of 20 or so, you should absolutely, positively do the paging on the browser. Anything up to a couple thousand rows can *easily* be handled by any modern browser. Keep the parsed JSON data on the client side and page it there. This approach will be very performant on both the client and server, and won't require any state. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, July 28, 2011 3:01 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Jim, thanks for the explanation. I understand your constraints, but thinking about it more, I'm actually even more baffled -- how can we actually make use of this paged traversal API in a web scenario? [neo4j db] [web server] [web client, e.g. browser] If I want to show the results of a search, let's say, in a way that supports paging, it just seems like I can't take advantage of the Neo4j API at all here. If the user wants to be able to refresh the page, or click Prev, the browser isn't keeping state across page changes, so it has to rely on the web server to give it the right page. If the server now has to keep state of all results, it defeats the purpose of using the Neo4j paging API altogether: the server necessarily has to fetch everything and handle paging itself. Am I missing something? Or is this (typical in our case) scenario not something you guys are designing for? (And if not, what *is* the use case you guys envisioned for this?) It seems to me that the root cause of this dilemma is the IterableT constraint you mentioned -- it only goes forward. Perhaps a better API/model then would be to use an offset parameter. This is different than page because it doesn't require the server to keep state; it just tells the server, return me the results starting *after* this result (which would generally be the last result I saw on my previous page of results). E.g. the following requests: - Get me the results of this traverse would return everything - Get me the results of this traverse, paged with size 20 would return 20 results - Get me the results of this traverse, paged with size 20, starting with the node/rel after this one [given by an ID] would return the next 20 results - etc. Aseem On Thu, Jul 28, 2011 at 5:09 AM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, When you GET a resource, you're asking for its state at a particular instant in time. GET's don't always return the same content, what's important about them is that clients can't be held responsible for any (unintended) side-effects of a GET operation. For example, if you GET the BBC new page now, and then GET
Re: [Neo4j] Paged traversal REST API suggestion for improvement
Haha. I think we're speaking past each other. Here's my point: Disable JS. Open up Google. Do a search. Click to the next page. Click to back to the previous page. Click page 3. Refresh. Etc. All of this works, without JS. That means the client (your browser) isn't the party doing the paging or keeping track of all results. It's something on the server side. My original question was: I don't see how one can achieve this using Neo4j's paged traversal API. Is that not a reasonable scenario / use case for it? If not, what is? My follow-up suggestion was: could we not support this scenario / use case by changing (or expanding) the API to support an offset parameter? In other words, random access -- like keying into a graph by index and then following edges -- instead of forward-only and once-only IterableT. Aseem On Thu, Jul 28, 2011 at 4:27 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: Um, I'm guessing that you aren't aware of how Google's UI/API works... Open up Firebug, Chrome Tools, or Fiddler, and you'll see that the biggest chunk of traffic when you switch from one page of Google search results to another page (via the main Google search page) is a JSON packet...which uses their search API...which returns JSON. Do they page? Yes. Can you call the API and get hundreds of results at once? Yes... -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, July 28, 2011 6:42 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Sorry, I don't understand. Are you going to tell Google and Bing to send JSON for their search results also, and page on the client side? There is a very valid use case for not wanting to require JavaScript to page. Aseem On Thu, Jul 28, 2011 at 3:25 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: Not really, unless the JSON content is HUGE - which is rare. The small price in bandwidth versus a responsive UI and the need for not managing state is a good tradeoff. The only exception might be mobile apps, but in that case, I'd just fetch a smaller block (maybe 4 or 5 pages worth). Very few users have the patience to page through more than a few pages worth of information. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Daniel Gasienica Sent: Thursday, July 28, 2011 6:20 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Sure but isn't it a huge waste of bandwidth if you load hundreds of results and the user only looks at the first dozen? On Thu, Jul 28, 2011 at 15:16, Rick Bullotta rick.bullo...@thingworx.com wrote: BTW, paging is a relic of the dial-up modem days, IMNSHO. If a machine is the client of the REST API call, it should get all the data in a single, atomic call. If it is a browser or mobile app, there's a natural limit to the # of items that a human will bother paging through, which is fairly small (20 - 500 typically), so again, it is painless to get all the data in a single call. -Original Message- From: user-boun...@lists.neo4j.org [mailto: user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Thursday, July 28, 2011 6:11 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement If you don't keep state paging will not work properly if the data changes often. What may have been record #21 when you are viewing the first page of 20 result might not be record #21 when you go to fetch the 2nd page. If you're only concerned with pages of 20 or so, you should absolutely, positively do the paging on the browser. Anything up to a couple thousand rows can *easily* be handled by any modern browser. Keep the parsed JSON data on the client side and page it there. This approach will be very performant on both the client and server, and won't require any state. -Original Message- From: user-boun...@lists.neo4j.org [mailto: user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, July 28, 2011 3:01 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Jim, thanks for the explanation. I understand your constraints, but thinking about it more, I'm actually even more baffled -- how can we actually make use of this paged traversal API in a web scenario? [neo4j db] [web server] [web client, e.g. browser] If I want to show the results of a search, let's say, in a way that supports paging, it just seems like I can't take advantage of the Neo4j API at all here. If the user wants to be able to refresh the page, or click Prev, the browser isn't keeping
[Neo4j] Paged traversal REST API suggestion for improvement
Hey there, Neo4j's REST API is overall quite awesome and impressive -- it's just such a great *actual* REST API. I was just reading into paged traversals ( http://docs.neo4j.org/chunked/stable/rest-api-traverse.html ) and couldn't help but feel that this isn't the best way to do paging over REST. I dig that you POST to create a paged traverser, which returns a 201 with a Location set to the traverser -- the traverser is indeed a real resource on the server. But, it caught me off guard that you page by repeatedly GETting the same resource over and over. In other words, the resource is changing each time you GET it. Doesn't that violate GET, that state on the server shouldn't be visibly changed, or that GETting the same resource multiple times should return that same resource each time? I've seen other paging APIs return a next token in each response, and you include that token in your next request, either as a header or in the query string. I've also seen the simple ?page=2 or /page/2 approach. This is obviously not a dealbreaker, but it is a limitation -- I have to remember the results on my end and can't refetch them if e.g. the user refreshes the page in their browser. That kinda sucks. What are your guys' thoughts? Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Index returns all nodes for value=undefined!
Thanks guys. This must have been a bug on our end then, and it must have been transient, because the returned nodes were old ones. We added an explicit not {null, undefined or empty string} check a while back, so that must have saved us. Aseem On Sun, Jul 17, 2011 at 1:00 PM, Michael Hunger michael.hun...@neotechnology.com wrote: Perhaps it is also an issue of your REST-client? If the value when rendered as part of a string evaluates to the empty string the resulting URL would be the same as the one that retrieves all nodes. i.e. http://localhost:7474/db/data/index/node/nodes/username Just an idea, could you verify the REST url that is used? Am 17.07.2011 um 17:06 schrieb Mattias Persson: No, there's no such undefined constant which will return everything. It definitely sounds like a JavaScript issue, and that it turns nulls into undefined. And perhaps your code which tries to index null values. 2011/7/16 Aseem Kishore aseem.kish...@gmail.com We're seeing this crazy behavior where our exact index is returning all nodes if we pass a lookup value of undefined! E.g.: http://localhost:7474/db/data/index/node/nodes/username/undefined Is this a bug in Neo4j / REST API? Or is this a symptom of some bug we have in our own code? It could very well be our code, since we're using JS, which turns undefined values into the string undefined if you try to add it to a string (definitely a wtfjs candidate), but I wanted to see if others are seeing this in Neo4j too! Thanks, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Auto-indexing feature request: transform
Thanks Chris, that makes sense. FYI, we still *really* wanted to use auto-indexing since it's a really comforting concept (we can't mess up), so we supported these transforms on our end by adding shadow properties in our database that have the transform applied to them, and it's these shadow properties that we index. E.g. if we set an email property on a node of he...@world.com, we'll set an additional email_indexed property of he...@world.com (lowercased), and it's the email_indexed property that we auto-index. Aseem On Mon, Jul 18, 2011 at 3:54 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Aseem, This is a use case that auto indexing may not be the best fit. The purpose of auto indexing is taking care of the common use case in the simplest possible manner, which is a property added is a property indexed. Indexing a property with a different value that is present on the primitive (that is what I understood from your description, correct if wrong) is a case that is best suited for custom indexing operations. Additionally, it is a desirable quality to have an index which follows the contents of the actual database with guaranteed consistency. I would be interested to know if there are others that do similar transformations and in what manner - maybe there is room for providing a customizing API. cheers, CG On Sat, Jul 16, 2011 at 9:48 AM, Aseem Kishore aseem.kish...@gmail.com wrote: We transform some of our properties currently before (manually) indexing them. Simple examples are lowercase (fulltext indexes supports a lowercase config that defaults to true, but exact ones don't, I was disappointed to find), but more complex ones are e.g. stripping special characters. It would be great if the auto-indexer could support this notion of transforming property values (and queries) through user-defined function, just like traversal accepts a user-defined filter function. (And, fwiw, we're users of the REST API, so ideally this functionality would be exposed over REST too.) Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] auto indexes in REST in 1.4?
Mike, we just did successfully try out the auto-indexing feature, and we're able to use it over the REST API just fine, FYI. Good luck! Aseem On Mon, Jul 18, 2011 at 6:37 AM, mike_t t.pr...@gmx.net wrote: Has anyone tried out the auto indexing feature? Did it works? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-auto-indexes-in-REST-in-1-4-tp3159205p3178756.html Sent from the Neo4J Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Auto-index fulltext?
Got it, thanks. =) On Mon, Jul 18, 2011 at 7:04 AM, Chris Gioran chris.gio...@neotechnology.com wrote: On Mon, Jul 18, 2011 at 1:50 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Okay, we look forward to the multiple auto-indexes feature then! There will still be one auto index for each primitive category. Can you explain what a primitive category is then I apologize for the confusion. Primitives (or primitive categories) in Neo4j are Nodes and Relationships. Named so because their in-kernel classes, NodeImpl and RelationshipImpl extend org.neo4j.kernel.impl.core.Primitive Aseem On Mon, Jul 18, 2011 at 6:15 AM, Chris Gioran chris.gio...@neotechnology.com wrote: No, that is not what i meant. The main idea is to provide the means to configure at least some aspects of the auto index instead of relying on the default settings only. There will still be one auto index for each primitive category. However, one feature under consideration is to actually allow an arbitrary number of auto indexes, each of which will allow for individual configuration. So, when that comes along you will have what you described (and more, actually). thanks, CG On Mon, Jul 18, 2011 at 12:29 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Awesome to hear, Chris, thanks. Just to clarify/confirm then: in the future, we will be able to have *both* an exact auto-index and a fulltext auto-index side-by-side? Cheers, Aseem On Mon, Jul 18, 2011 at 3:18 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Aseem, On Sat, Jul 16, 2011 at 9:46 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Is the 1.4 auto-index only exact? Or can it be configured to be a fulltext index? Yes, currently the auto-indexes are only exact, there is no straightforward way to configure them explicitly. This is a known shortcoming and will be remedied pretty soon. Another addition that is coming, by the way, is the visibility of changes of the auto index within the transaction, instead of waiting for commit time as is now. (Btw, it would be awesome if we could have two auto-indexes: one exact, one full-text. It would be great in general if all indexing could be auto. Not sure when you would ever want/need manual indexing.) Well, a lot of use cases call for manual indexing, when for example it is conditional or the use of multiple indexes is required. If, however, auto indexing covers all your needs then by all means, do just that. cheers, CG Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Auto-index fulltext?
[...] node type. It could provide an unambiguous link between a node and its indexing strategy(ies). Hey I'll just toss out that for us, we need multiple indexing strategies for the same node! E.g. if you have a movie, you may want to exact index an alias or username (a short but human-readable ID for popular movies, like lotr2 or hp5) while at the same-time fulltext index a title and description to facilitate search. Aseem On Mon, Jul 18, 2011 at 8:15 AM, Rick Bullotta rick.bullo...@thingworx.comwrote: Chris, I think that auto indexing is another great reason for a formal concept of node type. It could provide an unambiguous link between a node and its indexing strategy(ies). - Reply message - From: Chris Gioran chris.gio...@neotechnology.com Date: Mon, Jul 18, 2011 6:16 am Subject: [Neo4j] Auto-index fulltext? To: Neo4j user discussions user@lists.neo4j.org No, that is not what i meant. The main idea is to provide the means to configure at least some aspects of the auto index instead of relying on the default settings only. There will still be one auto index for each primitive category. However, one feature under consideration is to actually allow an arbitrary number of auto indexes, each of which will allow for individual configuration. So, when that comes along you will have what you described (and more, actually). thanks, CG On Mon, Jul 18, 2011 at 12:29 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Awesome to hear, Chris, thanks. Just to clarify/confirm then: in the future, we will be able to have *both* an exact auto-index and a fulltext auto-index side-by-side? Cheers, Aseem On Mon, Jul 18, 2011 at 3:18 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Aseem, On Sat, Jul 16, 2011 at 9:46 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Is the 1.4 auto-index only exact? Or can it be configured to be a fulltext index? Yes, currently the auto-indexes are only exact, there is no straightforward way to configure them explicitly. This is a known shortcoming and will be remedied pretty soon. Another addition that is coming, by the way, is the visibility of changes of the auto index within the transaction, instead of waiting for commit time as is now. (Btw, it would be awesome if we could have two auto-indexes: one exact, one full-text. It would be great in general if all indexing could be auto. Not sure when you would ever want/need manual indexing.) Well, a lot of use cases call for manual indexing, when for example it is conditional or the use of multiple indexes is required. If, however, auto indexing covers all your needs then by all means, do just that. cheers, CG Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Reification of RelationshipType
This is a bit above my head, as we're users of the REST API rather than working with the core server in Java directly, but I just want to ask: will this pollute the database's pool of node IDs? Not the end of the world if it will, but it would be nice if it didn't. =) Maybe I just don't understand the use case or am not affected by it. I'm also not sure whether we need or use RelationshipType (we do of course specify relationship types when traversing, etc.). Aseem On Mon, Jul 18, 2011 at 12:53 PM, Niels Hoogeveen pd_aficion...@hotmail.com wrote: Thanks for the cheer. Would someone currently working on core be able to look at the feasibility of this request? Niels Date: Mon, 18 Jul 2011 18:30:23 +0200 From: neubauer.pe...@gmail.com To: user@lists.neo4j.org Subject: Re: [Neo4j] Reification of RelationshipType Nils, Sounds great, I am on parental leave all summer so I can only cheer :) /peter Sent from my phone. On Jul 18, 2011 5:22 PM, Niels Hoogeveen pd_aficion...@hotmail.com wrote: In fact, this feature request wouldn't have to be more than a setNodeId(long id) and getNodeId() method on RelationshipType. This would eliminate the need to look up additional relationship type information through the Lucene Index. Niels From: pd_aficion...@hotmail.com To: user@lists.neo4j.org Date: Mon, 18 Jul 2011 15:10:25 +0200 Subject: [Neo4j] Reification of RelationshipType In another thread today, I defended the schemaless nature of Neo4J, however I believe there is one area where Neo4J has some sense of schema, which could use some improvement in my opinion, the RelationshipType. Whenever creating a typing layer on top of Neo4J, some duplication of functionality will be introduced with regard to RelationshipTypes. There is the RelationshipType in core, which is basically a name, and there will be a node in the typing layer, which will represent that same RelationshipType as a node. Linking these two RelationshipTypes needs to be done by means of the Lucene index, which is not necessarily the fastest operation. Would it be possible to reify the RelationshipType as a node in core, so relationshiptypes can be traversed. Different typing layers can then use that node to add additional information to the relationship (for example stating the transitivity, symmetry, reflexivity of the relationship, or whether the relationship is functional, injective or one-to-one, or the domain and range of the relationship). Different typing layers may implement the relationshiptype differently, but having a node to attach this information to would suffice for all possible scenarios. The creation of a RelationshipType-node, can even be done lazy, so Neo4J users that don't need a RelationshipType-node won't be bothered with additional nodes in the database. Niels ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Auto-index fulltext?
Is the 1.4 auto-index only exact? Or can it be configured to be a fulltext index? (Btw, it would be awesome if we could have two auto-indexes: one exact, one full-text. It would be great in general if all indexing could be auto. Not sure when you would ever want/need manual indexing.) Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Auto-indexing feature request: transform
We transform some of our properties currently before (manually) indexing them. Simple examples are lowercase (fulltext indexes supports a lowercase config that defaults to true, but exact ones don't, I was disappointed to find), but more complex ones are e.g. stripping special characters. It would be great if the auto-indexer could support this notion of transforming property values (and queries) through user-defined function, just like traversal accepts a user-defined filter function. (And, fwiw, we're users of the REST API, so ideally this functionality would be exposed over REST too.) Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Index returns all nodes for value=undefined!
We're seeing this crazy behavior where our exact index is returning all nodes if we pass a lookup value of undefined! E.g.: http://localhost:7474/db/data/index/node/nodes/username/undefined Is this a bug in Neo4j / REST API? Or is this a symptom of some bug we have in our own code? It could very well be our code, since we're using JS, which turns undefined values into the string undefined if you try to add it to a string (definitely a wtfjs candidate), but I wanted to see if others are seeing this in Neo4j too! Thanks, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API paging: specify sorting? and for index results?
Thanks Michael. On Thu, Jul 14, 2011 at 2:21 AM, Michael Hunger michael.hun...@neotechnology.com wrote: They will in 1.5 and using lucene query syntax you can probably sort them. Michael mobile mail please excuse brevity and typos Am 14.07.2011 um 07:49 schrieb Aseem Kishore aseem.kish...@gmail.com: Okay, thanks for explaining. Would still love an answer to whether index results will support paging or not. =) Thanks! Aseem On Thu, Jul 14, 2011 at 1:23 AM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, I think in that case it's non-deterministic - which means you page until you have enough results. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Constraints on index key names?
Thanks all! Re: lucene syntax - is there no way to escape the plus in queries? Aseem On Wed, Jul 13, 2011 at 4:56 PM, Mattias Persson matt...@neotechnology.comwrote: Jacob, you're talking about index names... I think the initial question was regarding keys in key/value pairs, for which neo4j adds no restrictions. 2011/7/13 Jacob Hansson ja...@voltvoodoo.com I'm not sure if there is anything keeping you from using them, but the names 'node_auto_index' and 'relationship_auto_index' are used by the auto indexing subsystem. On Wed, Jul 13, 2011 at 8:35 AM, Mattias Persson matt...@neotechnology.comwrote: Yes, the neo4j integration doesn't impose any additional restrictions other than those (if any) of Lucene. 2011/7/12 Rick Bullotta rick.bullo...@thingworx.com That would conflict with lucene's query syntax, I'd think. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore [aseem.kish...@gmail.com] Sent: Monday, July 11, 2011 7:41 PM To: Neo4j user discussions Subject: [Neo4j] Constraints on index key names? Is there any constraint on what an index key is allowed to be named? E.g. can it have a '+' in its name? Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] How to index/lookup from an array property?
I just learned that node/relationship property values can be arrays of primitives, not just primitives. That's pretty cool, but I'm wondering, how does this work (if it does work) with indexing and lookups? I'm referring to the REST API. E.g. if the URL is: GET /index/node/my_nodes/the_key/the_value What should I be putting in for the_value? I don't think it should be URL-encoded JSON, because for simple strings, quotes don't go into the URL. So is this even possible? Help appreciated! Thanks much. Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Constraints on index key names?
Awesome. Thank you. For this particular property, we're not using Lucene syntax since this is an exact index, and fortunately, having the + (URL-encoded as %2B) in the REST API index lookup URL works just fine. Aseem On Wed, Jul 13, 2011 at 5:18 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: Yes, there is. Just escape it with the \ character. See: http://ifdefined.com/btnet/lucene_syntax.html#+ -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Wednesday, July 13, 2011 5:02 PM To: Neo4j user discussions Subject: Re: [Neo4j] Constraints on index key names? Thanks all! Re: lucene syntax - is there no way to escape the plus in queries? Aseem On Wed, Jul 13, 2011 at 4:56 PM, Mattias Persson matt...@neotechnology.comwrote: Jacob, you're talking about index names... I think the initial question was regarding keys in key/value pairs, for which neo4j adds no restrictions. 2011/7/13 Jacob Hansson ja...@voltvoodoo.com I'm not sure if there is anything keeping you from using them, but the names 'node_auto_index' and 'relationship_auto_index' are used by the auto indexing subsystem. On Wed, Jul 13, 2011 at 8:35 AM, Mattias Persson matt...@neotechnology.comwrote: Yes, the neo4j integration doesn't impose any additional restrictions other than those (if any) of Lucene. 2011/7/12 Rick Bullotta rick.bullo...@thingworx.com That would conflict with lucene's query syntax, I'd think. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore [aseem.kish...@gmail.com] Sent: Monday, July 11, 2011 7:41 PM To: Neo4j user discussions Subject: [Neo4j] Constraints on index key names? Is there any constraint on what an index key is allowed to be named? E.g. can it have a '+' in its name? Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] What do people do for complex properties?
Node/relationship properties are simple key/value pairs, where the values are simple primitive values (exception: array of a single type of primitive value), so I was wondering what people do to store more complex properties. One solution is of course to store a JSON string. We'd like to avoid that monolithic route though as it's a hack that'll make maintenance difficult. E.g. we can't just go into the web admin and tweak a property as needed. We currently have a property schema that looks something like this, for one of our nodes: title: description: image: url: width: # height: # We've mapped that to this schema in Neo4j: title: description: image.url: image.width: # image.height: # You can see we've flattened the object to derive the flat list of keys and values. This works okay, but it makes discovery of image properties difficult. E.g. with a regular JS object, you could just do this to see if this node had an image: if (node.image) { ... } But the equivalent in Neo4j won't work (pseudocode): if node has property image ... So you have to check for specific subproperties. Not a huge annoyance for simple structures, but for deep structures that could get pretty silly, so you would then have to iterate over all properties and see if any begin with image. for example. Anyway, just wanted to see if any of you guys have come up with any better / more creative solutions for this problem. =D Would love to hear if so! Thanks. Cheers, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to index/lookup from an array property?
Oh man, having the index automatically handle arrays and index each value separately sounds killer. Any pointers on how to do this via the REST API? Still not clear to me what format URL I should be using. Thanks! Aseem On Wed, Jul 13, 2011 at 5:50 PM, Mattias Persson matt...@neotechnology.comwrote: 2011/7/13 Michael Hunger michael.hun...@neotechnology.com Indexed values are actually not bound to be properties. The index framework just calls toString() on indexed values that are not ValueContexts. So you should find a suitable representation of your arrays (e.g. a comma separated list or a json array string) if you want to lookup them. And then index that representation. Although if your object is some kind of array it will index all array items separately, so that each of them is query:able individually. Cheers Michael Am 13.07.2011 um 23:39 schrieb Aseem Kishore: I just learned that node/relationship property values can be arrays of primitives, not just primitives. That's pretty cool, but I'm wondering, how does this work (if it does work) with indexing and lookups? I'm referring to the REST API. E.g. if the URL is: GET /index/node/my_nodes/the_key/the_value What should I be putting in for the_value? I don't think it should be URL-encoded JSON, because for simple strings, quotes don't go into the URL. So is this even possible? Help appreciated! Thanks much. Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to index/lookup from an array property?
Okay. Thanks! Would love to see this improved. Aseem On Wed, Jul 13, 2011 at 6:08 PM, Mattias Persson matt...@neotechnology.comwrote: http://help.neo4j.org/discussions/questions/7-re-neo4j-how-to-indexlookup-from-an-array-property 2011/7/14 Mattias Persson matt...@neotechnology.com I think that indexing values other than strings are impossible through the REST API and that's a known shortcoming I hope will be fixed pretty soon. Creating a support discussion for this. 2011/7/13 Aseem Kishore aseem.kish...@gmail.com Oh man, having the index automatically handle arrays and index each value separately sounds killer. Any pointers on how to do this via the REST API? Still not clear to me what format URL I should be using. Thanks! Aseem On Wed, Jul 13, 2011 at 5:50 PM, Mattias Persson matt...@neotechnology.comwrote: 2011/7/13 Michael Hunger michael.hun...@neotechnology.com Indexed values are actually not bound to be properties. The index framework just calls toString() on indexed values that are not ValueContexts. So you should find a suitable representation of your arrays (e.g. a comma separated list or a json array string) if you want to lookup them. And then index that representation. Although if your object is some kind of array it will index all array items separately, so that each of them is query:able individually. Cheers Michael Am 13.07.2011 um 23:39 schrieb Aseem Kishore: I just learned that node/relationship property values can be arrays of primitives, not just primitives. That's pretty cool, but I'm wondering, how does this work (if it does work) with indexing and lookups? I'm referring to the REST API. E.g. if the URL is: GET /index/node/my_nodes/the_key/the_value What should I be putting in for the_value? I don't think it should be URL-encoded JSON, because for simple strings, quotes don't go into the URL. So is this even possible? Help appreciated! Thanks much. Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] REST API paging: specify sorting? and for index results?
It's great to hear that traverse results over the REST API will now support paging in Neo4j 1.4. But we were wondering: will you be able to specify the sorting to determine which results are returned first? E.g. suppose I have a node for me, and I follow a bunch of other people. To get a list of the people I follow, I would do a traverse of depth 1 from me for follow. Would I be able to control the sorting of the pages? And what's the default sorting based on, in the case of a traverse? Related question: will index results also support paging? Those (for nodes at least) appear to be sorted by last modified time (most recently modified first) by default. It would be great to control that sorting, e.g. by creation time, or by alphabetical order of a certain property, etc. Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API paging: specify sorting? and for index results?
Hey, that totally makes sense. No worries about disappointing. =) What's the default sorting based on, then? Is it unspecified? I think it would be helpful to at least know, to help is in design. Cheers, Aseem On Wed, Jul 13, 2011 at 7:06 PM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, The paged traversers in the REST API don't support sorting, and probably never will do. Here's why: In order to sort, we need a set of results to sort. If we're going to have a completely sorted result set, we need a complete set of results which may be large. The irony is, even as paged traversals try to keep resource use low by holding in memory only the state needed to generate the current page, sorted paged results require massive resource consumption. It's even worse than non-paged results because now we have to have that state hanging around on the server until you've finished with it (previously at least we could dump it down the wire to the client and forget about it). So in order to influence the ordering of your paged (or non-paged) results, you write the traverser logic to favour certain sub-graphs. Does that make sense? Sorry to disappoint. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API paging: specify sorting? and for index results?
Sorry for not being clear -- I meant when the sorting isn't constrained by the algorithm or traverse parameters. A simple example is a social network, and you want to get a person's friends. This would be a breadth-first search of depth 1 for one relationship type (FOLLOWS/FRIEND/whatever). How will the returned nodes (or relationships, or both if it's a [full]path) be sorted? Thanks! Aseem On Wed, Jul 13, 2011 at 8:06 PM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, [snip] What's the default sorting based on, then? Is it unspecified? I think it would be helpful to at least know, to help is in design. At the most basic level, it can be down to whether you're doing a depth- or breadth-first search. It will be influenced by the relationship types you allow your traverser to explore, and it'll be influenced by your graph structure. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API paging: specify sorting? and for index results?
Okay, thanks for explaining. Would still love an answer to whether index results will support paging or not. =) Thanks! Aseem On Thu, Jul 14, 2011 at 1:23 AM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, I think in that case it's non-deterministic - which means you page until you have enough results. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] auto indexes in REST in 1.4?
+1 re: REST API supporting auto-indexing. =) Aseem On Mon, Jul 11, 2011 at 9:39 AM, Boris Kizelshteyn bo...@popcha.com wrote: Hi All, Does the new improved REST api support the auto indexes feature? What about indexing and querying numbers? If so I would really appreciate some examples. Many thanks! ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Constraints on index key names?
Is there any constraint on what an index key is allowed to be named? E.g. can it have a '+' in its name? Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Unique Constaint on Index
I'll strongly +1 that having a concept of unique index values should be built into Neo4j. It's just too common of a requirement. Aseem On Thu, Jul 7, 2011 at 11:48 AM, etc3 e...@nextideapartners.com wrote: I'm new to this framework, is there an example that demonstrates removing a non-existent property and how it would be used in this context? Thanks -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Chris Gioran Sent: Thursday, July 07, 2011 11:04 AM To: Neo4j user discussions Subject: Re: [Neo4j] Unique Constaint on Index Hi, the ability to acquire locks cluster-wide exists, albeit in an ad hoc fashion. Grabbing a write lock on the node you want to ensure is uniquely indexed will ensure that the operations are serialized across all cluster members. The most simple way to get that lock currently is the (somewhat hackish but entirely correct) removal of a non-existing property. cheers, CG On Thu, Jul 7, 2011 at 5:53 PM, etc3 e...@nextideapartners.com wrote: How do I ensure another request is not performing the same operation on another node in the cluster? -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Marko Rodriguez Sent: Thursday, July 07, 2011 10:35 AM To: Neo4j user discussions Subject: Re: [Neo4j] Unique Constaint on Index Hi, We are testing Neo4J and need to support unique emails across all users. Is this possible with the current API? You can add such a constraint when updating the indices: if(index.get('email', address).hasNext()) { throw new RuntimeException(There are two nodes that share the same email address.); } else { index.put('email', address, node); } Marko. http://markorodriguez.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Announcing the latest milestone (1.4.M05)
Aw, that's too bad. =( We were really looking forward to auto-indexing, and we only use Neo4j via the REST API, as our app is a Node.js app, not a Java one. It would be great to see even something basic. If there's anything we can do to help, let us know. Cheers, Aseem On Wed, Jun 29, 2011 at 12:22 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Aseem, there is currently no ability to configure the auto indexer from the REST API. It is definite that this functionality will be added but it is unclear at this time when this will happen. We would be happy as well though if this was included in 1.4 GA. cheers, CG On Wed, Jun 29, 2011 at 8:46 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Awesome stuff! One question: is there any ability yet to use (i.e. configure) auto-indexing from the REST API? If not, will that be a part of 1.4 final? Cheers, Aseem On Tue, Jun 28, 2011 at 7:03 AM, Andres Taylor andres.tay...@neotechnology.com wrote: Hello graphistas, Today we’re releasing the fifth and final milestone in our 1.4 “Kiruna Stol” family. We’ve expanded our feature set to include Auto Indexing and paged traversers for the REST API, the Cypher query language has seen some major improvements and as always, we’ve squeezed more performance out of our database engine. Auto indexing -- In this milestone we have included an improved version of the auto indexing functionality that was announced shortly after 1.4.M04 and has been available in development snapshots since then. You’ll now find new configuration options for separately enabling auto indexing for node and relationship properties and for defining which properties will be indexed. For example, if you want to auto-index all node properties called “name” and all relationship properties called “since” the proper configuration is: MapString, String config = new HashMapString, String(); config.put(Config.NODE_AUTO_INDEXING, “true”); config.put(Config.NODE_KEYS_INDEXABLE, “name”); config.put(Config.RELATIONSHIP_AUTO_INDEXING, “true”); config.put(Config.RELATIONSHIP_KEYS_INDEXABLE, “since”); Passing the above to the GraphDatabaseService constructor will enable auto indexing for matching nodes and relationships, and on every commit the auto index will be updated automatically. For more details, check out the javadocs and the examples in the manual available at: http://docs.neo4j.org/chunked/1.4.M05/auto-indexing.html Paged REST Traversers After including batch operations in the previous milestone, now we also provide the option to page results of traversals via a special URI contained in Node representations and that is similar to the existing Traverser REST API. You can define the page size and lease time, having this way finer control the retrieval of traverser results from the client (and also help improve performance on the server). Cypher improvements - The newly introduced Cypher query language is growing more sophisticated and in this milestone we’ve added aggregates, ordering and limits. This allows for more expressive queries, making the following a valid statement against the cineasts database: START user=(User,login,'micha') MATCH (user)-[:FRIEND]-(friend)-[r,:RATED]-(movie) RETURN movie.title, AVG(r.stars), COUNT(*) ORDER BY AVG(r.stars) DESC, COUNT(*) DESC limit 7 To see Cypher in action, check out the screencasthttp://neo4j.vidcaster.com/U2Y/introduction-to-cypher/by our own Michael Hunger. Almost there Of course that is not all. As always, we have bug fixes, performance improvements and usability enhancements, the most prevalent of which is the removal of YAJSW as the wrapper and service installer, which should bring joy to the mac users among us. With the GA release to follow shortly, we’d like you to download, try out and provide us with your valuable feedback on these new features so that we can deliver the best possible 1.4 to you, our community. For more details and links go to the release announcement here http://blog.neo4j.org/2011/06/neo4j-14-m05-kiruna-stol-midsummer.html . Andrés Taylor ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Announcing the latest milestone (1.4.M05)
Awesome stuff! One question: is there any ability yet to use (i.e. configure) auto-indexing from the REST API? If not, will that be a part of 1.4 final? Cheers, Aseem On Tue, Jun 28, 2011 at 7:03 AM, Andres Taylor andres.tay...@neotechnology.com wrote: Hello graphistas, Today we’re releasing the fifth and final milestone in our 1.4 “Kiruna Stol” family. We’ve expanded our feature set to include Auto Indexing and paged traversers for the REST API, the Cypher query language has seen some major improvements and as always, we’ve squeezed more performance out of our database engine. Auto indexing -- In this milestone we have included an improved version of the auto indexing functionality that was announced shortly after 1.4.M04 and has been available in development snapshots since then. You’ll now find new configuration options for separately enabling auto indexing for node and relationship properties and for defining which properties will be indexed. For example, if you want to auto-index all node properties called “name” and all relationship properties called “since” the proper configuration is: MapString, String config = new HashMapString, String(); config.put(Config.NODE_AUTO_INDEXING, “true”); config.put(Config.NODE_KEYS_INDEXABLE, “name”); config.put(Config.RELATIONSHIP_AUTO_INDEXING, “true”); config.put(Config.RELATIONSHIP_KEYS_INDEXABLE, “since”); Passing the above to the GraphDatabaseService constructor will enable auto indexing for matching nodes and relationships, and on every commit the auto index will be updated automatically. For more details, check out the javadocs and the examples in the manual available at: http://docs.neo4j.org/chunked/1.4.M05/auto-indexing.html Paged REST Traversers After including batch operations in the previous milestone, now we also provide the option to page results of traversals via a special URI contained in Node representations and that is similar to the existing Traverser REST API. You can define the page size and lease time, having this way finer control the retrieval of traverser results from the client (and also help improve performance on the server). Cypher improvements - The newly introduced Cypher query language is growing more sophisticated and in this milestone we’ve added aggregates, ordering and limits. This allows for more expressive queries, making the following a valid statement against the cineasts database: START user=(User,login,'micha') MATCH (user)-[:FRIEND]-(friend)-[r,:RATED]-(movie) RETURN movie.title, AVG(r.stars), COUNT(*) ORDER BY AVG(r.stars) DESC, COUNT(*) DESC limit 7 To see Cypher in action, check out the screencasthttp://neo4j.vidcaster.com/U2Y/introduction-to-cypher/by our own Michael Hunger. Almost there Of course that is not all. As always, we have bug fixes, performance improvements and usability enhancements, the most prevalent of which is the removal of YAJSW as the wrapper and service installer, which should bring joy to the mac users among us. With the GA release to follow shortly, we’d like you to download, try out and provide us with your valuable feedback on these new features so that we can deliver the best possible 1.4 to you, our community. For more details and links go to the release announcement herehttp://blog.neo4j.org/2011/06/neo4j-14-m05-kiruna-stol-midsummer.html . Andrés Taylor ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Traversals versus Indexing
Cool, thanks! On Fri, Jun 17, 2011 at 5:01 AM, Mattias Persson matt...@neotechnology.comwrote: getNodeById is fast/immediate 2011/6/16 Aseem Kishore aseem.kish...@gmail.com Indeed getNodeById(). Aseem On Thu, Jun 16, 2011 at 10:11 AM, Mattias Persson matt...@neotechnology.com wrote: Are you thinking of GraphDatabaseService#getNodeById / #getRelationshipById ? Or index lookups? 2011/6/16 Aseem Kishore aseem.kish...@gmail.com Are lookups by ID also so much slower than traversals? Aseem On Thu, Jun 16, 2011 at 3:45 AM, Mattias Persson matt...@neotechnology.comwrote: I don't know your use case at all, but one of the benefits you get with traversing compared to index lookups is that one hop from a node to another is instantaneous ( 1 million hops / second on a fully cached graph), whereas index lookups are several order of magnitudes slower than that. But index lookups are good for when you f.ex. have thousands/millions of names and you'd like to get the node with a certain name. Then that would be your starting point for doing a traversal to find other information local to that node, or in its vicinity. 2011/6/13 Aman aman.6...@gmail.com What is faster - Traversals or Indexing? I mean if one has a database model that can offer a choice between the two, what should one choose? Also, what about when the scalability factor comes in? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Announcing Neo4j 1.4 M04 “Kiruna Stol”
Quick q: will the new REST API batching feature treat the operations as a single transaction? That would be convenient for some scenarios I think, like swapping two nodes in a linked list. Aseem On Fri, Jun 10, 2011 at 7:48 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Hello graphistas, today brings us the (hopefully) latest milestone towards the “Kiruna Stol” 1.4 release of Neo4j. You will find it contains two new major features that are meant to make your interaction with the database easier and more efficient as well as the usual range of minor bug fixes. Cypher, an expressive query language for graphs == So far, the main means of interacting with the graph have been either the Traversal API or the REST API, leaving Neo4j wanting of a portable, expressive query language. Starting today, however, we are pleased to release Cypher, the first iteration of a query language designed for both application developers and operations specialists that want to create ad-hoc, portable queries against a Neo4j instance. You will find that the syntax has an SQL flavour to it so it is easy to get used to but all the graphy semantics goodness is there, ready to be harnessed. So, go right ahead and check out the manual at http://docs.neo4j.org/chunked/1.4.M04/query-lang.html and tell us what you think. Batching for the REST API Cypher is not the only major feature in this milestone. The battle-proven REST interface gets new functionality, allowing definition and execution of batch operations. You can now post JSON encoded requests that contain any number of commands to the “batch URI” of the server and have the server execute them in one go, reducing this way the load on the server, the client and the network. Documentation on this experimental feature is available at http://docs.neo4j.org/chunked/1.4.M04/rest-api-batch-ops.html The above might be the stars of the show but, as always, there are many bug fixes and minor improvements scattered around the code. If you are curious, the changelogs have all the juicy details. So, go to http://neo4j.org to download 1.4.M04 and to http://blog.neo4j.org/2011/06/kiruna-stol-14-milestone-4.html to read more about it and let us know what you think. Cheers, /peter ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Announcing Neo4j 1.4 M04 “Kiruna Stol”
That's awesome! Thanks. Aseem On Fri, Jun 17, 2011 at 11:54 AM, Michael Hunger michael.hun...@neotechnology.com wrote: yes it does. one tx per batch M Sent from my iBrick4 Am 17.06.2011 um 19:17 schrieb Aseem Kishore aseem.kish...@gmail.com: Quick q: will the new REST API batching feature treat the operations as a single transaction? That would be convenient for some scenarios I think, like swapping two nodes in a linked list. Aseem On Fri, Jun 10, 2011 at 7:48 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Hello graphistas, today brings us the (hopefully) latest milestone towards the “Kiruna Stol” 1.4 release of Neo4j. You will find it contains two new major features that are meant to make your interaction with the database easier and more efficient as well as the usual range of minor bug fixes. Cypher, an expressive query language for graphs == So far, the main means of interacting with the graph have been either the Traversal API or the REST API, leaving Neo4j wanting of a portable, expressive query language. Starting today, however, we are pleased to release Cypher, the first iteration of a query language designed for both application developers and operations specialists that want to create ad-hoc, portable queries against a Neo4j instance. You will find that the syntax has an SQL flavour to it so it is easy to get used to but all the graphy semantics goodness is there, ready to be harnessed. So, go right ahead and check out the manual at http://docs.neo4j.org/chunked/1.4.M04/query-lang.html and tell us what you think. Batching for the REST API Cypher is not the only major feature in this milestone. The battle-proven REST interface gets new functionality, allowing definition and execution of batch operations. You can now post JSON encoded requests that contain any number of commands to the “batch URI” of the server and have the server execute them in one go, reducing this way the load on the server, the client and the network. Documentation on this experimental feature is available at http://docs.neo4j.org/chunked/1.4.M04/rest-api-batch-ops.html The above might be the stars of the show but, as always, there are many bug fixes and minor improvements scattered around the code. If you are curious, the changelogs have all the juicy details. So, go to http://neo4j.org to download 1.4.M04 and to http://blog.neo4j.org/2011/06/kiruna-stol-14-milestone-4.html to read more about it and let us know what you think. Cheers, /peter ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Traversals versus Indexing
Are lookups by ID also so much slower than traversals? Aseem On Thu, Jun 16, 2011 at 3:45 AM, Mattias Persson matt...@neotechnology.comwrote: I don't know your use case at all, but one of the benefits you get with traversing compared to index lookups is that one hop from a node to another is instantaneous ( 1 million hops / second on a fully cached graph), whereas index lookups are several order of magnitudes slower than that. But index lookups are good for when you f.ex. have thousands/millions of names and you'd like to get the node with a certain name. Then that would be your starting point for doing a traversal to find other information local to that node, or in its vicinity. 2011/6/13 Aman aman.6...@gmail.com What is faster - Traversals or Indexing? I mean if one has a database model that can offer a choice between the two, what should one choose? Also, what about when the scalability factor comes in? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Traversals versus Indexing
Indeed getNodeById(). Aseem On Thu, Jun 16, 2011 at 10:11 AM, Mattias Persson matt...@neotechnology.com wrote: Are you thinking of GraphDatabaseService#getNodeById / #getRelationshipById ? Or index lookups? 2011/6/16 Aseem Kishore aseem.kish...@gmail.com Are lookups by ID also so much slower than traversals? Aseem On Thu, Jun 16, 2011 at 3:45 AM, Mattias Persson matt...@neotechnology.comwrote: I don't know your use case at all, but one of the benefits you get with traversing compared to index lookups is that one hop from a node to another is instantaneous ( 1 million hops / second on a fully cached graph), whereas index lookups are several order of magnitudes slower than that. But index lookups are good for when you f.ex. have thousands/millions of names and you'd like to get the node with a certain name. Then that would be your starting point for doing a traversal to find other information local to that node, or in its vicinity. 2011/6/13 Aman aman.6...@gmail.com What is faster - Traversals or Indexing? I mean if one has a database model that can offer a choice between the two, what should one choose? Also, what about when the scalability factor comes in? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Auto Indexing for Neo4j
Awesome to hear, and great work! Will we be able to configure+use this from the REST API? Cheers, Aseem On Tue, Jun 14, 2011 at 8:30 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Good news everyone, A request that's often come up on the mailing list is a mechanism for automatically indexing properties of nodes and relationships. As of today's SNAPSHOT, auto-indexing is part of Neo4j which means nodes and relationships can now be indexed based on convention, requiring far less effort and code from the developer's point of view. Getting hold of an automatic index is straightforward: AutoIndexerNode nodeAutoIndexer = graphDb.index().getNodeAutoIndexer(); AutoIndexNode nodeAutoIndex = nodeAutoIndexer.getAutoIndex(); Once you've got an instance of AutoIndex, you can use it as a read-only IndexNode. The AutoIndexer interface also supports runtime changes and enabling/disabling the auto indexing functionality. To support the new features, there are new Config options you can pass to the startup configuration map in EmbeddedGraphDatabase, the most important of which are: Config.NODE_AUTO_INDEXING (defaults to false) Config.RELATIONSHIP_AUTO_INDEXING (defaults to false) If set to true (independently of each other) these properties will enable auto indexing functionality and at the successful finish() of each transaction, all newly added properties on the primitives for which auto indexing is enabled will be added to a special AutoIndex (and deleted or changed properties will be updated accordingly too). There are options for fine grained control to determine properties are indexed, default behaviors and so forth. For example, by default all properties are indexed. If you want only properties name and age for Nodes and since and until for Relationships to be auto indexed, simply set the initial configuration as follows: Config.NODE_KEYS_INDEXABLE = name, age; Config.RELATIONSHIP_KEYS_INDEXABLE=since, until; For the semantics of the auto-indexing operations, constraints and more detailed examples, see the documentation available at http://docs.neo4j.org/chunked/1.4-SNAPSHOT/auto-indexing.html We're pretty excited about this feature since we think it'll make your lives as developers much more productive in a range of use-cases. If you're comfortable with using SNAPSHOT versions of Neo4j, please try it out and let us know what you think - we'd really value your feedback. If you're happier with using packaged milestones then this feature will be available from 1.4 M05 in a couple of weeks from now. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Does direction matter?
If I have a relationship from node A to node B, does the direction I make the relationship matter? It doesn't seem to affect *what* I can do (e.g. regardless of the direction, I can retrieve that relationship from both A and B in queries, and I can retrieve both A and B from each other in a traversal). But does it affect *how* the stuff I can do happens? E.g. does it affect perf? If the direction is from A to B, are discoveries/fetches of B from A faster than of A from B? Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Does direction matter?
Thanks McKinley! For what it's worth, we use Neo4j only via the REST API, so I tend not to look at the Javadocs. It would be great to move some of this sort of documentation into a general Neo4j documentation area, since one would think to go to the Javadocs mainly only if they were using the Java API. Cheers, Aseem On Fri, Jun 10, 2011 at 11:32 AM, McKinley mckinley1...@gmail.com wrote: Aseem, Even though all relationships have a direction they are equally well traversed in both directions so there's no need to create duplicate relationships in the opposite direction (with regard to traversal or performance). http://api.neo4j.org/current/org/neo4j/graphdb/Relationship.html Use relationship direction to express the model and meaning. Performance will be the same. Obviously, creating extra bidirectional relationships could have some meaning in your model and if so, you should document it. When it does not provide meaning, extra relationships can only get in the way and cause confusion and perhaps hurt performance. Cheers, McKinley On Fri, Jun 10, 2011 at 11:06 AM, Aseem Kishore aseem.kish...@gmail.com wrote: But does it affect *how* the stuff I can do happens? E.g. does it affect perf? If the direction is from A to B, are discoveries/fetches of B from A faster than of A from B? Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Design question: Del.icio.us
Max, just to be clear, are you basically saying that this problem is best solved by a relational db instead of a graph db? Thanks, Aseem On Wed, Jun 8, 2011 at 11:21 PM, Max De Marzi Jr. maxdema...@gmail.comwrote: Aseem, Which users tagged what? SELECT DISTINCT user_id FROM USER_TAGS WHERE bookmark_id = x Which bookmarks did a user tag? SELECT DISTINCT bookmark_id FROM USER_TAGS WHERE user_id = x Which bookmarks were tagged with x? SELECT DISTINCT bookmark_id FROM USER_TAGS WHERE tag_id = x What question are you trying to answer that you can't get from a relational table? I think it's better to make your model depending on the questions you are trying to answer. I have a similar model (users tagging users) and I needed to answer: Which users have been tagged by me or my network 6 degrees out? What is the tagged distance a user is away from another user? Turns out, all I needed was User nodes with Tagged links. User - tagged - User - tagged - User Everything else I get from the relational db. When I start to care about which users have been tagged for by me or my network 6 degrees out for this KEYWORD, then I'll create a more complex model: User - UserKeyword - tagged - UserKeyword - User Keyword ---/Keyword ---/ Regards, Max On Thu, Jun 9, 2011 at 12:33 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Also forgot to add: now if you want to show all links (globally) for a given tag, how do you do that? Are you indexing each tag-link (as a key-value pair) every time a tag is added? Removing from the index every time a tag is deleted? Thanks, Aseem On Wed, Jun 8, 2011 at 10:31 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I forgot to state this, but a key requirement of tags of course is that you can apply multiple of them to the same link. Would you still go that route then? If you want to delete a link from your bookmarks, it's a matter then of deleting each tag edge? Aseem On Wed, Jun 8, 2011 at 10:28 PM, Siddhartha Kasivajhula countvajh...@gmail.com wrote: Couldn't you use edges for tags instead of nodes? user1 --tag-- link1 user2 --tag-- link1 user1 --tag-- link2 On Wed, Jun 8, 2011 at 10:23 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hi guys, We're thinking about implementing tags in our system, and we're pondering a few ideas for how we'd go about it, and none of them seem perfect. I'd like to ask you guys: how would you design an analogous case: Del.icio.us ? At the core, *users bookmark links*, so it makes sense to have users and links as types of nodes, and bookmark as the type of edge that connects them. Now, you can see all the links a given user has bookmarked, and all the users that have bookmarked a given link. Now users can also *tag* links, of course, but the most interesting requirement is that, when I'm looking at *my links*, I only want to see *my tags*. Thus, tags can't be tied just to links; they have to be tied to users too. But most importantly, you have to maintain *which* user and *which* link *each* tag is for. Thus, it seems most intuitive to us if we could have tags be nodes that *point to the bookmark edges*. Unfortunately, that's a hypergraph, which Neo4j doesn't support. We could use intermediate nodes for bookmarks maybe, but that sort of sucks to lose the intuitiveness of the graph and introduce redundant edges, more complexity which makes consistency harder, etc. We have some other ideas, but I'm curious if any of you guys have thought about this already or have done it, or if you guys have thoughts. Thanks in advance! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Design question: Del.icio.us
Thanks for the input, Michael and Josef! I'm intrigued by the possibility of being able to have comment threads, etc. when you consider a tagging action as an event, and thus consider it a node in the graph. I'm worried though about having the duplicate edges. How do you make sure everything remains consistent? If I delete a tag, do you delete the node for the previous tag event too? Or just the edge? If you delete the node too, that deletes the comment thread too then? Some great food for thought. Let me chew on this a bit more and get back to you guys. Thanks again, Aseem On Thu, Jun 9, 2011 at 1:55 AM, Josef Holy ho...@centrum.cz wrote: Hi, few notes from the top of my head on how we modeled this in the past: User is a Node Resource (Document) is a Node Tag is a Node (having a label, language property, can be connected via relationships to other Tag Nodes representing its synonyms, can become a concept in (tree-)structured thesauri, etc...) Scenario: User U tags Resource R with Tag T: - Create special 'TaggingEvent' TE Node, with timestamp property and connect it to U, R and T by creating [TE, U], [TE, R] and [TE, T] relationships - TE nodes are indexed by timestamp, allowing to easily generate 'Tag activity streams' for different contexts - for users, resources or even tags (if it makes sense) - create some additional relationship(s) as needed between [U, T], [U, R] and [T, R] to simplify graph traversals and also to aggregate/cache the information like 'how many times was given R tagged with given T ~ this is f.e. expressed by some 'weight' property on the [T,R] relationship - and comes handy for generating things like tag clouds (~ get all [T,R] relationships with their weights) Scenario: User U bookmarks Resource R: - here it may be enough to create only [U,R] relationship with properties like timestamp, but we followed similar model as above - create special 'BookmarkEvent' Node BE, with timestamp and ralationships [BE, U] and [BE, R] and [U,R] - this again allows to easily create bookmarking activity streams, or to mix tagging+bookmarking events into one stream, if there is a requirement to do so. That's about it. Hope this helps. Regards, Josef. -- www.subject.cz __ Od: Max De Marzi Jr. maxdema...@gmail.com Komu: Neo4j user discussions user@lists.neo4j.org Datum: 09.06.2011 08:21 Předmět: Re: [Neo4j] Design question: Del.icio.us Aseem, Which users tagged what? SELECT DISTINCT user_id FROM USER_TAGS WHERE bookmark_id = x Which bookmarks did a user tag? SELECT DISTINCT bookmark_id FROM USER_TAGS WHERE user_id = x Which bookmarks were tagged with x? SELECT DISTINCT bookmark_id FROM USER_TAGS WHERE tag_id = x What question are you trying to answer that you can't get from a relational table? I think it's better to make your model depending on the questions you are trying to answer. I have a similar model (users tagging users) and I needed to answer: Which users have been tagged by me or my network 6 degrees out? What is the tagged distance a user is away from another user? Turns out, all I needed was User nodes with Tagged links. User - tagged - User - tagged - User Everything else I get from the relational db. When I start to care about which users have been tagged for by me or my network 6 degrees out for this KEYWORD, then I'll create a more complex model: User - UserKeyword - tagged - UserKeyword - User Keyword ---/Keyword ---/ Regards, Max On Thu, Jun 9, 2011 at 12:33 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Also forgot to add: now if you want to show all links (globally) for a given tag, how do you do that? Are you indexing each tag-link (as a key-value pair) every time a tag is added? Removing from the index every time a tag is deleted? Thanks, Aseem On Wed, Jun 8, 2011 at 10:31 PM, Aseem Kishore aseem.kish...@gmail.com wrote: I forgot to state this, but a key requirement of tags of course is that you can apply multiple of them to the same link. Would you still go that route then? If you want to delete a link from your bookmarks, it's a matter then of deleting each tag edge? Aseem On Wed, Jun 8, 2011 at 10:28 PM, Siddhartha Kasivajhula countvajh...@gmail.com wrote: Couldn't you use edges for tags instead of nodes? user1 --tag-- link1 user2 --tag-- link1 user1 --tag-- link2 On Wed, Jun 8, 2011 at 10:23 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hi guys, We're thinking about implementing tags in our system, and we're pondering a few ideas for how we'd go about it, and none of them seem perfect. I'd like to ask you guys: how would you design an analogous case: Del.icio.us? At the core, *users bookmark links*, so it makes sense to have users and links as types of nodes
[Neo4j] Design question: Del.icio.us
Hi guys, We're thinking about implementing tags in our system, and we're pondering a few ideas for how we'd go about it, and none of them seem perfect. I'd like to ask you guys: how would you design an analogous case: Del.icio.us? At the core, *users bookmark links*, so it makes sense to have users and links as types of nodes, and bookmark as the type of edge that connects them. Now, you can see all the links a given user has bookmarked, and all the users that have bookmarked a given link. Now users can also *tag* links, of course, but the most interesting requirement is that, when I'm looking at *my links*, I only want to see *my tags*. Thus, tags can't be tied just to links; they have to be tied to users too. But most importantly, you have to maintain *which* user and *which* link *each* tag is for. Thus, it seems most intuitive to us if we could have tags be nodes that *point to the bookmark edges*. Unfortunately, that's a hypergraph, which Neo4j doesn't support. We could use intermediate nodes for bookmarks maybe, but that sort of sucks to lose the intuitiveness of the graph and introduce redundant edges, more complexity which makes consistency harder, etc. We have some other ideas, but I'm curious if any of you guys have thought about this already or have done it, or if you guys have thoughts. Thanks in advance! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Design question: Del.icio.us
I forgot to state this, but a key requirement of tags of course is that you can apply multiple of them to the same link. Would you still go that route then? If you want to delete a link from your bookmarks, it's a matter then of deleting each tag edge? Aseem On Wed, Jun 8, 2011 at 10:28 PM, Siddhartha Kasivajhula countvajh...@gmail.com wrote: Couldn't you use edges for tags instead of nodes? user1 --tag-- link1 user2 --tag-- link1 user1 --tag-- link2 On Wed, Jun 8, 2011 at 10:23 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hi guys, We're thinking about implementing tags in our system, and we're pondering a few ideas for how we'd go about it, and none of them seem perfect. I'd like to ask you guys: how would you design an analogous case: Del.icio.us? At the core, *users bookmark links*, so it makes sense to have users and links as types of nodes, and bookmark as the type of edge that connects them. Now, you can see all the links a given user has bookmarked, and all the users that have bookmarked a given link. Now users can also *tag* links, of course, but the most interesting requirement is that, when I'm looking at *my links*, I only want to see *my tags*. Thus, tags can't be tied just to links; they have to be tied to users too. But most importantly, you have to maintain *which* user and *which* link *each* tag is for. Thus, it seems most intuitive to us if we could have tags be nodes that *point to the bookmark edges*. Unfortunately, that's a hypergraph, which Neo4j doesn't support. We could use intermediate nodes for bookmarks maybe, but that sort of sucks to lose the intuitiveness of the graph and introduce redundant edges, more complexity which makes consistency harder, etc. We have some other ideas, but I'm curious if any of you guys have thought about this already or have done it, or if you guys have thoughts. Thanks in advance! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Design question: Del.icio.us
Also forgot to add: now if you want to show all links (globally) for a given tag, how do you do that? Are you indexing each tag-link (as a key-value pair) every time a tag is added? Removing from the index every time a tag is deleted? Thanks, Aseem On Wed, Jun 8, 2011 at 10:31 PM, Aseem Kishore aseem.kish...@gmail.comwrote: I forgot to state this, but a key requirement of tags of course is that you can apply multiple of them to the same link. Would you still go that route then? If you want to delete a link from your bookmarks, it's a matter then of deleting each tag edge? Aseem On Wed, Jun 8, 2011 at 10:28 PM, Siddhartha Kasivajhula countvajh...@gmail.com wrote: Couldn't you use edges for tags instead of nodes? user1 --tag-- link1 user2 --tag-- link1 user1 --tag-- link2 On Wed, Jun 8, 2011 at 10:23 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Hi guys, We're thinking about implementing tags in our system, and we're pondering a few ideas for how we'd go about it, and none of them seem perfect. I'd like to ask you guys: how would you design an analogous case: Del.icio.us? At the core, *users bookmark links*, so it makes sense to have users and links as types of nodes, and bookmark as the type of edge that connects them. Now, you can see all the links a given user has bookmarked, and all the users that have bookmarked a given link. Now users can also *tag* links, of course, but the most interesting requirement is that, when I'm looking at *my links*, I only want to see *my tags*. Thus, tags can't be tied just to links; they have to be tied to users too. But most importantly, you have to maintain *which* user and *which* link *each* tag is for. Thus, it seems most intuitive to us if we could have tags be nodes that *point to the bookmark edges*. Unfortunately, that's a hypergraph, which Neo4j doesn't support. We could use intermediate nodes for bookmarks maybe, but that sort of sucks to lose the intuitiveness of the graph and introduce redundant edges, more complexity which makes consistency harder, etc. We have some other ideas, but I'm curious if any of you guys have thought about this already or have done it, or if you guys have thoughts. Thanks in advance! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API (optimistic or transactional) concurrency?
the problem with that is that the last-modified since would have to be kept in the graph store as well. Isn't it already for nodes and relationships, and wouldn't it be easy to add this for indexes? Apologies if I'm trivializing anything! So ETag (i.e. hashing the target seems more sensible). Question is what to include in that hash, imho everything that either is returned by the request (for instance on a get onto the node also its properties, relationships (and indexed values) which makes it really expensive to (re-)compute. So that should be something that is switched on by a configuration setting. It can certainly be made configurable, but supporting concurrency seems like a very sensible default, no? I'm not sure I understand why computing an E-Tag (hash) has to be expensive. A simplified view: the db returns a JSON string for every request; why not just use the string's native hashCode as the E-Tag? (And this would only be needed in cases where Last-Modified time isn't already being tracked, e.g. individual properties.) I'm also not quite sure about the semantics of ETags in the scope of more dynamic queries like traversals. As you would create an E-Tag for the whole result but one per returned element (node, relationship, path) would be more sensible, don't know if there is a way to specify that. Not sure I agree. The HTTP E-Tag header's purpose is exactly to be a hash of the response body for that request. It makes perfect sense for index queries, regardless of whether they're simple or advanced: give me a hash of the results of that query. That lets me update the index conditionally, saying only update the index if the state of the index is what I expect it to be: this hash. If you've got some time feel free to fork the neo4j-community github repo and add this capability to RestfulGraphDatabase to see how it comes out. Would be really interested in the result. I would love to help out, but as is usually the case, I have more pressing stuff on my plate at the moment. =) Let's keep this dialogue going, and if I get some time, I'd love to play around and see what I can do. Thanks for the offer! Cheers, Aseem On Wed, May 11, 2011 at 5:11 PM, Michael Hunger michael.hun...@neotechnology.com wrote: Asseem, the problem with that is that the last-modified since would have to be kept in the graph store as well. So ETag (i.e. hashing the target seems more sensible). Question is what to include in that hash, imho everything that either is returned by the request (for instance on a get onto the node also its properties, relationships (and indexed values) which makes it really expensive to (re-)compute. So that should be something that is switched on by a configuration setting. I'm also not quite sure about the semantics of ETags in the scope of more dynamic queries like traversals. As you would create an E-Tag for the whole result but one per returned element (node, relationship, path) would be more sensible, don't know if there is a way to specify that. If you've got some time feel free to fork the neo4j-community github repo and add this capability to RestfulGraphDatabase to see how it comes out. Would be really interested in the result. Otherwise let's continue the discussion. Cheers Michael Am 11.05.2011 um 22:09 schrieb Aseem Kishore: Thanks for the update Jake! Given that one of the principal goals behind HTTP and REST is scalability (distributability), your mail reaffirms my appreciation for optimistic concurrency -- no complex transaction semantics needed. The way Windows Azure implemented it was pretty straightforward: 1. All RESTful responses for a resource included a Last-Modified header and/or an E-Tag. (Off the top of my head, I can't remember if it was both or one or the other depending on the resource.) 2. You could send all RESTful requests to resources with an If-Unmodified-Since: Date or If-Match: E-Tag header. 3. If the condition you specified in your request is no longer true, the request fails w/ a response of 412 Precondition Failed. Any thoughts on applying these semantics to Neo4j? Here are a couple of examples: - Have *GET /node/123/properties* return a Last-Modified and/or E-Tag header, and let me send an If-Unmodified-Since or If-Match header when I put *PUT* or *DELETE* /node/123/properties. - Same for individual properties. (This may be a good candidate for E-Tag rather than tracking Last-Modified for every property.) Same for relationship properties too. - And most importantly, same for adding to the index: *GET* and *POST* to* /index/type/name/key/value*. *This would solve my problem*. =) Thanks for your consideration! Aseem On Wed, May 11, 2011 at 8:01 AM, Rick Bullotta rick.bullo...@thingworx.comwrote: I think you have to still do it over HTTP *in addition* to any other transport, if for no other reason than the ubiquity
Re: [Neo4j] Neo4j EC2 AMI image available
That's indeed a nice and simple approach. It would be nice if admins could access the web interface from a browser though. Would that be possible using the Amazon security approach? Aseem On Thu, May 12, 2011 at 7:00 PM, Daniel Gasienica dan...@gasienica.chwrote: Thanks, Peter. I've found that piece of documentation before but Jussi just outlined an even more straightforward way of securing Neo4j on AWS: http://jussiheinonen.blogspot.com/2011/05/neo4j-graph-database-server-image-in.html?showComment=1305213734091#c177328379806909558 –Daniel On Thu, May 12, 2011 at 11:31, Peter Neubauer peter.neuba...@neotechnology.com wrote: Daniel, I would put some Apache server in front of it and secure it on URL patterns, see http://docs.neo4j.org/chunked/snapshot/operations-security.html for a simplicstic example. Would that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, May 11, 2011 at 10:57 PM, Daniel Gasienica dan...@gasienica.ch wrote: This is great news! What is the simplest way to secure a Neo4j instance on AWS? Daniel On Tue, May 10, 2011 at 06:29, Peter Neubauer peter.neuba...@neotechnology.com wrote: Hi all, Jussi Heinonen from OpenCredo has created a Neo4j Server EC2 image to try out things with. Well done, thanks for the effort Jussi! Read more abotu it and feel free to give feedback - http://jussiheinonen.blogspot.com/2011/05/neo4j-graph-database-server-image-in.html Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Looping through all values of an index, is it possible?
+1 would find this really valuable and useful as well -- indeed for a browse/filter UI. Aseem On Mon, May 9, 2011 at 8:57 AM, Rick Bullotta rick.bullo...@thingworx.comwrote: Related - since you would typically drive the selection/filtering process using both. Sorta like Solr facets. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Mattias Persson [matt...@neotechnology.com] Sent: Monday, May 09, 2011 8:51 AM To: Neo4j user discussions Subject: Re: [Neo4j] Looping through all values of an index, is it possible? Well, the title says looping through all values of an index whereas the text in the mail says looping through all keys of an index. That's why I asked for the use case because they seem separate. 2011/5/9 Rick Bullotta rick.bullo...@thingworx.com FYI, that's one of the capabilities I proposed adding to the Index Framework (no one has responded positive or negatively yet). Common use case is to drive a user interface with a list of valid values to facilitate filtering. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Mattias Persson [matt...@neotechnology.com] Sent: Monday, May 09, 2011 8:07 AM To: Neo4j user discussions Subject: Re: [Neo4j] Looping through all values of an index, is it possible? No there isn't a way to do that currently. What's your use case? 2011/5/9 Pablo Pareja ppar...@era7.com Hi all, I don't know it this has already been asked at some point but I've been looking up for it in the mail-list archive and didn't find it. So the question is pretty simple, as the name of this thread says, is it somehow possible retrieving all keys belonging to an specific index? Cheers, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Examples of multiple indices in use?
Interesting. That's assuming a person and an organization can share the same name. Maybe an edge case in this example, but I can understand. Thanks. Aseem On Thu, May 5, 2011 at 12:58 AM, Mattias Persson matt...@neotechnology.comwrote: Let's say you have persons and organisations in your graph... they both have the name property. Now you could index both persons and organisations in the same index with the name property, but then you'd get mixed results back. Or you could invent some key convention, like org_name and person_name but then the key will differ from the actual property key. Whereas that is possible the more natural separation would be to have those in two separate indexes, one called persons and one called organisations and just index the name properties for the corresponding entities in there. 2011/5/5 Aseem Kishore aseem.kish...@gmail.com Does anyone here actually use multiple node/relationship indices? If so, I'd love to understand the use case. What do multiple indices get you that you couldn't get with just one index? Thanks, Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user