Re: [Neo4j] WebAdmin visualization tool for large DB ?
Hi, I just wanted to ask if there has been any advance on this/ Cheers, Pablo On Mon, Oct 17, 2011 at 2:58 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: Of course! -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jacob Hansson Sent: Monday, October 17, 2011 8:53 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? That would be really interesting! I'll make a note of it and ping you when this gets up to the top of the backlog, if that's ok with you? /Jake On Mon, Oct 17, 2011 at 2:50 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: We're doing something similar as part of a visual search engine for ThingWorx. Our UX designers might be able to provide some recommendations/input on Neo4J DB visualization also. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jacob Hansson Sent: Monday, October 17, 2011 8:30 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? On Mon, Oct 17, 2011 at 12:49 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: I think that both Neo4J and webadmin should have the option of specifying first N relationships as a limiter. This limit should be on a per-relationship basis (e.g. the first N of each type, rather than simply the first N relationships). This way, you would be able to visualize all of the relationship types that go in/out of the node, but not be overwhelmed with one specific relationship type. Also, it would be ideal if some visual cue that there were more relationships of a given type could be provided (e.g. a relationship line to a box labeled ...and 10,679 more From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Pablo Pareja [ppar...@era7.com] Sent: Monday, October 17, 2011 6:44 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? Hi Mattias, what about the hypothetical scenario where there are ~ 1.000.000 relationships going out from a node, would the visualization tool try to plot all theses relationships? If not, would it first fetch them all and then plot a 1.000.000 group node? Or would it rather plot something like* more than 'x' relationships coming out from here*? Currently, it would pull down all relationships, and render a group. Since we've added support for paging traversals since that code was written, we could do something like what Rick is suggesting. It would be fun to sketch a bit on how a UI like that could work.. There is quite a bit of stuff in the webadmin backlog, so this won't get done right now, but it will eventually bubble to the top of the list :) Cheers, Pablo On Mon, Oct 17, 2011 at 12:38 PM, Mattias Persson matt...@neotechnology.com wrote: Hi Pablo, I don't think if there are any hard limits in the webadmin visualizer, It mostly depends on your computer and browser (what with the implementation of web workers and all) I'd say. 2011/10/13 Pablo Pareja ppar...@era7.com Hi, I was looking for a way to visualize and interact with small subsets of large Neo4j databases, *(somehow* *navigating through the data in order to have a general idea of how things are organized at specific parts* *of my DB)*. So before starting to create my own solution to that, I'd like to know which are the limits/constraints of the new WebAdmin tool ?* (in terms of number of rels/nodes in the neighborhood/ total DB)* Thanks, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https
Re: [Neo4j] Neo4j upcoming features importance poll
Thank you all for participating and give your votes to the options you feel are more important. For sure it'd be great to have another platform to do this that could be somehow integrated with GitHub, ( I just created the poll with doodle because it was the one I found after a quick google search which could fit the needs at some level...) @Peter yeah I think he'll be :D Cheers, Pablo On Tue, Nov 22, 2011 at 3:51 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Uservoice seems great. If rapportive uses it, http://feedback.rapportive.com/forums/42557-general then it is good in my book. I think we should try it if we can integrate this with GIThub issues. Pablo, impressive feedback on http://www.doodle.com/wg8k77vwq6b654bv ! I think Mattias will be delighted that the supernode support is on top ;) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Sat, Nov 19, 2011 at 9:29 PM, Peter Bell li...@pbell.com wrote: Uservoice might be a good fit. I used it for feature voting on as OSS project and it worked out pretty well... Sent from my iPhone On Nov 19, 2011, at 2:11 PM, Nigel Small ni...@nigelsmall.name wrote: Actually sounds like we may have finally found a use for Google Wave! :-P On 19 Nov 2011 13:09, Pablo Pareja ppar...@era7.com wrote: Yeah it'd be great having something more wiki-like that everyone could edit. I have no idea though about how could this be done Any ideas? Pablo On Sat, Nov 19, 2011 at 7:54 PM, Nigel Small ni...@nigelsmall.name wrote: How about something like Wufoo? http://www.wufoo.com/ http://www.wufoo.com/ *Nigel Small* Phone: +44 7814 638 246 Blog: http://nigelsmall.name/ GTalk: ni...@nigelsmall.name MSN: nasm...@live.co.uk Skype: technige Twitter: @technige https://twitter.com/#!/technige LinkedIn: http://uk.linkedin.com/in/nigelsmall On 19 November 2011 18:45, Peter Neubauer peter.neuba...@neotechnology.comwrote: I really like this. Is there any other transparent public method you poll, like a Google form that everyone can edit? On Nov 19, 2011 7:19 PM, Pablo Pareja ppar...@era7.com wrote: I just added a link for every possible upcoming feature and created an issue for those which didn't have one so far. Sorry for those who voted already but since the options changed, their vote was lost, could you please vote again? From now on every time we add a new feature to the poll we should create its respective issue before adding it. At least, whenever a new option is added, the votes for the rest of options are conserved, so we should be able update our votes just adding our vote (or not) to the new ones. Sorry for the inconvenience! Pablo On Sat, Nov 19, 2011 at 5:21 PM, Pablo Pareja ppar...@era7.com wrote: Ok, I just did that for the first one; the bad thing about this is that every time I edit one of the options, all the votes cast for it get lost and you have to edit your vote again... So maybe from now on I'd be better adding new features to the poll only once their respective issues has been risen in github. What do you think? Pablo On Sat, Nov 19, 2011 at 5:16 PM, Pablo Pareja ppar...@era7.com wrote: Yeah that'd be cool, if you give me the links I can put them as part of the options themselves (with bit.ly or something like that). Cheers, Pablo On Sat, Nov 19, 2011 at 1:30 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Guys, This is great! Could you raise issues for these and we mitigate missing voting on Github with this, linking back to github for discussion? On Nov 19, 2011 1:09 PM, Pablo Pareja ppar...@era7.com wrote: @Linan get_or_create feature added ;) @Mattias I mean being required to specify a node type at creation time, (as how things are right now with relationships) On Sat, Nov 19, 2011 at 1:01 PM, Mattias Persson matt...@neotechnology.comwrote: hat exactly does mandatory node types mean? 2011/11/19 Pablo Pareja ppar...@era7.com Hi all, I was thinking it'd be cool to create a sort of a poll in order to know which features (that are missing right now...) are the most important ones for the community. I just did a quick google search for free online poll creation platforms and found doodle site, (btw do you know a better site to do this?). The address for the poll is: http://www.doodle.com/wg8k77vwq6b654bv So far I just added three features that came to my mind while I was creating it, so please say which features you're missing and I'll add them so
[Neo4j] Exporting Neo4j subgraphs to graphml/gexf....
Hi, I was wondering whether there's (or are any plans to implement) any way to export sub-graphs to graphml/gexf files? I did a quick google search and found this: http://stackoverflow.com/questions/2204440/convert-neo4j-db-to-xml However I think it'd be better being able to do this as part of Neo4j core API, so that once you are situated in an specific node/relationship of the graph you could just export its neighborhood (specifying some parameters like depth) to one of these formats. Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4j upcoming features importance poll
Hi all, I was thinking it'd be cool to create a sort of a poll in order to know which features (that are missing right now...) are the most important ones for the community. I just did a quick google search for free online poll creation platforms and found doodle site, (btw do you know a better site to do this?). The address for the poll is: http://www.doodle.com/wg8k77vwq6b654bv So far I just added three features that came to my mind while I was creating it, so please say which features you're missing and I'll add them so that we can all vote for them or not. What do you think about all this? Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j upcoming features importance poll
Yeah that'd be cool, if you give me the links I can put them as part of the options themselves (with bit.ly or something like that). Cheers, Pablo On Sat, Nov 19, 2011 at 1:30 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Guys, This is great! Could you raise issues for these and we mitigate missing voting on Github with this, linking back to github for discussion? On Nov 19, 2011 1:09 PM, Pablo Pareja ppar...@era7.com wrote: @Linan get_or_create feature added ;) @Mattias I mean being required to specify a node type at creation time, (as how things are right now with relationships) On Sat, Nov 19, 2011 at 1:01 PM, Mattias Persson matt...@neotechnology.comwrote: hat exactly does mandatory node types mean? 2011/11/19 Pablo Pareja ppar...@era7.com Hi all, I was thinking it'd be cool to create a sort of a poll in order to know which features (that are missing right now...) are the most important ones for the community. I just did a quick google search for free online poll creation platforms and found doodle site, (btw do you know a better site to do this?). The address for the poll is: http://www.doodle.com/wg8k77vwq6b654bv So far I just added three features that came to my mind while I was creating it, so please say which features you're missing and I'll add them so that we can all vote for them or not. What do you think about all this? Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j upcoming features importance poll
Ok, I just did that for the first one; the bad thing about this is that every time I edit one of the options, all the votes cast for it get lost and you have to edit your vote again... So maybe from now on I'd be better adding new features to the poll only once their respective issues has been risen in github. What do you think? Pablo On Sat, Nov 19, 2011 at 5:16 PM, Pablo Pareja ppar...@era7.com wrote: Yeah that'd be cool, if you give me the links I can put them as part of the options themselves (with bit.ly or something like that). Cheers, Pablo On Sat, Nov 19, 2011 at 1:30 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Guys, This is great! Could you raise issues for these and we mitigate missing voting on Github with this, linking back to github for discussion? On Nov 19, 2011 1:09 PM, Pablo Pareja ppar...@era7.com wrote: @Linan get_or_create feature added ;) @Mattias I mean being required to specify a node type at creation time, (as how things are right now with relationships) On Sat, Nov 19, 2011 at 1:01 PM, Mattias Persson matt...@neotechnology.comwrote: hat exactly does mandatory node types mean? 2011/11/19 Pablo Pareja ppar...@era7.com Hi all, I was thinking it'd be cool to create a sort of a poll in order to know which features (that are missing right now...) are the most important ones for the community. I just did a quick google search for free online poll creation platforms and found doodle site, (btw do you know a better site to do this?). The address for the poll is: http://www.doodle.com/wg8k77vwq6b654bv So far I just added three features that came to my mind while I was creating it, so please say which features you're missing and I'll add them so that we can all vote for them or not. What do you think about all this? Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j upcoming features importance poll
I just added a link for every possible upcoming feature and created an issue for those which didn't have one so far. Sorry for those who voted already but since the options changed, their vote was lost, could you please vote again? From now on every time we add a new feature to the poll we should create its respective issue before adding it. At least, whenever a new option is added, the votes for the rest of options are conserved, so we should be able update our votes just adding our vote (or not) to the new ones. Sorry for the inconvenience! Pablo On Sat, Nov 19, 2011 at 5:21 PM, Pablo Pareja ppar...@era7.com wrote: Ok, I just did that for the first one; the bad thing about this is that every time I edit one of the options, all the votes cast for it get lost and you have to edit your vote again... So maybe from now on I'd be better adding new features to the poll only once their respective issues has been risen in github. What do you think? Pablo On Sat, Nov 19, 2011 at 5:16 PM, Pablo Pareja ppar...@era7.com wrote: Yeah that'd be cool, if you give me the links I can put them as part of the options themselves (with bit.ly or something like that). Cheers, Pablo On Sat, Nov 19, 2011 at 1:30 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Guys, This is great! Could you raise issues for these and we mitigate missing voting on Github with this, linking back to github for discussion? On Nov 19, 2011 1:09 PM, Pablo Pareja ppar...@era7.com wrote: @Linan get_or_create feature added ;) @Mattias I mean being required to specify a node type at creation time, (as how things are right now with relationships) On Sat, Nov 19, 2011 at 1:01 PM, Mattias Persson matt...@neotechnology.comwrote: hat exactly does mandatory node types mean? 2011/11/19 Pablo Pareja ppar...@era7.com Hi all, I was thinking it'd be cool to create a sort of a poll in order to know which features (that are missing right now...) are the most important ones for the community. I just did a quick google search for free online poll creation platforms and found doodle site, (btw do you know a better site to do this?). The address for the poll is: http://www.doodle.com/wg8k77vwq6b654bv So far I just added three features that came to my mind while I was creating it, so please say which features you're missing and I'll add them so that we can all vote for them or not. What do you think about all this? Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j upcoming features importance poll
Yeah it'd be great having something more wiki-like that everyone could edit. I have no idea though about how could this be done Any ideas? Pablo On Sat, Nov 19, 2011 at 7:54 PM, Nigel Small ni...@nigelsmall.name wrote: How about something like Wufoo? http://www.wufoo.com/ http://www.wufoo.com/ *Nigel Small* Phone: +44 7814 638 246 Blog: http://nigelsmall.name/ GTalk: ni...@nigelsmall.name MSN: nasm...@live.co.uk Skype: technige Twitter: @technige https://twitter.com/#!/technige LinkedIn: http://uk.linkedin.com/in/nigelsmall On 19 November 2011 18:45, Peter Neubauer peter.neuba...@neotechnology.comwrote: I really like this. Is there any other transparent public method you poll, like a Google form that everyone can edit? On Nov 19, 2011 7:19 PM, Pablo Pareja ppar...@era7.com wrote: I just added a link for every possible upcoming feature and created an issue for those which didn't have one so far. Sorry for those who voted already but since the options changed, their vote was lost, could you please vote again? From now on every time we add a new feature to the poll we should create its respective issue before adding it. At least, whenever a new option is added, the votes for the rest of options are conserved, so we should be able update our votes just adding our vote (or not) to the new ones. Sorry for the inconvenience! Pablo On Sat, Nov 19, 2011 at 5:21 PM, Pablo Pareja ppar...@era7.com wrote: Ok, I just did that for the first one; the bad thing about this is that every time I edit one of the options, all the votes cast for it get lost and you have to edit your vote again... So maybe from now on I'd be better adding new features to the poll only once their respective issues has been risen in github. What do you think? Pablo On Sat, Nov 19, 2011 at 5:16 PM, Pablo Pareja ppar...@era7.com wrote: Yeah that'd be cool, if you give me the links I can put them as part of the options themselves (with bit.ly or something like that). Cheers, Pablo On Sat, Nov 19, 2011 at 1:30 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Guys, This is great! Could you raise issues for these and we mitigate missing voting on Github with this, linking back to github for discussion? On Nov 19, 2011 1:09 PM, Pablo Pareja ppar...@era7.com wrote: @Linan get_or_create feature added ;) @Mattias I mean being required to specify a node type at creation time, (as how things are right now with relationships) On Sat, Nov 19, 2011 at 1:01 PM, Mattias Persson matt...@neotechnology.comwrote: hat exactly does mandatory node types mean? 2011/11/19 Pablo Pareja ppar...@era7.com Hi all, I was thinking it'd be cool to create a sort of a poll in order to know which features (that are missing right now...) are the most important ones for the community. I just did a quick google search for free online poll creation platforms and found doodle site, (btw do you know a better site to do this?). The address for the poll is: http://www.doodle.com/wg8k77vwq6b654bv So far I just added three features that came to my mind while I was creating it, so please say which features you're missing and I'll add them so that we can all vote for them or not. What do you think about all this? Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https
Re: [Neo4j] Scalability Roadmap
Hi Serge, Regarding supernodes I already opened an issue about this some time ago: https://github.com/neo4j/community/issues/19 and as you can read there, at the end of the conversation Peter said: we will hopefully be on it for 1.6 ! I really hope they keep thinking of fixing this for 1.6 release, I'd actually say that this is one of the most urgent points that should be covered right now... Cheers, Pablo Pareja On Fri, Nov 18, 2011 at 5:38 PM, serge s.fedoro...@gmail.com wrote: Are these following topics will be treated in future release (and when if you know) ? 1/ Supernode I know there is a big downside in handle of super-nodes, which can be a big issue in a twitter-like website with, for example a user followed by more than 200k users (i have in head, real case) or in a recommendation system which have sophisticated rules. I would like to know if the super-node issue (as we name it) is planned to be investigated in futures releases ? 2/ Sharding and horizontal scalability I guess sharding is a complex problem to handle with graph db but is it planned to address the horizontal scalability goal ? and that, even if it should bring us towards kind of inconsistensy but acceptable situation (for example, there are many cases of synchronization latency website can accept when it have a big load) Thanks -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Scalability-Roadmap-tp3519034p3519034.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Use Cases - Please tell your story
Maybe we could build a Neo4j graph with all the use-cases grouped by their field or application, creating the relationships that may link them and so on and then visualize all with the webadmin ;) Pablo On Fri, Nov 11, 2011 at 11:14 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Jim, that is a very good idea! I think we should maybe have some common place to collect these things. I personally have a number of slides demonstrating different use cases, but nothing crowdsourced. Does anyone have a good idea of collecting usecases and demos in a distributed way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Thu, Nov 10, 2011 at 11:25 PM, jimkaskade jim.kask...@gmail.com wrote: All Neo4J users out there. Can you provide a few sentences on your application/use-case? Some generic examples might include: 1. We use Neo4J to traverse a social graph, calculating degree of separation, centrality, who's connected to whom, who you should know / recommendations, influencers, etc. in our social network. 2. We use Neo4J to analyze spatial information for telecom call records...clusters, social connections, influencers, etc. 3. We perform LBS analysis for wireless carrier's customers usage patterns (over time and place) 4. We analyze ACH information for our bank...for fraud detection 5. We have a transportation/delivery application = routing optimization 6. Our implementation is part of a datacenter management/provisioning system 7. Network topology analysis/optimization 8. Cell tower utilization/optimization What's your killer application? Please elaborate ;-) Why am I asking? I'd like to present/blog/evangelize the successful use-cases of Neo4j ;-) Thanks! Jim -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Use-Cases-Please-tell-your-story-tp3498155p3498155.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] WebAdmin visualization (are rels. labels configurable?)
Cool, I'll be waiting for it ;) Pablo On Tue, Oct 18, 2011 at 9:40 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: They can't currently, but it is actually the feature I'm currently working on, so it's coming :) On Mon, Oct 17, 2011 at 10:03 PM, Pablo Pareja ppar...@era7.com wrote: Hi! I just tried the WebAdmin visualization time for the first time and it seems pretty cool. I have one question though, can be relationships labels be different than the rel. name? I've been playing a bit with the display options for nodes but didn't find anything for relationhips. Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Visualizing all nodes + relationships on WebAdmin
Yeah, you're right this wouldn't be a good idea for big graphs, and probably having different behaviors implemented depending on the size of the graph neither. What about showing nodes that fulfill a set of rules based on their properties, i.e. something like which is already implemented for the visualization profiles? In my case that would be sufficient cause I always add a property NODE_TYPE for all nodes, that way I could just create a couple of filters like 'where property node_type = X ' and then retrieve only the nodes of the type I'm interested in (regardless where they are situated in the graph). On Tue, Oct 18, 2011 at 2:56 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Pablo, I think this is mostly an issue of bandwidth and performance in processing the whole graph on the client side, alternatively generating some aggregated view on the server side but be less responsive on the client side. How do you imagine such interaction working? Got a hunch? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Oct 18, 2011 at 2:51 PM, Pablo Pareja ppar...@era7.com wrote: Well, I'd rather like to have the option of visualizing the whole graph without having to specify a depth value (cypher option) or direct IDs/Index lookup(Peter's) , cause I can think of different scenarios where that wouldn't be possible in principle. Would it be very difficult to implement the whole graph option? There already is a method in GraphDatabaseService class where you can access all nodes (then rels. implicitly as well) in your graph; so, couldn't you use this method to somehow achieve this? Cheers, Pablo On Tue, Oct 18, 2011 at 1:23 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Or, as long as you can address the nodes, e.g. via Index Lookup or direct IDs, which should return disconnected graphs, too. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Oct 18, 2011 at 1:22 PM, Jacob Hansson jacob.hans...@neotechnology.com wrote: No, there is currently no way to do that. Once we add cypher support to the databrowser search bar, you would be able to write a cypher query that returns everything at some arbitrary depth, which in practice could be used to return the whole graph, as long as it's all connected, would that be good enough? /jake On 2011 10 18 10:52, Pablo Pareja ppar...@era7.com wrote: Hi, Can this be achieved without having to access a subset based on some property/id/etc... and then opening all incoming/outgoing relationships and so on? Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User
Re: [Neo4j] WebAdmin visualization tool for large DB ?
Hi Mattias, what about the hypothetical scenario where there are ~ 1.000.000 relationships going out from a node, would the visualization tool try to plot all theses relationships? If not, would it first fetch them all and then plot a 1.000.000 group node? Or would it rather plot something like* more than 'x' relationships coming out from here*? Cheers, Pablo On Mon, Oct 17, 2011 at 12:38 PM, Mattias Persson matt...@neotechnology.com wrote: Hi Pablo, I don't think if there are any hard limits in the webadmin visualizer, It mostly depends on your computer and browser (what with the implementation of web workers and all) I'd say. 2011/10/13 Pablo Pareja ppar...@era7.com Hi, I was looking for a way to visualize and interact with small subsets of large Neo4j databases, *(somehow* *navigating through the data in order to have a general idea of how things are organized at specific parts* *of my DB)*. So before starting to create my own solution to that, I'd like to know which are the limits/constraints of the new WebAdmin tool ?* (in terms of number of rels/nodes in the neighborhood/ total DB)* Thanks, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] WebAdmin visualization tool for large DB ?
I thoroughly agree with Rick. Pablo On Mon, Oct 17, 2011 at 12:49 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: I think that both Neo4J and webadmin should have the option of specifying first N relationships as a limiter. This limit should be on a per-relationship basis (e.g. the first N of each type, rather than simply the first N relationships). This way, you would be able to visualize all of the relationship types that go in/out of the node, but not be overwhelmed with one specific relationship type. Also, it would be ideal if some visual cue that there were more relationships of a given type could be provided (e.g. a relationship line to a box labeled ...and 10,679 more From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Pablo Pareja [ppar...@era7.com] Sent: Monday, October 17, 2011 6:44 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? Hi Mattias, what about the hypothetical scenario where there are ~ 1.000.000 relationships going out from a node, would the visualization tool try to plot all theses relationships? If not, would it first fetch them all and then plot a 1.000.000 group node? Or would it rather plot something like* more than 'x' relationships coming out from here*? Cheers, Pablo On Mon, Oct 17, 2011 at 12:38 PM, Mattias Persson matt...@neotechnology.com wrote: Hi Pablo, I don't think if there are any hard limits in the webadmin visualizer, It mostly depends on your computer and browser (what with the implementation of web workers and all) I'd say. 2011/10/13 Pablo Pareja ppar...@era7.com Hi, I was looking for a way to visualize and interact with small subsets of large Neo4j databases, *(somehow* *navigating through the data in order to have a general idea of how things are organized at specific parts* *of my DB)*. So before starting to create my own solution to that, I'd like to know which are the limits/constraints of the new WebAdmin tool ?* (in terms of number of rels/nodes in the neighborhood/ total DB)* Thanks, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Size limits for REST Server?
Hi Jim and Peter, thanks for the info and the links, I'll have a look at them. As always happy for the great support around here ;) Pablo On Thu, Oct 13, 2011 at 5:49 PM, Jim Webber j...@neotechnology.com wrote: Hi Pablo, What do you exactly mean by discoverability ? Always getting all in/out rels for a node? Yup, exactly. By default the REST server projects the graph (naively) onto the Web (itself a graph). About writing an extension, has anyone already done this at some level? Is there any kind of docs about this? Yes, and I just noticed Peter responded with those links :-) And lastly (I just realized I only asked more questions so far... ) is the REST-Server somehow prepared for these extensions? I mean, could I use the 'skeleton' of the Server and only change some parts in order to get all this? With the managed and unmanaged extensions, you add your code into the server through APIs. The managed API gives you a very graph-centric view and manages creation of JSON and HTML representations on your behalf. The unmanaged API gives you a JAX-RS API plus direct access to the underlying database. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Article: The Coming SQL Collapse
it out even more as they continue to fill it up and expand their Domain Model. A bottom up approach (and I mean really BOTTOM) is the best practice, but it does take thought and research in finding where that generic bottom is at. A Friend - what is a Friend, what attributes does a Friend have ? Which ones do I care about now and can worry about later ? A Stock Issuing Company, do I need to think of my relationships first for those, or should I start with attributes ? Will those attributes be shared with others ? Do I care about it now at this early beginning of my Domain Model, or can I wait on this ? Should I start with attributes (properties) always, and leave relationships alone until the very end, when I build a Domain Model ? (the answer to that last question is YES, because Neo4j actually DOES allow you to) He attacks our pattern of building domain models with Neo4j, calling it ugly, smelly and in violation of every law of separation of concerns and logical vs. physical models. Is he right? My feeling is that he is brain washed with too many so called best practices, but Neo4j has been my main model for a long time now, my perspective is likely skewed. I'd like to hear your thoughts. Your main model IS NOT Neo4j. That is your main database that holds your Domain Model. And if Neo4j is not really doing this from top to bottom, then I would agree with Richard that the Neo4j infrastructure is not in the best place right now and might need more work. -- -Thad http://www.freebase.com/view/en/thad_guidry ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] WebAdmin visualization tool for large DB ?
Hi, I was looking for a way to visualize and interact with small subsets of large Neo4j databases, *(somehow* *navigating through the data in order to have a general idea of how things are organized at specific parts* *of my DB)*. So before starting to create my own solution to that, I'd like to know which are the limits/constraints of the new WebAdmin tool ?* (in terms of number of rels/nodes in the neighborhood/ total DB)* Thanks, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Size limits for REST Server?
Hi, I'd like to know which are the limits in terms of number of nodes/relationships, topology *(e.g. supernodes* *with millions of incoming/outgoing relationship in the same node)* for Neo4j REST-Server? Has it already been used for large DB / BigData ? Otherwise, is there anyway to tune the core functionality so that fetching methods for specific kind of nodes/rels/situations can be overwritten ? Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Size limits for REST Server?
Hi Jim, What do you exactly mean by discoverability ? Always getting all in/out rels for a node? About writing an extension, has anyone already done this at some level? Is there any kind of docs about this? And lastly (I just realized I only asked more questions so far... ) is the REST-Server somehow prepared for these extensions? I mean, could I use the 'skeleton' of the Server and only change some parts in order to get all this? Thanks! Pablo On Thu, Oct 13, 2011 at 4:32 PM, Jim Webber j...@neotechnology.com wrote: Hi Pablo, For the very specific use-cases I'd recommend writing an extension rather than using the default representations. The default behaviour of the REST API emphasises discoverability which in the case of supernodes isn't usually desirable. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Boston meet-up
Hi! I'll be in Boston from October 25th to October 29th, will any of you be in the area any of these days? Cheers, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] We now have an automatic discussion forum on the Nabble Mailing List Archive
Cool ! It looks much more usable than the mailing list ;) Cheers, Pablo On Thu, Jul 14, 2011 at 10:04 PM, Michael Hunger michael.hun...@neotechnology.com wrote: I forgot two things: 1) I changed all the links on neo4j.org that previously pointed to the ugly mailman archive page to point to nabble instead. 2) The discussion mailing list vs. forum is over, I'm very happy about that. Michael Am 14.07.2011 um 22:01 schrieb Michael Hunger: They seem to have updated their UI massively, it now looks and works like a forum while interacting with this neo4j mailing list. Landing page is here: http://neo4j-user-list.438527.n3.nabble.com You can browse and search the forums from there, some hints about mailing list usage and forum integration can be found here: http://neo4j-user-list.438527.n3.nabble.com/mailing_list/MailingListOptions.jtp?forum=438527 There are also some more options for people that are interested like Advanced Search, People, Topics view and an green online dot for logged in people. It is embeddable as well, so we'll investigate if and how to embed it on neo4j.org. If you register once at nabble with the same email address you used for this list, you can reply and also post from there without ever having to use your e-mail program. (You can even turn off to receive messages, so all your interaction can be through that forum). You can also send HTML emails, but I strongly discourage that, if you don't need it (for instance for embedding relevant images or formatted source code (monospace). Cheers Michael P.S. This post was written on nabble, so no e-mail program was harmed. http://neo4j-user-list.438527.n3.nabble.com/file/n3170182/reality_is_a_graph.png -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/We-now-have-an-automatic-discussion-forum-on-the-Nabble-Mailing-List-Archive-tp3170182p3170182.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Fulltext indexing apparently not working when done in Batch insertion phase...
Hi all! Recently I noticed that some fulltext indexes I created and populated in my project in the Batch insertion phase are not working as expected. I've carried out some tests and cannot come up with what can be going wrong. That's why I thought, ok let's try with a really basic example (mostly taken from the official wiki ) and see if it works right first. This is the code: BatchInserter inserter = new BatchInserterImpl(target/neo4jdb-batchinsert); BatchInserterIndexProvider indexProvider = new LuceneBatchInserterIndexProvider(inserter); BatchInserterIndex actors = indexProvider.nodeIndex(actors, MapUtil.stringMap(type, fulltext, provider, lucene)); MapString, Object properties = MapUtil.map(name, Keanu Reeves); long node = inserter.createNode(properties); actors.add(node, properties); indexProvider.shutdown(); inserter.shutdown(); GraphDatabaseService graphDbService = new EmbeddedGraphDatabase(CommonData.DATABASE_FOLDER); MapString, String indexFullTextProps = new HashMapString, String(); indexFullTextProps.put(provider, lucene); indexFullTextProps.put(type, fulltext); //--node indexes--- IndexNode index = graphDbService.index().forNodes(actors, indexFullTextProps); System.out.println(index.get(actors, Keanu).size()); System.out.println(index.get(actors, keanu).size()); System.out.println(index.get(actors, Keanu Reeves).size()); graphDbService.shutdown(); I'm probably doing something wrong but I'm getting 0 size in the three queries I perform... Otherwise, is there any kind of known issue for this scenario (I mean, fulltext indexing with Batch insertion) ? I'd really appreciate any help with this. Cheers, Pablo Pareja -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Fulltext indexing apparently not working when done in Batch insertion phase...
Hi again, I just found out that I had the wrong name for querying the actors index, still when I change it for name like this: System.out.println(index.get(name, Keanu).size()); System.out.println(index.get(name, keanu).size()); System.out.println(index.get(name, Keanu Reeves).size()); I still get unexpected results for a fulltext index: 0 0 1 so it seems that it is actually an exact index what was created, even though I specified the string 'fulltext' ... Pablo On Thu, Jun 9, 2011 at 4:49 PM, Pablo Pareja ppar...@era7.com wrote: Hi all! Recently I noticed that some fulltext indexes I created and populated in my project in the Batch insertion phase are not working as expected. I've carried out some tests and cannot come up with what can be going wrong. That's why I thought, ok let's try with a really basic example (mostly taken from the official wiki ) and see if it works right first. This is the code: BatchInserter inserter = new BatchInserterImpl(target/neo4jdb-batchinsert); BatchInserterIndexProvider indexProvider = new LuceneBatchInserterIndexProvider(inserter); BatchInserterIndex actors = indexProvider.nodeIndex(actors, MapUtil.stringMap(type, fulltext, provider, lucene)); MapString, Object properties = MapUtil.map(name, Keanu Reeves); long node = inserter.createNode(properties); actors.add(node, properties); indexProvider.shutdown(); inserter.shutdown(); GraphDatabaseService graphDbService = new EmbeddedGraphDatabase(CommonData.DATABASE_FOLDER); MapString, String indexFullTextProps = new HashMapString, String(); indexFullTextProps.put(provider, lucene); indexFullTextProps.put(type, fulltext); //--node indexes--- IndexNode index = graphDbService.index().forNodes(actors, indexFullTextProps); System.out.println(index.get(actors, Keanu).size()); System.out.println(index.get(actors, keanu).size()); System.out.println(index.get(actors, Keanu Reeves).size()); graphDbService.shutdown(); I'm probably doing something wrong but I'm getting 0 size in the three queries I perform... Otherwise, is there any kind of known issue for this scenario (I mean, fulltext indexing with Batch insertion) ? I'd really appreciate any help with this. Cheers, Pablo Pareja -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Fulltext indexing apparently not working when done in Batch insertion phase...
yeah, it works just fine using query(...) method instead of get(...) Thanks ;) Pablo On Thu, Jun 9, 2011 at 5:31 PM, Michael Hunger michael.hun...@neotechnology.com wrote: and you should use index.query(key,querystring) not get for fulltext queries Sent from my iBrick4 Am 09.06.2011 um 17:04 schrieb Pablo Pareja ppar...@era7.com: Hi again, I just found out that I had the wrong name for querying the actors index, still when I change it for name like this: System.out.println(index.get(name, Keanu).size()); System.out.println(index.get(name, keanu).size()); System.out.println(index.get(name, Keanu Reeves).size()); I still get unexpected results for a fulltext index: 0 0 1 so it seems that it is actually an exact index what was created, even though I specified the string 'fulltext' ... Pablo On Thu, Jun 9, 2011 at 4:49 PM, Pablo Pareja ppar...@era7.com wrote: Hi all! Recently I noticed that some fulltext indexes I created and populated in my project in the Batch insertion phase are not working as expected. I've carried out some tests and cannot come up with what can be going wrong. That's why I thought, ok let's try with a really basic example (mostly taken from the official wiki ) and see if it works right first. This is the code: BatchInserter inserter = new BatchInserterImpl(target/neo4jdb-batchinsert); BatchInserterIndexProvider indexProvider = new LuceneBatchInserterIndexProvider(inserter); BatchInserterIndex actors = indexProvider.nodeIndex(actors, MapUtil.stringMap(type, fulltext, provider, lucene)); MapString, Object properties = MapUtil.map(name, Keanu Reeves); long node = inserter.createNode(properties); actors.add(node, properties); indexProvider.shutdown(); inserter.shutdown(); GraphDatabaseService graphDbService = new EmbeddedGraphDatabase(CommonData.DATABASE_FOLDER); MapString, String indexFullTextProps = new HashMapString, String(); indexFullTextProps.put(provider, lucene); indexFullTextProps.put(type, fulltext); //--node indexes--- IndexNode index = graphDbService.index().forNodes(actors, indexFullTextProps); System.out.println(index.get(actors, Keanu).size()); System.out.println(index.get(actors, keanu).size()); System.out.println(index.get(actors, Keanu Reeves).size()); graphDbService.shutdown(); I'm probably doing something wrong but I'm getting 0 size in the three queries I perform... Otherwise, is there any kind of known issue for this scenario (I mean, fulltext indexing with Batch insertion) ? I'd really appreciate any help with this. Cheers, Pablo Pareja -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Color suggestions for the Self-Relationship bike shed
Hi, definitely +1 for option 1 Pablo On Mon, May 16, 2011 at 2:40 PM, Adriano Henrique de Almeida adrianoalmei...@gmail.com wrote: I agree with Marko. In fact, I'd say that if you can choose more than one option, I'd go with option 1 and 2. The option one when you're not sure you're creating a loop (not sure if the end node will be the start node), so in order to avoid ifs on the client code, option 1 will go for it. If you're sure, and the operation is explicitly a loop, use the 2nd option. Regards 2011/5/16 Marko Rodriguez okramma...@gmail.com Hi, I say option 1. Marko. On May 16, 2011 8:12 AM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: There has been a few discussions around supporting Loops, relationships with the same start node as end node, recently. Both here on the mailing list, and even more at the Neo Technology office. We have a working patch for handling loops in Neo4j, but one final piece is missing: what should the API look like for creating a loop? There are three suggestions for this, I'll list them with the pros and cons we've found for each. It would be great if you could provide some feedback on what you think on the matter, and which alternative you prefer. The alternatives: 1. let the existing createRelationshipTo(Node,RelationshipType) create loops if the same node is passed as argument. 2. add a new createLoop(RelationshipType) method for creating loops. 3. add a createRelationshipOrLoop(Node,RelationshipType) method that would work like createRelationshipTo, except it would permit creating loops. The pros and cons: PRO 1: does not add a new method to the API that developers have to learn. CON 1: changes the semantics of the createRelationshipTo method slightly from what it is today. CON 1: will not help you catch programming errors where you've mistakenly written code that creates a relationship to the same node (most of the cases where code creates relationships to the wrong node it is to the same node). PRO 2: will let you be explicit of when creating a loop. PRO 2: will let createRelationshipTo preserve the semantics it has today. Which will help catching many create relationships to the wrong node cases. CON 2: will force you to be explicit about loops, most applications that wan't loops will just treat them as any relationship where the start node and end node will just happen to be the same. PRO 3: adds loops as a generic construct (start and end just happens to be the same) without changing the current semantics of createRelationshipTo CON 3: Introduces a new method that creates relationships between any two nodes. It would of course be possible to go with both 2 and 3, and I think option 3 makes more sense as an addition to option 2, rather than as an alternative to it. What do you think? -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Adriano Almeida Caelum | Ensino e Inovação www.caelum.com.br ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Retrieving a node linked to the reference node takes forever
Ok, I'm running iostat command and this is what I'm getting so far: *(First results)* avg-cpu: %user %nice %system %iowait %steal %idle 0.050.000.030.420.30 99.20 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvdap11.8964.9814.25 161562 35424 xvdg 8.7096.3821.01 239634 52232 *(After ~ 10 minutes with the program stuck)* Linux 2.6.35.11-83.9.amzn1.x86_64 (ip-10-230-37-212)09/05/11 _x86_64_(4 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 0.070.050.213.020.26 96.39 Device:tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvdap13.8380.9515.04 227594 42280 xvdg107.06 880.5118.582475482 52232 On Mon, May 9, 2011 at 10:17 AM, Mattias Persson matt...@neotechnology.comwrote: So I'm assuming there's a high I/O load when this happens? It seems to be loading up one or more persistence windows (a part of the database) into memory. Could you verify that high I/O load with the iostat command (if you're on linux) or any process/system monitor? 2011/5/6 Pablo Pareja ppar...@era7.com There you go: Full thread dump Java HotSpot(TM) 64-Bit Server VM (19.0-b09 mixed mode): Low Memory Detector daemon prio=10 tid=0x7fa5ac0b7000 nid=0x526 runnable [0x] java.lang.Thread.State: RUNNABLE CompilerThread1 daemon prio=10 tid=0x7fa5ac0b5000 nid=0x525 waiting on condition [0x] java.lang.Thread.State: RUNNABLE CompilerThread0 daemon prio=10 tid=0x7fa5ac0b2000 nid=0x524 waiting on condition java.lang.Thread.State: RUNNABLE Signal Dispatcher daemon prio=10 tid=0x7fa5ac0b nid=0x523 waiting on condition [0x] java.lang.Thread.State: RUNNABLE Finalizer daemon prio=10 tid=0x7fa5ac091800 nid=0x522 in Object.wait() [0x7fa5a20a2000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on 0x000103b4afb0 (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118) - locked 0x000103b4afb0 (a java.lang.ref.ReferenceQueue$Lock) [ec2-user@ip-10-230-55-104 bio4j_volume]$ at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) Reference Handler daemon prio=10 tid=0x7fa5ac08f800 nid=0x521 in Object.wait() [0x7fa5a21a3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on 0x000103b4af48 (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:485) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) - locked 0x000103b4af48 (a java.lang.ref.Reference$Lock) main prio=10 tid=0x7fa5ac007000 nid=0x51b runnable [0x7fa5b06cd000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.FileDispatcher.pread0(Native Method) at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:31) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:199) at sun.nio.ch.IOUtil.read(IOUtil.java:175) at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:612) at org.neo4j.kernel.impl.nioneo.store.PersistenceRow.readPosition(PersistenceRow.java:91) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:182) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:525) at org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:326) at org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getMoreRelationships(ReadTransaction.java:102) at org.neo4j.kernel.impl.nioneo.xa.NioNeoDbPersistenceSource$ReadOnlyResourceConnection.getMoreRelationships(NioNeoDbPersistenceSource.java:280) at org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:100) at org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:585) at org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:357) - locked 0x0005b3cf02d0 (a org.neo4j.kernel.impl.core.NodeImpl) at org.neo4j.kernel.impl.core.IntArrayIterator.hasNext(IntArrayIterator.java:115) at org.neo4j.kernel.impl.core.IntArrayIterator.next(IntArrayIterator.java:164) at org.neo4j.kernel.impl.core.IntArrayIterator.next(IntArrayIterator.java:34) at com.era7.bioinfo.bio4j.programs.ImportProteinInteractions.main
[Neo4j] Looping through all values of an index, is it possible?
Hi all, I don't know it this has already been asked at some point but I've been looking up for it in the mail-list archive and didn't find it. So the question is pretty simple, as the name of this thread says, is it somehow possible retrieving all keys belonging to an specific index? Cheers, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Retrieving a node linked to the reference node takes forever
Hi ! I just run into a problem I don't understand at all. The scenario is actually quite simple, basically it consists of a simple node retrieving through a relationship: 1. - There is a node (Protein self-interactions node) which is linked to the reference node by means of one relationship (Protein self-interactions relationship). This is the piece of code for the node creation/linkage public static ProteinSelfInteractionsRel proteinSelfInteractionsRel = new ProteinSelfInteractionsRel(null); ... proteinSelfInteractionsProperties.put(ProteinSelfInteractionsNode.NODE_TYPE_PROPERTY, ProteinSelfInteractionsNode.NODE_TYPE); long proteinSelfInteractionsNodeId = inserter.createNode(proteinSelfInteractionsProperties); inserter.createRelationship(inserter.getReferenceNode(), proteinSelfInteractionsNodeId, proteinSelfInteractionsRel, null); You can check the real code for this file herehttps://github.com/pablopareja/Bio4j/blob/master/src/com/era7/bioinfo/bio4j/programs/InitBio4jDB.java. (line 157) 2.- Later on when I try to get the id of that node: IterableRelationship iterable = manager.getReferenceNode().getRelationships(new ProteinSelfInteractionsRel(null),Direction.OUTGOING); logger.log(Level.INFO,getting node); long proteinSelfInteractionsNodeId = iterable.iterator().next().getEndNode().getId(); logger.log(Level.INFO,done!); (You can also check the code for this here https://github.com/pablopareja/Bio4j/blob/master/src/com/era7/bioinfo/bio4j/programs/ImportProteinInteractions.java(line 117) It just gets stuck forever after getting node There's no exception thrown and the program just seems to freeze forever (I've already been waiting for more than 1h with no success) Do you have any ideas of what could be going on?? I really have no clue of what could I be doing wrong... In case it may be useful, the size of the DB at this point is around 46.000.000 nodes and 380.000.000 relationships (only a dozen or so coming/going to the reference node) Thanks in advance, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Retrieving a node linked to the reference node takes forever
[0xfae0, 0xfc2c, 0x0001) object space 21248K, 46% used [0xfae0,0xfb7ad920,0xfc2c) Hope this can help Pablo On Fri, May 6, 2011 at 2:28 PM, Pablo Pareja ppar...@era7.com wrote: Ok, I'm gonna run it again and get the stack trace as you say Pablo On Fri, May 6, 2011 at 1:15 PM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: Could you provide a stack trace of what the stuck thread is doing? If you issue 'kill -3' to the process it will print a threaddump to stdout, that will contain a stack trace for the stuck thread. If you send that to me it would greatly help me diagnose the issue. Cheers, Tobias On Fri, May 6, 2011 at 12:53 PM, Pablo Pareja ppar...@era7.com wrote: Hi ! I just run into a problem I don't understand at all. The scenario is actually quite simple, basically it consists of a simple node retrieving through a relationship: 1. - There is a node (Protein self-interactions node) which is linked to the reference node by means of one relationship (Protein self-interactions relationship). This is the piece of code for the node creation/linkage public static ProteinSelfInteractionsRel proteinSelfInteractionsRel = new ProteinSelfInteractionsRel(null); ... proteinSelfInteractionsProperties.put(ProteinSelfInteractionsNode.NODE_TYPE_PROPERTY, ProteinSelfInteractionsNode.NODE_TYPE); long proteinSelfInteractionsNodeId = inserter.createNode(proteinSelfInteractionsProperties); inserter.createRelationship(inserter.getReferenceNode(), proteinSelfInteractionsNodeId, proteinSelfInteractionsRel, null); You can check the real code for this file here https://github.com/pablopareja/Bio4j/blob/master/src/com/era7/bioinfo/bio4j/programs/InitBio4jDB.java . (line 157) 2.- Later on when I try to get the id of that node: IterableRelationship iterable = manager.getReferenceNode().getRelationships(new ProteinSelfInteractionsRel(null),Direction.OUTGOING); logger.log(Level.INFO,getting node); long proteinSelfInteractionsNodeId = iterable.iterator().next().getEndNode().getId(); logger.log(Level.INFO,done!); (You can also check the code for this here https://github.com/pablopareja/Bio4j/blob/master/src/com/era7/bioinfo/bio4j/programs/ImportProteinInteractions.java (line 117) It just gets stuck forever after getting node There's no exception thrown and the program just seems to freeze forever (I've already been waiting for more than 1h with no success) Do you have any ideas of what could be going on?? I really have no clue of what could I be doing wrong... In case it may be useful, the size of the DB at this point is around 46.000.000 nodes and 380.000.000 relationships (only a dozen or so coming/going to the reference node) Thanks in advance, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Self-referencing relationships
a node to itself, then there is value in the few cases where you absolutely must have them. Being a database, the decision has been to err on the side of avoiding problems, even at the cost of some convenience. An approach that could allow for intentional self-referencing, while still protecting against accidental self-references, would be implement an explicit Node.relateToSelfAs(RelationshipType type). What does you model look like, that you expect to require self-referencing? Best, Andreas On Apr 26, 2011, at 11:15 AM, Shaunak Kashyap wrote: I know this topic has been discussed before[1] and that a trac issue was also created[2]. I see that a patch was submitted as part of the issue but from browsing the source code[3] it appears that self-referencing relationships are still a no-go in Neo4J. Are there any plans to apply the patch submitted by tobias OR, in general, to provide for self-referencing relationships in Neo4J? We are evaluating Neo4J against OrientDB for an internal tool at Yahoo. While Neo4J looks like a far more mature product overall, the lack of self-referencing relationships might become a sticking point (yes, I understand they can be worked around with a middle node but that requires the application to handle special cases which we would like to avoid). References: [1] http://www.mail-archive.com/user@lists.neo4j.org/msg03996.html [2] https://trac.neo4j.org/ticket/239 [3] https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/impl/core/RelationshipImpl.java#L45 Thank you, shaunak kashyap technical yahoo shau...@yahoo-inc.com direct 408-349-4024mobile 408-203-2450 701 first avenue, sunnyvale, ca, 94089-0703, us phone (408) 349 3300fax (408) 349 3301 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Rick Otten rot...@windfish.net O=='=+ ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Rick Otten rot...@windfish.net O=='=+ ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4j's projects delicious Bookmarks
Hi all! I was wondering how can Neo4j projects be added to delicious Neo4j's projects bookmarks ? I already created the link to Bio4j http://www.bio4j.com project in my delicious account http://www.delicious.com/pablopareja tagging it with neo4j and projects, however it's not displayed yet under the projects list. should I do something else? Cheers, Pablo Pareja -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Dynamic GEXF node attributes in neo4j database
Hi Matěj, Regarding the number of nodes/edges in the database you can use the following lines of code in case you haven't deleted any (because of id re-use): *l*ong number = ((EmbeddedGraphDatabase) graphDatabaseService).getConfig().getGraphDbModule().getNodeManager().getNumberOfIdsInUse(Node.class); System.out.println(nodes number = + number); number = ((EmbeddedGraphDatabase) graphDatabaseService).getConfig().getGraphDbModule().getNodeManager().getNumberOfIdsInUse(Relationship.class); System.out.println(relationships number = + number); Cheers, Pablo On Tue, Mar 22, 2011 at 10:57 PM, Matěj Plch plchm...@fit.cvut.cz wrote: Hi! Do you have any experience how to store changing attributes of node? It's like to build a table of changes for each changed node attribute. In SQL datastore quite simple, but in no-SQL? And one more question. Is it possible with some easy call though Java API how to get number of nodes and edges stored in the database? Or I have to traversal all graph? Thank you Matej Plch ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Beer and Talk
Hi! I really like the idea :) what about Granada, Spain? Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Batch Insertion, how to index relationships?
Hi! I just checked the wiki looking for information on how to index relationships in batch insertion mode but didn't find anything so far. This can be found in the wiki regarding relationship indexing: RelationshipIndex friendships = graphDb.index().forRelationships( friendships ); // type isn't a reserved key and isn't indexed automatically Relationship relationship = friendships.get( type, knows, morpheus, trinity ).getSingle(); However I cannot find any code snippet for adding relationships to the index, not just querying it. How can these two different cases (*batch insertion and standard mode*) be carried out? Besides that, I was wondering how node relationships retrieval is implemented. In my domain model I have some nodes which have hundred of thousands of relationships, including different types of them. Suppose you are already situated on one of these nodes and you want to get only one specific type of incoming relationships, retrieval time would be dependent on how many relationships are there including other types? or once you specify the relationship type in the node.getRelationships(...) method it doesn't matter how many relationships are there of other types in terms of relationship retrieval time? Thanks in advance, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Help with IndexWriter ramBufferSize exception
Hi all, I just got this exception: ramBufferSize 3584.0 is too large; should be comfortably less than 2048 SEVERE: org.apache.lucene.index.IndexWriter.setRAMBufferSizeMB(IndexWriter.java:1368) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.instantiateWriter(LuceneBatchInserterIndex.java:209) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.init(LuceneBatchInserterIndex.java:66) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider.index(LuceneBatchInserterIndexProvider.java:135) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider.nodeIndex(LuceneBatchInserterIndexProvider.java:102) SEVERE: com.era7.bioinfo.bio4j.programs.ImportGeneOntology.main(ImportGeneOntology.java:107) SEVERE: com.era7.bioinfo.bio4j.programs.ImportGeneOntology.execute(ImportGeneOntology.java:75) SEVERE: com.era7.lib.bioinfo.bioinfoutil.ExecuteFromFile.main(ExecuteFromFile.java:66) SEVERE: com.era7.bioinfo.bio4j.programs.CreateWholeBio4j.main(CreateWholeBio4j.java:28) when launching my Batch insertion program once I had updated the indexation to the new API any ideas about what should I do? I'm launching the program in a 68GB RAM machine with the following batchInserter.properties: neostore.nodestore.db.mapped_memory=10G neostore.relationshipstore.db.mapped_memory=12G neostore.propertystore.db.mapped_memory=12G neostore.propertystore.db.strings.mapped_memory=10G neostore.propertystore.db.arrays.mapped_memory=512M I don't know how or where to change that ramBufferSize' value, which by the way didn't specified at any point. Thanks in advance, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help with IndexWriter ramBufferSize exception
Hi Mattias, I guess that my only option would then be limiting the max amount to 2GB as you say, however this will make the insertion phase way longer than what I expected and counted on. Just to confirm, am I then supposed to launch my java process with at most -Xmx2G ?? I really think this is an important flaw to work on. I look forward to seeing it fixed in the near future. Cheers, Pablo On Thu, Feb 24, 2011 at 3:19 PM, Mattias Persson matt...@neotechnology.comwrote: There's this hardcoded setting for batch inserter indexes that increases the ram buffer size to increase overall performance during batch insertion. The formula is: double atLeast = writer.getRAMBufferSizeMB(); double heapHint = (double)(Runtime.getRuntime().maxMemory()/(1024*1024*16)); double ramBufferSize = Math.max( atLeast, heapHint ); writer.getRAMBufferSizeMB( ramBufferSize ); It's just something which I ballparked and obviously has problems for heaps of your size. It's a tough one to fix in your case since you cannot parameterize that calculation. The only way would be to modify source code and build a new jar, or wait for it to be fixed (maybe by limiting the max amount to 2Gb as Lucene seem to suggest). Hopefully it can be addressed soon. 2011/2/24 Pablo Pareja ppar...@era7.com: Hi all, I just got this exception: ramBufferSize 3584.0 is too large; should be comfortably less than 2048 SEVERE: org.apache.lucene.index.IndexWriter.setRAMBufferSizeMB(IndexWriter.java:1368) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.instantiateWriter(LuceneBatchInserterIndex.java:209) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.init(LuceneBatchInserterIndex.java:66) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider.index(LuceneBatchInserterIndexProvider.java:135) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider.nodeIndex(LuceneBatchInserterIndexProvider.java:102) SEVERE: com.era7.bioinfo.bio4j.programs.ImportGeneOntology.main(ImportGeneOntology.java:107) SEVERE: com.era7.bioinfo.bio4j.programs.ImportGeneOntology.execute(ImportGeneOntology.java:75) SEVERE: com.era7.lib.bioinfo.bioinfoutil.ExecuteFromFile.main(ExecuteFromFile.java:66) SEVERE: com.era7.bioinfo.bio4j.programs.CreateWholeBio4j.main(CreateWholeBio4j.java:28) when launching my Batch insertion program once I had updated the indexation to the new API any ideas about what should I do? I'm launching the program in a 68GB RAM machine with the following batchInserter.properties: neostore.nodestore.db.mapped_memory=10G neostore.relationshipstore.db.mapped_memory=12G neostore.propertystore.db.mapped_memory=12G neostore.propertystore.db.strings.mapped_memory=10G neostore.propertystore.db.arrays.mapped_memory=512M I don't know how or where to change that ramBufferSize' value, which by the way didn't specified at any point. Thanks in advance, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help with IndexWriter ramBufferSize exception
Cool, I thought I'd have to decrease the heap size much more. Anyways, thanks a lot for the quick response and support. Cheers, Pablo On Thu, Feb 24, 2011 at 3:44 PM, Mattias Persson matt...@neotechnology.comwrote: Using the formula I supplied in my previous reply I'd say you can go with a 32G heap or just slightly lower. 2011/2/24 Pablo Pareja ppar...@era7.com: Hi Mattias, I guess that my only option would then be limiting the max amount to 2GB as you say, however this will make the insertion phase way longer than what I expected and counted on. Just to confirm, am I then supposed to launch my java process with at most -Xmx2G ?? I really think this is an important flaw to work on. I look forward to seeing it fixed in the near future. Cheers, Pablo On Thu, Feb 24, 2011 at 3:19 PM, Mattias Persson matt...@neotechnology.comwrote: There's this hardcoded setting for batch inserter indexes that increases the ram buffer size to increase overall performance during batch insertion. The formula is: double atLeast = writer.getRAMBufferSizeMB(); double heapHint = (double)(Runtime.getRuntime().maxMemory()/(1024*1024*16)); double ramBufferSize = Math.max( atLeast, heapHint ); writer.getRAMBufferSizeMB( ramBufferSize ); It's just something which I ballparked and obviously has problems for heaps of your size. It's a tough one to fix in your case since you cannot parameterize that calculation. The only way would be to modify source code and build a new jar, or wait for it to be fixed (maybe by limiting the max amount to 2Gb as Lucene seem to suggest). Hopefully it can be addressed soon. 2011/2/24 Pablo Pareja ppar...@era7.com: Hi all, I just got this exception: ramBufferSize 3584.0 is too large; should be comfortably less than 2048 SEVERE: org.apache.lucene.index.IndexWriter.setRAMBufferSizeMB(IndexWriter.java:1368) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.instantiateWriter(LuceneBatchInserterIndex.java:209) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.init(LuceneBatchInserterIndex.java:66) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider.index(LuceneBatchInserterIndexProvider.java:135) SEVERE: org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider.nodeIndex(LuceneBatchInserterIndexProvider.java:102) SEVERE: com.era7.bioinfo.bio4j.programs.ImportGeneOntology.main(ImportGeneOntology.java:107) SEVERE: com.era7.bioinfo.bio4j.programs.ImportGeneOntology.execute(ImportGeneOntology.java:75) SEVERE: com.era7.lib.bioinfo.bioinfoutil.ExecuteFromFile.main(ExecuteFromFile.java:66) SEVERE: com.era7.bioinfo.bio4j.programs.CreateWholeBio4j.main(CreateWholeBio4j.java:28) when launching my Batch insertion program once I had updated the indexation to the new API any ideas about what should I do? I'm launching the program in a 68GB RAM machine with the following batchInserter.properties: neostore.nodestore.db.mapped_memory=10G neostore.relationshipstore.db.mapped_memory=12G neostore.propertystore.db.mapped_memory=12G neostore.propertystore.db.strings.mapped_memory=10G neostore.propertystore.db.arrays.mapped_memory=512M I don't know how or where to change that ramBufferSize' value, which by the way didn't specified at any point. Thanks in advance, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https
[Neo4j] Indexing with BatchInserter (with new API) how to doubt
Hi all, I'm updating my code in order to user the new Indexing API but I have a doubt, when using the BatchInserter, (at least with the old one) you were supposed to use the classes LuceneIndexBatchInserter or LuceneFulltextIndexBatchInserter but, what about now? should I just use the new way of indexing in the middle of the batch insertions? wouldn't that indexations be supposed to be done inside transaction blocks? As you can see I'm a bit confused with this, I'd really appreciate any clarification. Cheers, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] No index provider 'lucene' found with Tomcat/Trinidad
) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:188) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:288) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: org.jruby.exceptions.RaiseException: Native Exception: 'class java.lang.IllegalArgumentException'; Message: No index provider 'lucene' found; StackTrace: java.lang.IllegalArgumentException: No index provider 'lucene' found at org.neo4j.kernel.IndexManagerImpl.getIndexProvider(IndexManagerImpl.java:71) at org.neo4j.kernel.IndexManagerImpl.findIndexConfig(IndexManagerImpl.java:111) at org.neo4j.kernel.IndexManagerImpl.getOrCreateIndexConfig(IndexManagerImpl.java:173) at org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:250) Caused by: java.lang.IllegalArgumentException: No index provider 'lucene' found at org.neo4j.kernel.IndexManagerImpl.getIndexProvider(IndexManagerImpl.java:71) at org.neo4j.kernel.IndexManagerImpl.findIndexConfig(IndexManagerImpl.java:111) at org.neo4j.kernel.IndexManagerImpl.getOrCreateIndexConfig(IndexManagerImpl.java:173) at org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:250) /Andreas ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- You received this message because you are subscribed to the Google Groups neo4jrb group. To post to this group, send email to neo4...@googlegroups.com. To unsubscribe from this group, send email to neo4jrb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/neo4jrb?hl=en. -- You received this message because you are subscribed to the Google Groups neo4jrb group. To post to this group, send email to neo4...@googlegroups.com. To unsubscribe from this group, send email to neo4jrb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/neo4jrb?hl=en. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Indexing with BatchInserter (with new API) how to doubt
Hi Mattias, thanks for the link, I don't know how but after checking that page a couple of times I somehow didn't see/reach the BatchInserter indexing part... (guess I'd be too tired) Cheers, Pablo On Tue, Feb 22, 2011 at 8:40 PM, Mattias Persson matt...@neotechnology.comwrote: 2011/2/22 Pablo Pareja ppar...@era7.com Hi all, I'm updating my code in order to user the new Indexing API but I have a doubt, when using the BatchInserter, (at least with the old one) you were supposed to use the classes LuceneIndexBatchInserter or LuceneFulltextIndexBatchInserter but, what about now? should I just use the new way of indexing in the middle of the batch insertions? wouldn't that indexations be supposed to be done inside transaction blocks? The new batch inserter index thingie works almost the same way as the old batch insert index. Have you looked at http://wiki.neo4j.org/content/Index_Framework#Batch_insertion ? If you still have questions please let me know so I can add that information there. As you can see I'm a bit confused with this, I'd really appreciate any clarification. Cheers, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Exception creating EmbeddedGraphDatabase from a Servlet
Hi David, I just found a workaround for it this afternoon. I moved my neo4j db folder with all the data into the jetty installation folder and changed the db folder String using a relative path instead of an absolute one. (I guess it's not the most elegant one but it works fine so far). Cheers, Pablo On Tue, Feb 22, 2011 at 7:40 PM, David Montag david.mon...@neotechnology.com wrote: Pablo, Did you resolve this, or is this still an issue for you? David On Mon, Feb 21, 2011 at 2:08 AM, Pablo Pareja ppar...@era7.com wrote: Hi Jim, I already did it, I'm using a constant defined in the code for the DB folder which it's the same I used for testing things with a really simple jar. I also tried changing the permissions for every DB file granting every kind of permission to any kind of user (I know that's kind of crazy but just wanted to make sure it didn't have anything to do with that...) Pablo On Mon, Feb 21, 2011 at 11:00 AM, Jim Webber j...@neotechnology.com wrote: Hi Pablo, This caught my eye in your stacktrace: Unable to create directory path[] for Neo4j Can you confirm that you have provided the right path for your database into your Jetty app? Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- David Montag Neo Technology, www.neotechnology.com Cell: 650.556.4411 david.mon...@neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] No index provider 'lucene' found with Tomcat/Trinidad
Hi Mattias, I'll try that tomorrow and I'll let you know, Pablo On Tue, Feb 22, 2011 at 9:00 PM, Mattias Persson matt...@neotechnology.comwrote: 2011/2/22 Pablo Pareja ppar...@era7.com Hi, I just moved my code to the new index API but I cannot deploy my server. I've already tried with glassfishv3 and jetty and neither of them works. The exception I get is: java.lang.Exception: java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: java.lang.IllegalArgumentException: No index provider 'lucene' found I've already tried directly adding the apache bundle lucene jar to with no luck. any ideas of what could be happening? Could be java service API not loading stuff as it should probably (index providers are loaded that way). You say you tried adding the apache bundle lucene jar directly... did you also add neo4j-lucene-index jar file? The lucene index provider for neo4j consists of Lucene (lucene-core or lucene bundle) and a wrapper to make it participate nicely in Neo4j transactions/lifecycle (neo4j-lucene-index). Cheers, Pablo On Thu, Feb 17, 2011 at 11:31 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Mmmh, the main issue here I suspect is that the Java Service Loader is not finding the META-INF/services files. It might be that it is not triggered when you add files dynamically to the classloader. That is, the JARs are only scanned at startup time for any non-class files. Could that be an issue? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Feb 17, 2011 at 11:28 AM, Andreas Ronge andreas.ro...@gmail.com wrote: Hi I have found a work around - I added all the jar files needed in the classpath before I start Tomcat/Trinidad (put them in the lib folder) This is awkward since I don't know which jar files should be added (it's in a configuration file). Also, I have not managed to get the online backup working in Tomcat even when I add all the jar files to the classpath. It works when I run it standalone by just adding the online jar file at runtime, but not from Tomcat because of some classloading issue ? So, why can't I load the JAR files dynamically like this in Tomcat ? (which is the same as in ruby: require 'myjar.jar') URL url = getURL(jarFile); getRuntime().getJRubyClassLoader().addURL(url); Is there an alternative of loading the jar files dynamically at runtime ? (from http://thinkincode.net/2011/02/09/jruby-class-loader-by-example.html) /Andreas On Thu, Feb 17, 2011 at 9:33 AM, Anders Nawroth and...@neotechnology.com wrote: Hi! This is included as a dependency: org.apache.servicemix.bundles:org.apache.servicemix.bundles.lucene:jar:3.0.1_2:compile That's an OSGi-friendly packaging of Lucene. /anders On 02/17/2011 09:11 AM, Michael Hunger wrote: Sorry, didn't see your mail (had Andreas' still open for replying). It worked, thanks very much. But funnily the lucene jar is not recognized as dependency by maven for the server (mvn dependency:copy-dependencies). I assume it is added by the server-build project? Cheers Michael Am 17.02.2011 um 08:49 schrieb Mattias Persson: It's probably a classpath issue as you say. You added the neo4j-lucene-index-XXX.jar file manually to the classpath, but also make sure the lucene-core-3.0.1.jar is on the classpath. You can also look in the neo4j-db-directory/messages.log and see which extensions are loaded tigether with the graph database and which of those failed to load a.s.o. 2011/2/16 Andreas Rongeandreas.ro...@gmail.com Hi When I run my Rails application using Tomcat/Trinidad I get No index provider 'lucene' found, see stack trace below I think I have seen a similar problem on the gremlin email list. I guess it's because of a class loading issue, or maybe the META-INF/services files, or because in JRuby the neo4j jar files are loaded at runtime. I need some hints where I should start looking. I have tried to manually add the neo4j-lucene-index-1.3.M02.jar file to the classpath without any luck (jruby -J-cp /home/... -S trinidad) The problem occurs in neo4j.rb version= 1.0.0.beta.17 when the new lucene index framework was introduced. It works in WEBrick/rails s at org.jruby.rack.RackServlet.service(RackServlet.java:56
[Neo4j] Exception creating EmbeddedGraphDatabase from a Servlet
Hi! I'm having some trouble getting Neo4j DB to work from servlets in my Jetty server. There's no problem when I open the DB from a simple jar program file, however when I put the same code inside a servlet I get this exception: org.neo4j.graphdb.TransactionFailureException: Could not create data source [nioneodb], see nested exception for cause of error at org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:154) at org.neo4j.kernel.GraphDbInstance.start(GraphDbInstance.java:107) at org.neo4j.kernel.EmbeddedGraphDbImpl.init(EmbeddedGraphDbImpl.java:168) at org.neo4j.kernel.EmbeddedGraphDatabase.init(EmbeddedGraphDatabase.java:81) at org.neo4j.kernel.EmbeddedGraphDatabase.init(EmbeddedGraphDatabase.java:65) at com.era7.bioinfo.bioinfoneo4j.Neo4jManager.init(Neo4jManager.java:28) at com.era7.bioinfo.servletlibraryneo4j.servlet.BasicServletNeo4j.servletLogic(BasicServletNeo4j.java:138) at com.era7.bioinfo.servletlibraryneo4j.servlet.BasicServletNeo4j.doGet(BasicServletNeo4j.java:285) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:530) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:426) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:457) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:931) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:361) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:867) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:245) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113) at org.eclipse.jetty.server.Server.handle(Server.java:337) at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:581) at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1005) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:560) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:222) at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:474) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:437) at java.lang.Thread.run(Thread.java:636) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:532) at org.neo4j.kernel.impl.transaction.XaDataSourceManager.create(XaDataSourceManager.java:74) at org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:148) ... 31 more Caused by: java.io.IOException: Unable to create directory path[] for Neo4j store. at org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.autoCreatePath(NeoStoreXaDataSource.java:178) at org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.init(NeoStoreXaDataSource.java:124) ... 37 more I have to say that I'm not an expert at all dealing with jetty so it's possible that this exception is related to server configuration stuff I don't know about. Any ideas? Thanks in advance, Cheers -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Exception creating EmbeddedGraphDatabase from a Servlet
Hi Jim, I already did it, I'm using a constant defined in the code for the DB folder which it's the same I used for testing things with a really simple jar. I also tried changing the permissions for every DB file granting every kind of permission to any kind of user (I know that's kind of crazy but just wanted to make sure it didn't have anything to do with that...) Pablo On Mon, Feb 21, 2011 at 11:00 AM, Jim Webber j...@neotechnology.com wrote: Hi Pablo, This caught my eye in your stacktrace: Unable to create directory path[] for Neo4j Can you confirm that you have provided the right path for your database into your Jetty app? Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help with exception using BatchInserter
Hi Massimo, It's too bad you are running into the same kind of situations, (specially when the conclusion you came up to is that Neo4j just degrades...). However, did you already try dividing the big insertion process into smaller steps? I mean, do you think Neo4j degradation is just proportional to DB size ? or rather just to the amount of data being inserted in the same Batch Insertion? Cheers, Pablo On Thu, Feb 17, 2011 at 12:44 PM, Massimo Lusetti mluse...@gmail.comwrote: On Wed, Feb 16, 2011 at 11:12 AM, Mark Harwood markharw...@gmail.com wrote: However, the underlying Neo4J database doesn't seem to be able to cope with inserting these volumes of data on my available hardware and I don't have (and would hope not to need) 10s of gigabytes of RAM to throw at the problem. I'm using different data (see other threads) but this seems the same situations I've been running into... Neo4J degrade performance as soon as the db grows in size and it can't finish the work. Cheers -- Massimo http://meridio.blogspot.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://about.me/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help with exception using BatchInserter
Hi, First of all I wanted to thank the Neo4j team for the great support with this mail list. I'm answering here to Johan Svensson reply to the other thread with the same name, (*since this is the original I started before it got replicated somehow*). Pablo, you had a problem with batch inserter throwing an exception upon shutdown that I suspected was due to not enough available disk space. Then there was the the to many open files. Are you still experiencing problems with that? I already solved those exceptions. The first one was just that I run out of space as you said. As simply as that. I just didn't realize it since I had a volume of 300G built from a snapshot of 150G in which I forgot to run e2fsck -f and resize2fs so the partition table was not updated and it actually was like it had only 150G. Regarding the latter, as one of your team pointed out, it had something to do with the OS configuration regarding the allowed max number of files open at the same time. I managed to change the ulimit in my AWS instance and everything seems to work so far now. However one of your colleagues said that even if that could be a possible workaround, there should be something weird going on so that the BatchInserter needed to open so many files at the same time. I have to say that it's a really intensive insertion with millions of index lookups, millions of nodes and relationships insertions plus indexing nodes and relationships in the middle of all that. What do you think? Anyways I hope these two changes will make it and I won't run into more exceptions anymore. I'll let you all know how it goes once I finish this Batch Insertion step. Cheers, Pablo On Wed, Feb 16, 2011 at 10:05 AM, Pablo Pareja ppar...@era7.com wrote: Hi, I managed to change the ulimit in my machine and didn't get the Too many open files error again. However, even though no exceptions were thrown, this morning when I tested some queries against the DB results were quite weird and completely unexpected. Most nodes that should have been inserted were not, and inferring the total number of nodes/relationships on the DB using the call: * GraphService().getConfig().getGraphDbModule().getNodeManager().getNumberOfIdsInUse(Node.class/Relationship.class) * * * I got these numbers: nodes number = 2224916 (there should be at least 30/40.000.000) relationships number = 286701382 Then I checked the messages.log file and was really surprised when I saw this: Wed Feb 16 08:45:27 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore non clean shutdown detected Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore.relationshipstore.db non clean shutdown detected Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore.nodestore.db non clean shutdown detected Wed Feb 16 08:45:28 UTC 2011: Opened [/mnt/bio4j_volume/bio4jdb/nioneo_logical.log.1] clean empty log, version=0 Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore.relationshipstore.db rebuild id generator, highId=286701382 defragged count=0 Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore.nodestore.db rebuild id generator, highId=2224916 defragged count=0 Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore rebuild id generator, highId=4 defragged count=0 Wed Feb 16 08:45:28 UTC 2011: Opened [/mnt/bio4j_volume/bio4jdb/lucene/lucene.log.1] clean empty log, version=0 Wed Feb 16 08:45:28 UTC 2011: Opened [/mnt/bio4j_volume/bio4jdb/lucene-fulltext/lucene.log.1] clean empty log, version=0 Wed Feb 16 08:45:28 UTC 2011: Extension org.neo4j.kernel.KernelExtension[kernel jmx] initialized ok Wed Feb 16 08:45:28 UTC 2011: TM new log: tm_tx_log.1 How come there was a non clean shutdown() operation without any exceptions thrown? I'd really appreciate any help, Regarding the boiled down version of my code I guess I could prepare it but it's quite a big project and it'd probably be difficult to reproduce the exact situation for the insertion due to its dimensions. Cheers, Pablo On Tue, Feb 15, 2011 at 9:20 PM, Mattias Persson matt...@neotechnology.com wrote: If that many files are kept open at one point in time there's probably a bug in the code. I don't think that increasing number of files your OS allows is a correct solution, but may work as a temporary solution. I'd love to get a boiled down (or the code as is) version of you code so that I can replicate that behaviour and fix such bugs. 2011/2/15 Anders Nawroth and...@neotechnology.com Hi! 2011-02-15 13:19, Pablo Pareja: (Too many open files) This could be that you have old processes around that still keep lots of files open. Or that you simply have to few file handles available to begin with. Just google for Too many open files + your operating system, and you'll find information on how to configure this. /anders org.neo4j.index.lucene.LuceneIndexBatchInserterImpl.index
Re: [Neo4j] Help with exception using BatchInserter
Hi, I managed to change the ulimit in my machine and didn't get the Too many open files error again. However, even though no exceptions were thrown, this morning when I tested some queries against the DB results were quite weird and completely unexpected. Most nodes that should have been inserted were not, and inferring the total number of nodes/relationships on the DB using the call: * GraphService().getConfig().getGraphDbModule().getNodeManager().getNumberOfIdsInUse(Node.class/Relationship.class) * * * I got these numbers: nodes number = 2224916 (there should be at least 30/40.000.000) relationships number = 286701382 Then I checked the messages.log file and was really surprised when I saw this: Wed Feb 16 08:45:27 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore non clean shutdown detected Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore.relationshipstore.db non clean shutdown detected Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore.nodestore.db non clean shutdown detected Wed Feb 16 08:45:28 UTC 2011: Opened [/mnt/bio4j_volume/bio4jdb/nioneo_logical.log.1] clean empty log, version=0 Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore.relationshipstore.db rebuild id generator, highId=286701382 defragged count=0 Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore.nodestore.db rebuild id generator, highId=2224916 defragged count=0 Wed Feb 16 08:45:28 UTC 2011: /mnt/bio4j_volume/bio4jdb/neostore rebuild id generator, highId=4 defragged count=0 Wed Feb 16 08:45:28 UTC 2011: Opened [/mnt/bio4j_volume/bio4jdb/lucene/lucene.log.1] clean empty log, version=0 Wed Feb 16 08:45:28 UTC 2011: Opened [/mnt/bio4j_volume/bio4jdb/lucene-fulltext/lucene.log.1] clean empty log, version=0 Wed Feb 16 08:45:28 UTC 2011: Extension org.neo4j.kernel.KernelExtension[kernel jmx] initialized ok Wed Feb 16 08:45:28 UTC 2011: TM new log: tm_tx_log.1 How come there was a non clean shutdown() operation without any exceptions thrown? I'd really appreciate any help, Regarding the boiled down version of my code I guess I could prepare it but it's quite a big project and it'd probably be difficult to reproduce the exact situation for the insertion due to its dimensions. Cheers, Pablo On Tue, Feb 15, 2011 at 9:20 PM, Mattias Persson matt...@neotechnology.comwrote: If that many files are kept open at one point in time there's probably a bug in the code. I don't think that increasing number of files your OS allows is a correct solution, but may work as a temporary solution. I'd love to get a boiled down (or the code as is) version of you code so that I can replicate that behaviour and fix such bugs. 2011/2/15 Anders Nawroth and...@neotechnology.com Hi! 2011-02-15 13:19, Pablo Pareja: (Too many open files) This could be that you have old processes around that still keep lots of files open. Or that you simply have to few file handles available to begin with. Just google for Too many open files + your operating system, and you'll find information on how to configure this. /anders org.neo4j.index.lucene.LuceneIndexBatchInserterImpl.index(LuceneIndexBatchInserterImpl.java:186) com.era7.bioinfo.bio4j.programs.ImportUniprot.main(ImportUniprot.java:467) com.era7.bioinfo.bio4j.programs.ImportUniprot.execute(ImportUniprot.java:192) com.era7.lib.bioinfo.bioinfoutil.ExecuteFromFile.main(ExecuteFromFile.java:66) com.era7.bioinfo.bio4j.programs.CreateWholeBio4j.main(CreateWholeBio4j.java:16) java.lang.RuntimeException: java.io.FileNotFoundException: /mnt/bio4j_volume/bio4jdb/lucene/instit ute_name_index/_c7.frq (Too many open files) at org.neo4j.index.lucene.LuceneIndexBatchInserterImpl.optimize(LuceneIndexBatchInserterImpl.java:273) at org.neo4j.index.lucene.LuceneIndexBatchInserterImpl.shutdown(LuceneIndexBatchInserterImpl.java:214) at com.era7.bioinfo.bio4j.programs.ImportUniprot.main(ImportUniprot.java:552) at com.era7.bioinfo.bio4j.programs.ImportUniprot.execute(ImportUniprot.java:192) at com.era7.lib.bioinfo.bioinfoutil.ExecuteFromFile.main(ExecuteFromFile.java:66) at com.era7.bioinfo.bio4j.programs.CreateWholeBio4j.main(CreateWholeBio4j.java:16) Caused by: java.io.FileNotFoundException: /mnt/bio4j_volume/bio4jdb/lucene/institute_name_index/_c7.frq (Too many open files) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java:233) at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.init(SimpleFSDirectory.java:180) at org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) at org.apache.lucene.index.FormatPostingsDocsWriter.init(FormatPostingsDocsWriter.java:47
Re: [Neo4j] Exception when calling to optimize() method
Ok, I thought so at the beginning but since I keep getting different exceptions each time I try to populate my huge DB I have to consider every possible cause for the exceptions. On Tue, Feb 15, 2011 at 9:36 PM, Mattias Persson matt...@neotechnology.comwrote: You don't really need to call optimize at all... but it will optimize the read performance if you do it after a big phase of insertions, before heading into a phase where you query the index. Usually batch insertions consists of insertions of a lot of nodes (and indexing for those nodes) and then when it's time to create relationships: optimize the index, then do all your queries on it. Merging of index files (which optimize() will do very heavily) happens automatically in the background anyway when you insert stuff. 2011/2/14 Pablo Pareja ppar...@era7.com I just got an exception when calling to indexService.optimize(); java.io.IOException: background merge hit exception: _u6:c10863150 _u7:c2-_u7 into _u8 [optimize] [mergeDocStores] neo4j.index.lucene.LuceneIndexBatchInserterImpl.optimize(LuceneIndexBatchInserterImpl.java:273) any ideas? I'm calling the optimize() method once per ~ 30.000 indexations, is it that too much/few ? are you supposed to call this method always between an indexation and a query on that index? Cheers, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help with exception using BatchInserter
Hi Mark, Thanks for your contribution. I'm using a m2.4xlarge EC2 AWS instance type (*68.4 GB of memory, 26 EC2 Compute Units (8 virtual cores with 3.25 EC2 Compute Units each)).* Besides I'm parsing a 17G xml file for populating the DB and normally problems only arise when I have already inserted a decent amount of data (around 30.000.000 nodes and 50.000.000 relationships), that's why I was saying it'd be kind of harsh to reproduce the same situation. Right now I'm gonna try splitting the 17G xml file in at least 10-20 files and doing that number of separated batch insertions hoping it would work that way. I'll let you all know how it goes, Cheers, Pablo On Wed, Feb 16, 2011 at 11:12 AM, Mark Harwood markharw...@gmail.comwrote: Hi Pablo, Regarding the boiled down version of my code I guess I could prepare it but it's quite a big project Here's a boiled-down batch load demo I did earlier based on public Wikipedia data: http://code.google.com/p/graphdb-load-tester/ It includes what I believe to be a faster Lucene batch indexer service using a bloom filter and various other optimisations. However, the underlying Neo4J database doesn't seem to be able to cope with inserting these volumes of data on my available hardware and I don't have (and would hope not to need) 10s of gigabytes of RAM to throw at the problem. Cheers, Mark ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help with exception using BatchInserter
Hi David, there's another thread I started with the same name: Help with exception using BatchInserter. (*that's the one we are* *referring to)* I guess Mark started this one by mistake instead of replying to the other. Cheers, Pablo On Wed, Feb 16, 2011 at 6:55 PM, David Montag david.mon...@neotechnology.com wrote: Hi guys, I don't have the history of this thread. What issues are you running into? David On Wed, Feb 16, 2011 at 2:37 AM, Pablo Pareja ppar...@era7.com wrote: Hi Mark, Thanks for your contribution. I'm using a m2.4xlarge EC2 AWS instance type (*68.4 GB of memory, 26 EC2 Compute Units (8 virtual cores with 3.25 EC2 Compute Units each)).* Besides I'm parsing a 17G xml file for populating the DB and normally problems only arise when I have already inserted a decent amount of data (around 30.000.000 nodes and 50.000.000 relationships), that's why I was saying it'd be kind of harsh to reproduce the same situation. Right now I'm gonna try splitting the 17G xml file in at least 10-20 files and doing that number of separated batch insertions hoping it would work that way. I'll let you all know how it goes, Cheers, Pablo On Wed, Feb 16, 2011 at 11:12 AM, Mark Harwood markharw...@gmail.com wrote: Hi Pablo, Regarding the boiled down version of my code I guess I could prepare it but it's quite a big project Here's a boiled-down batch load demo I did earlier based on public Wikipedia data: http://code.google.com/p/graphdb-load-tester/ It includes what I believe to be a faster Lucene batch indexer service using a bloom filter and various other optimisations. However, the underlying Neo4J database doesn't seem to be able to cope with inserting these volumes of data on my available hardware and I don't have (and would hope not to need) 10s of gigabytes of RAM to throw at the problem. Cheers, Mark ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- David Montag Neo Technology, www.neotechnology.com Cell: 650.556.4411 david.mon...@neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Does BatchInserter auto-commit before shutting down?
Yeah, I'm inserting stuff into an index as you say and I always shutdown the BatchInserterIndexProvider prior to the inserter shutdown. I have to say that I'm a bit confused about all these exceptions and don't know what I could be doing wrong. I put another question regarding indexer and callings to optimize() method, which has been the reason for my last problem. Cheers Pablo On Tue, Feb 15, 2011 at 8:44 AM, Mattias Persson matt...@neotechnology.comwrote: It looks to me that you're inserting stuff into an index as well, http://wiki.neo4j.org/content/Index_Framework#Batch_insertion right? If you do that you'll have to call shutdown on that BatchInserterIndexProvider on conjunction with shutting down the BatchInserter. 2011/2/14 Pablo Pareja ppar...@era7.com Yeah, that's what I thought, I just wanted to confirm it. Thanks Pablo On Mon, Feb 14, 2011 at 4:57 PM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: BatchInserter.shutdown() is synchronous, it has to be, otherwise it wouldn't be safe at all. After shutdown() has returned it guarantees that all store files have been written, and are in a consistent state. -t On Mon, Feb 14, 2011 at 4:01 PM, Pablo Pareja ppar...@era7.com wrote: Great, thanks for the information. Just out of curiosity then, are calls to BatchInserter.shutdown() sync or asynchronous ? I mean, does the program flow stops in that call till every operation the shutdown implies has been performed? Cheers, Pablo On Mon, Feb 14, 2011 at 3:30 PM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: The batch inserter will write data to disk as needed. The memory mapped regions are used to speed up writes, so that the API level writes can be done to the memory mapped regions which can then be written in large sequential chunks to disk. You do not have to shut down the batch inserter for this to happen, it is taken care of automatically. Cheers, Tobias On Mon, Feb 14, 2011 at 10:53 AM, Pablo Pareja ppar...@era7.com wrote: Hi, I was wondering whether once the memory-mapping configuration for BatchInserter is full *(for example* *in the case of neostore.relationshipstore.db.mapped_memory) *the class would auto-commit these changes to disk or it would just throw an exception. In the latter case, what would the options be? In my case I have to insert a huge amount of data that cannot be allocated in RAM. First I thought the BatchInserter would deal with this kind of situation, however I keep getting exceptions whenever the program reaches the point where *(I guess) *memory mapping configuration is not enough. Since I cannot use more RAM than what I have, I thought, ok so I will shutdown both inserter and index service and create them again each time the program have already inserted a decent amount of data. Then, don't know why but after a couple of tests, at some point the program always halts throwing a really weird exception saying something about a index file that exists but cannot be accessed in the merge process ?!? I guess I'd be doing something wrong but cannot figure out what it is. Thanks in advance -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com
[Neo4j] Help with exception using BatchInserter
Hi all, I keep getting this exception and don't know what the reason could be: Unable to write record[244986398] @[32583190934] org.neo4j.kernel.impl.nioneo.store.AbstractPersistenceWindow.writeOut(AbstractPersistenceWindow.java:112) org.neo4j.kernel.impl.nioneo.store.AbstractPersistenceWindow.force(AbstractPersistenceWindow.java:124) org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.flushAll(PersistenceWindowPool.java:251) org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.close(PersistenceWindowPool.java:224) org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:614) org.neo4j.kernel.impl.nioneo.store.PropertyStore.closeStorage(PropertyStore.java:101) org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:611) org.neo4j.kernel.impl.nioneo.store.NeoStore.closeStorage(NeoStore.java:115) org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:611) org.neo4j.kernel.impl.batchinsert.BatchInserterImpl.shutdown(BatchInserterImpl.java:351) I'm using a 65G RAM machine and the java process is launched with -Xmx63G the conf values I pass to the batch inserter are the following: neostore.nodestore.db.mapped_memory=10G neostore.relationshipstore.db.mapped_memory=10G neostore.propertystore.db.mapped_memory=10G neostore.propertystore.db.strings.mapped_memory=10G neostore.propertystore.db.arrays.mapped_memory=0M Besides, there were still 50G free space in the disk when this exception was thrown I'd really appreciate any help, Cheers -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help with exception using BatchInserter
Ups... I thought memory mapping was using the Java heap space. Thanks Pablo On Mon, Feb 14, 2011 at 10:06 AM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: Your problem is likely related to the fact that 10G+10G+10G+10G + 63G 64G. Memory mapping uses memory outside of the Java heap. All memory you use, memory mapping plus Java heap must fit in the available RAM. Cheers, Tobias On Thu, Feb 10, 2011 at 10:42 PM, Pablo Pareja ppar...@era7.com wrote: Hi all, I keep getting this exception and don't know what the reason could be: Unable to write record[244986398] @[32583190934] org.neo4j.kernel.impl.nioneo.store.AbstractPersistenceWindow.writeOut(AbstractPersistenceWindow.java:112) org.neo4j.kernel.impl.nioneo.store.AbstractPersistenceWindow.force(AbstractPersistenceWindow.java:124) org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.flushAll(PersistenceWindowPool.java:251) org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.close(PersistenceWindowPool.java:224) org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:614) org.neo4j.kernel.impl.nioneo.store.PropertyStore.closeStorage(PropertyStore.java:101) org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:611) org.neo4j.kernel.impl.nioneo.store.NeoStore.closeStorage(NeoStore.java:115) org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.close(CommonAbstractStore.java:611) org.neo4j.kernel.impl.batchinsert.BatchInserterImpl.shutdown(BatchInserterImpl.java:351) I'm using a 65G RAM machine and the java process is launched with -Xmx63G the conf values I pass to the batch inserter are the following: neostore.nodestore.db.mapped_memory=10G neostore.relationshipstore.db.mapped_memory=10G neostore.propertystore.db.mapped_memory=10G neostore.propertystore.db.strings.mapped_memory=10G neostore.propertystore.db.arrays.mapped_memory=0M Besides, there were still 50G free space in the disk when this exception was thrown I'd really appreciate any help, Cheers -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Does BatchInserter auto-commit before shutting down?
Hi, I was wondering whether once the memory-mapping configuration for BatchInserter is full *(for example* *in the case of neostore.relationshipstore.db.mapped_memory) *the class would auto-commit these changes to disk or it would just throw an exception. In the latter case, what would the options be? In my case I have to insert a huge amount of data that cannot be allocated in RAM. First I thought the BatchInserter would deal with this kind of situation, however I keep getting exceptions whenever the program reaches the point where *(I guess) *memory mapping configuration is not enough. Since I cannot use more RAM than what I have, I thought, ok so I will shutdown both inserter and index service and create them again each time the program have already inserted a decent amount of data. Then, don't know why but after a couple of tests, at some point the program always halts throwing a really weird exception saying something about a index file that exists but cannot be accessed in the merge process ?!? I guess I'd be doing something wrong but cannot figure out what it is. Thanks in advance -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Does BatchInserter auto-commit before shutting down?
Great, thanks for the information. Just out of curiosity then, are calls to BatchInserter.shutdown() sync or asynchronous ? I mean, does the program flow stops in that call till every operation the shutdown implies has been performed? Cheers, Pablo On Mon, Feb 14, 2011 at 3:30 PM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: The batch inserter will write data to disk as needed. The memory mapped regions are used to speed up writes, so that the API level writes can be done to the memory mapped regions which can then be written in large sequential chunks to disk. You do not have to shut down the batch inserter for this to happen, it is taken care of automatically. Cheers, Tobias On Mon, Feb 14, 2011 at 10:53 AM, Pablo Pareja ppar...@era7.com wrote: Hi, I was wondering whether once the memory-mapping configuration for BatchInserter is full *(for example* *in the case of neostore.relationshipstore.db.mapped_memory) *the class would auto-commit these changes to disk or it would just throw an exception. In the latter case, what would the options be? In my case I have to insert a huge amount of data that cannot be allocated in RAM. First I thought the BatchInserter would deal with this kind of situation, however I keep getting exceptions whenever the program reaches the point where *(I guess) *memory mapping configuration is not enough. Since I cannot use more RAM than what I have, I thought, ok so I will shutdown both inserter and index service and create them again each time the program have already inserted a decent amount of data. Then, don't know why but after a couple of tests, at some point the program always halts throwing a really weird exception saying something about a index file that exists but cannot be accessed in the merge process ?!? I guess I'd be doing something wrong but cannot figure out what it is. Thanks in advance -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Exception when calling to optimize() method
I just got an exception when calling to indexService.optimize(); java.io.IOException: background merge hit exception: _u6:c10863150 _u7:c2-_u7 into _u8 [optimize] [mergeDocStores] neo4j.index.lucene.LuceneIndexBatchInserterImpl.optimize(LuceneIndexBatchInserterImpl.java:273) any ideas? I'm calling the optimize() method once per ~ 30.000 indexations, is it that too much/few ? are you supposed to call this method always between an indexation and a query on that index? Cheers, Pablo -- Pablo Pareja Tobes LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship indexing
Yes, I'm running embedded.
On Tue, Feb 1, 2011 at 1:38 PM, Michael Hunger
michael.hun...@neotechnology.com wrote:
I assume Pablo is running embedded.
Is that correct?
Michael
Am 01.02.2011 um 13:21 schrieb Peter Neubauer:
Also,
have you taken a look at the returned document? Transferring thousands
of relationships and parsing them as JSON on both client and server
might not be optimal.
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph
database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Tue, Feb 1, 2011 at 1:17 PM, Michael Hunger
michael.hun...@neotechnology.com wrote:
It sounds as if you are looping through all of them? Have they all the
same relationshiptype and direction?
Otherwise perhaps use node.getRelationships(type, direction)
Perhaps you can also use a traverser for accessing the relationship?
Cheers
Michael
Am 01.02.2011 um 12:37 schrieb Mattias Persson:
2011/2/1 Pablo Pareja ppar...@era7.com
Hi all,
I'm working on a project with millions of nodes and relationships and
I'm
using Neo4j for it.
I'm doing some tests and I'm surprised of how slow it gets when it
comes to
getting specific relationships
from one node...
How's it actually implemented?
The node I want to get the relationships from is indexed so getting
there
is
almost instantaneous, however
looping over its outgoing relationships seems to be really slow.
In the last test I just did it took about 10 minutes for retrieving
around
45.000 outgoing relationships.
What kind of heap settings and memory mapping configuration are you
using?
This sounds like disk swapping IMHO. 45000 relationships are quite few
and
should take well below a second to get. But if you just want to get a
specific relationship (based on a property value or other node or
something
similar) then indexing would be a faster solution. Maybe something like
this:
RelationshipIndex index = graphDb.index().forRelationships(
my-relationships );
...
Relationship myRelationship = myNode.createRelationship( myOtherNode,
myRelationshipType );
index.add( myRelationship, myKey, myValue );
...
Relationship relForMyKey = index.get( myKey, myValue
).getSingle();
for ( Relationships relBetweenMyNodes : index.get( null, null, myNode,
myOtherNode ) ) {
System.out.println( Relationship + relBetweenMyNodes +
found );
}
Does that help you?
I've also been checking the wiki site for the Index Framework and
there's
only one small paragraph talking about
indexing relationships but not about how to create them
RelationshipIndex friendships = graphDb.index().forRelationships(
friendships );
// type isn't a reserved key and isn't indexed automatically
Relationship relationship = friendships.get( type, knows,
morpheus,
trinity ).getSingle();
I'll try to update the wiki page with a little more/better information!
so do you have any idea of the proper way for indexing relationships?
Thanks in advance,
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Pablo Pareja Tobes
LinkedInhttp://www.linkedin.com/in/pabloparejatobes
Twitter http://www.twitter.com/pablopareja
http://www.ohnosequences.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship indexing
First of all thanks for your quick response,
I'm using BatchInserter for the creation of all nodes and relationships, I
pass a conf file with these
parameters to the inserter:
neostore.nodestore.db.mapped_memory=10G
neostore.relationshipstore.db.mapped_memory=15G
neostore.propertystore.db.mapped_memory=10G
neostore.propertystore.db.strings.mapped_memory=15G
neostore.propertystore.db.arrays.mapped_memory=0M
for a 64 GB of RAM computer.
However, I have to say that I don't pass any configuration file when
creating the GraphDatabaseService.
I just pass the db folder whenever I have to create a GraphDatabaseService
instance in order to perform any kind of query. (I'm not using the new neo4j
server feature, I navigate through the graph manually).
This is the code I use for retrieving the relationships:
IteratorRelationship relIterator =
node.getRelationships(SomeRelationship,Direction.INCOMING).iterator();
while(interproRelIterator.hasNext()){
Wrapper wrapper =
new Wrapper(interproRelIterator.next().getStartNode());
}
The current size of my db is somewhere around 35.000.000 nodes and
20.000.000 relationships.
Thanks in advance,
On Tue, Feb 1, 2011 at 12:37 PM, Mattias Persson
matt...@neotechnology.comwrote:
2011/2/1 Pablo Pareja ppar...@era7.com
Hi all,
I'm working on a project with millions of nodes and relationships and I'm
using Neo4j for it.
I'm doing some tests and I'm surprised of how slow it gets when it comes
to
getting specific relationships
from one node...
How's it actually implemented?
The node I want to get the relationships from is indexed so getting there
is
almost instantaneous, however
looping over its outgoing relationships seems to be really slow.
In the last test I just did it took about 10 minutes for retrieving
around
45.000 outgoing relationships.
What kind of heap settings and memory mapping configuration are you using?
This sounds like disk swapping IMHO. 45000 relationships are quite few and
should take well below a second to get. But if you just want to get a
specific relationship (based on a property value or other node or something
similar) then indexing would be a faster solution. Maybe something like
this:
RelationshipIndex index = graphDb.index().forRelationships(
my-relationships );
...
Relationship myRelationship = myNode.createRelationship( myOtherNode,
myRelationshipType );
index.add( myRelationship, myKey, myValue );
...
Relationship relForMyKey = index.get( myKey, myValue ).getSingle();
for ( Relationships relBetweenMyNodes : index.get( null, null, myNode,
myOtherNode ) ) {
System.out.println( Relationship + relBetweenMyNodes + found );
}
Does that help you?
I've also been checking the wiki site for the Index Framework and there's
only one small paragraph talking about
indexing relationships but not about how to create them
RelationshipIndex friendships = graphDb.index().forRelationships(
friendships );
// type isn't a reserved key and isn't indexed automatically
Relationship relationship = friendships.get( type, knows, morpheus,
trinity ).getSingle();
I'll try to update the wiki page with a little more/better information!
so do you have any idea of the proper way for indexing relationships?
Thanks in advance,
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Pablo Pareja Tobes
LinkedInhttp://www.linkedin.com/in/pabloparejatobes
Twitter http://www.twitter.com/pablopareja
http://www.ohnosequences.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationship indexing
Maybe I'm wrong but what you say would just apply for server feature uses
right?
(which is not my case)
Cheers,
On Tue, Feb 1, 2011 at 1:21 PM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Also,
have you taken a look at the returned document? Transferring thousands
of relationships and parsing them as JSON on both client and server
might not be optimal.
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Tue, Feb 1, 2011 at 1:17 PM, Michael Hunger
michael.hun...@neotechnology.com wrote:
It sounds as if you are looping through all of them? Have they all the
same relationshiptype and direction?
Otherwise perhaps use node.getRelationships(type, direction)
Perhaps you can also use a traverser for accessing the relationship?
Cheers
Michael
Am 01.02.2011 um 12:37 schrieb Mattias Persson:
2011/2/1 Pablo Pareja ppar...@era7.com
Hi all,
I'm working on a project with millions of nodes and relationships and
I'm
using Neo4j for it.
I'm doing some tests and I'm surprised of how slow it gets when it
comes to
getting specific relationships
from one node...
How's it actually implemented?
The node I want to get the relationships from is indexed so getting
there
is
almost instantaneous, however
looping over its outgoing relationships seems to be really slow.
In the last test I just did it took about 10 minutes for retrieving
around
45.000 outgoing relationships.
What kind of heap settings and memory mapping configuration are you
using?
This sounds like disk swapping IMHO. 45000 relationships are quite few
and
should take well below a second to get. But if you just want to get a
specific relationship (based on a property value or other node or
something
similar) then indexing would be a faster solution. Maybe something like
this:
RelationshipIndex index = graphDb.index().forRelationships(
my-relationships );
...
Relationship myRelationship = myNode.createRelationship( myOtherNode,
myRelationshipType );
index.add( myRelationship, myKey, myValue );
...
Relationship relForMyKey = index.get( myKey, myValue ).getSingle();
for ( Relationships relBetweenMyNodes : index.get( null, null, myNode,
myOtherNode ) ) {
System.out.println( Relationship + relBetweenMyNodes + found
);
}
Does that help you?
I've also been checking the wiki site for the Index Framework and
there's
only one small paragraph talking about
indexing relationships but not about how to create them
RelationshipIndex friendships = graphDb.index().forRelationships(
friendships );
// type isn't a reserved key and isn't indexed automatically
Relationship relationship = friendships.get( type, knows, morpheus,
trinity ).getSingle();
I'll try to update the wiki page with a little more/better information!
so do you have any idea of the proper way for indexing relationships?
Thanks in advance,
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Pablo Pareja Tobes
LinkedInhttp://www.linkedin.com/in/pabloparejatobes
Twitter http://www.twitter.com/pablopareja
http://www.ohnosequences.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Relationship indexing
Hi all, I'm working on a project with millions of nodes and relationships and I'm using Neo4j for it. I'm doing some tests and I'm surprised of how slow it gets when it comes to getting specific relationships from one node... How's it actually implemented? The node I want to get the relationships from is indexed so getting there is almost instantaneous, however looping over its outgoing relationships seems to be really slow. In the last test I just did it took about 10 minutes for retrieving around 45.000 outgoing relationships. I've also been checking the wiki site for the Index Framework and there's only one small paragraph talking about indexing relationships but not about how to create them RelationshipIndex friendships = graphDb.index().forRelationships( friendships ); // type isn't a reserved key and isn't indexed automatically Relationship relationship = friendships.get( type, knows, morpheus, trinity ).getSingle(); so do you have any idea of the proper way for indexing relationships? Thanks in advance, ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
