Re: [Neo4j] zero fromDepth and toDepth
Right, the toDepth implementation isn't matching the javadoc and it is a bit confusing. 2011/11/3 Alex a...@auv.name Done: http://neo4jdb.lighthouseapp.com/projects/77609-neo4j-community/tickets/17-consisnte-behavior-of-fromdepth-todepth-and-atdepth there's a typo in the title... time to get some sleep :) Alex -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/zero-fromDepth-and-toDepth-tp3474825p3476080.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Debian/Ubuntu packages for Neo4j?
Excellent - I'll jump in the repo and have a look! Nige On 3 November 2011 02:07, Peter Neubauer peter.neuba...@neotechnology.comwrote: There is the work Jake has been doing at https://github.com/jakewins/neo4j-server-debian . I would love if you could fork and update? Anders has made first attempts to set up a launchpad repo, not sure what the state there is. Welcome to chime in! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Wed, Nov 2, 2011 at 4:40 PM, Nigel Small ni...@nigelsmall.name wrote: Quick question: has anyone done any work on putting together .deb packages for Neo? And if so, has there been any consideration about getting them into Debian or Ubuntu repositories? Nige ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
Even without the synchronized ( _lock ) block your transactions would be
serialized as each transaction would have to grab a lock on that factory
node. So I propose you change that method implementation to:
private long generateId()
{
// This is a workaround for grabbing a write lock explicitly
_factoryNode.removeProperty(
non-existent-property-just-for-locking );
Long id;
if (_factoryNode.hasProperty(idseq))
{
id = (Long) _factoryNode.getProperty(idseq);
}
else
{
id = 1L;
}
_factoryNode.setProperty(idseq, id+1);
return id;
}
2011/11/2 Cres cre...@gmail.com
Hi David,
Thank you very much for your response. I can see now what caused the
deadlock.
However, I'm not really sure how I can solve this problem efficiently - as
you mentioned, your proposed solution will effectively serialize my
transactions. Unfortunately, it'd be very difficult for me to split each
transaction into two in the application I'm currently working on (the code
I
posted in the first message was just a sample code I wrote to explain the
problem more clearly) since there are many transactions which are higher
up
the chain and are not really nearby the node factory, which is where I
want
to use your lock-grabbing technique.
I thought about creating a nested transaction just for the update for the
factory node's idseq property, after which I'd commit (and thereby
release
the RW lock I grabbed when removing the non-existing property), but I now
realize it can't work because nested transactions aren't pure nested and
so the lock would remain until the parent transaction commits...
Is there any way I could possibly release the lock I have for the node
without committing the entire transaction? And since I presume such a way
doesn't exist, do you happen to know of any other possible solutions for my
problem?
Thanks again,
Ran.
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/Node-Id-generation-deadlock-tp3473118p3474747.html
Sent from the Neo4j Community Discussions mailing list archive at
Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Lucene uberfast indexing
Could be interesting, I'll keep an eye on when that goes into a released Lucene version! 2011/11/1 Rick Bullotta rick.bullo...@thingworx.com Hi, Neo team. ** ** Have you looked into the work Mike McCandless was doing with DocumentsWriterPerThread? It seems at first glance to fit in nicely with the Neo transaction isolation model and could have a significantly positive effect on (already fast) performance. ** ** http://www.searchworkings.org/blog/-/blogs/gimme-all-resources-you-have-i-can-use-them!/ ** ** Thoughts? ** ** Rick ** ** ** ** -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
This solution would have been ok if I had only one node created from that factory in each transaction. however, as shown in the sample code I posted in the original message, I have multiple nodes created in one transaction, and multiple such transactions in multiple threads. So while the creation of the actual nodes will of course be serialized, one thread's transaction will have to wait for the other thread's transaction to finish completely, and so if the first thread has some processing to do between the creation of the first and second nodes, the other third won't be able to create its two nodes in the meanwhile, because the first thread will have the lock on the factory node until the entire transaction completes. I'm looking for a way to do this having the nodes creation serialized but without having the entire transactions serialized, possibly by somehow releasing the lock on the factory node in mid-transaction, or by any other method. Thanks again, Ran. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Node-Id-generation-deadlock-tp3473118p3476498.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
On Nov 2, 2011, at 13:33 , Balazs E. Pataki wrote: Hi, I had a similar issue (also with ID generation), and I would be also interested in a solution, or how synchronizations should be done to avoid deadlocks like this in the transaction. Have you considered using IDs that can be generated without consulting the database, or even in-VM synchronization? E.g. UUID, GUID or VMID. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationships stored order
2011/10/31 Dmitriy Shabanov shaban...@gmail.com Hi, Berkeley Db index looks better for that. It was updated to serve our needs, but better performance required. Mattias, do you know low storage structure? Maybe, it'll possible to use some structural conditions to restore relationships order. At current design we have two writes for one relationship, that always 2 times slow than one =) Neo4j doesn't have this type of ordering on the store level, and my best guess would be some kind of index. If the index is just used for ordering and there's a performance problem maybe you could stack up relationships and batch index them at a later time? On Mon, Oct 31, 2011 at 6:45 PM, Mattias Persson matt...@neotechnology.comwrote: You can use what you wrote and then use lucene numeric range query to do the trick: IndexRelationship index = ... index.add( rel2, ORDER, ValueContext.numeric(2) ); index.add( rel3, ORDER, ValueContext.numeric(3) ); index.add( rel1, ORDER, ValueContext.numeric(1) ); // For all relationships index.query( new QueryContext( NumericRangeQuery.newIntRange( ORDER,null,null,true,true ) ).sortNumeric( ORDER, false ) ); // For a range index.query( new QueryContext( NumericRangeQuery.newIntRange( ORDER,2,10,true,true ) ).sortNumeric( ORDER, false ) ); -- Dmitriy Shabanov ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
Hi, there are times when a random ID would be OK, but I (and as I can see Ran) needed a sequentially generated ID (or sequence number). In this case, you have to persist the last ID in the DB and read-increment-store when the next ID is to be generated. But this multihreaded issue could come up in other use cases as well, so it is nice to know there's a Neo4j synchronization way to do this properly without causing a deadlock. --- balazs On 11/3/11 9:57 AM, Tuure Laurinolli wrote: On Nov 2, 2011, at 13:33 , Balazs E. Pataki wrote: Hi, I had a similar issue (also with ID generation), and I would be also interested in a solution, or how synchronizations should be done to avoid deadlocks like this in the transaction. Have you considered using IDs that can be generated without consulting the database, or even in-VM synchronization? E.g. UUID, GUID or VMID. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
2011/11/3 Cres cre...@gmail.com This solution would have been ok if I had only one node created from that factory in each transaction. It doesn't matter... after factory.setProperty is run that transaction has got a write lock on that factory node which is held until the transaction committs. The benefit you get from my proposal would be that you make sure you read the correct value. however, as shown in the sample code I posted in the original message, I have multiple nodes created in one transaction, and multiple such transactions in multiple threads. So while the creation of the actual nodes will of course be serialized, one thread's transaction will have to wait for the other thread's transaction to finish completely, and so if the first thread has some processing to do between the creation of the first and second nodes, the other third won't be able to create its two nodes in the meanwhile, because the first thread will have the lock on the factory node until the entire transaction completes. I'm looking for a way to do this having the nodes creation serialized but without having the entire transactions serialized, possibly by somehow releasing the lock on the factory node in mid-transaction, or by any other method. Thanks again, Ran. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Node-Id-generation-deadlock-tp3473118p3476498.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4j Heroku addons - specifications
Hi everyone, I hope to be able to use Neo4j in production soon in the form of the Heroku addon. I'm not a beta tester so I hoped someone could tell me whether this version includes some of the features I see locally. - Does is have the Gremlin extension/console? So does it allow to write Gremlin queries? - Does it have the data browser? - The examples of the extensions are in JRuby, would other JVM languages also work out nicely? And also important, is there already a price range mentioned? Thanks in advance. Regards, Jeroen ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Heroku addons - specifications
Am 03.11.2011 um 10:14 schrieb Jeroen van Dijk: Hi everyone, I hope to be able to use Neo4j in production soon in the form of the Heroku addon. I'm not a beta tester so I hoped someone could tell me whether this version includes some of the features I see locally. - Does is have the Gremlin extension/console? So does it allow to write Gremlin queries? - yes, both the console in the web-admin as well as the gremlin plugin are deployed - same for cypher You can access the web-admin and REST-URLS from your heroku-add-on UI. - Does it have the data browser? - yes - The examples of the extensions are in JRuby, would other JVM languages also work out nicely? - right now we only have (j)ruby, we're working on a generic dynamic languages extension mechanism And also important, is there already a price range mentioned? Thanks in advance. Regards, Jeroen ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher: ommitting relationship if not available
On Wed, Nov 2, 2011 at 11:54 AM, D. Frej dieter_f...@gmx.net wrote: my use case is the following: - a principal in a group has access to all other principals in a group = therefore I need to go one level up to find the other principals in the group - additionally: if there is another group as principal in a group (UserGroup 1 is in UserGroup 2) , then the principal also has access to the principals of the sub-group (User 3 has access to User 1 and User 2 which are in sub-group UserGroup 1) = that is the second part in the MATCH clause Your idea looks like it is working: MATCH (principals)-[:IS_MEMBER_OF*0..1]- Thanks. Unfortunately, this doesn't work either. This will output two subgraphs - one where there is one relationships from principals up, and one where there is zero relationships. What we would like to be able to say is as many as possible, and nothing less. There's no way of expressing that today, unfortunately. Trees are not well handled by Cypher today, and they should be... Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
Mattias: I see your point, and indeed if it weren't for the deadlock, the transactions would have gotten serialized anyway, even without taking the lock by removing a fake property, since we do setProperty. However this only strengthens the question, as it further provides the need to somehow get rid of that lock in mid-transaction (or by using non-flat nested transactions, which aren't supported at the moment). Tuure: We have considered it, but as Balazs mentioned, there are times when those solutions don't fit, and unfortunately I'm looking for a different solution at the moment. Thank you, Ran. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Node-Id-generation-deadlock-tp3473118p3476683.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Python embedding: problems with shutdown and threads
On Thu, Nov 3, 2011 at 6:55 AM, Tim McNamara paperl...@timmcnamara.co.nzwrote: Here are a few notes of my own from using Neo4j with Python embedded. I know that this isn't the right place . Ubuntu doesn't set $JAVA_HOME. I ended up hard coding the location of my JVM. I seem to encounter unclean shutdowns all the time when running with the iPython REPL. Is there a db.shutdown() command that I've been neglecting to use? Yeah, as long as you call db.shutdown(), things should be ok. See here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html We should perhaps set up the db to shut itself down properly automatically if it detects that python is shutting down.. We decided against that before, since we felt like we don't own the application lifecycle. WDYT? Tim On 3 November 2011 17:28, Michael Rene Armida m...@marmida.com wrote: I'm having some weird problems with threading on shutdown. As I previously discovered, using neo4j in a multi-threaded Python app requires complete segregation of neo4j into a single thread. That all works fine for creating a database, however, it appears that shutting one down is a bit of a mystery. I wrote a test program to minimally illustrate the difficulty: http://pastebin.com/gg204kae Threading is always easy to get wrong, so I added a simple switch to remove neo4j from the picture. When I disable neo, the program starts up and shuts down cleanly. When I re-enable neo, some of the print statements never get run, and the app mysteriously exits. I am testing this by saving it into a script and running it on the command line. Any time after the worker thread finishes creating the neo db, I hit ctrl-c, and the two scenarios play out as above. It seems that the embedded Python API is not exactly at the forefront of popular use, and so I understand if nobody knows what's up with this. I just thought I should add something, for the record. I hope my script doesn't have an obvious bug. Thanks, Michael I had a discussion with Tobias this morning about threading with JPype, and am currently writing a fix that will allow full threading support for neo4j-embedded. I'll make sure to run this script as well after that fix is implemented, to make sure it gets fixed as well. As far as the forefront of use goes, there is a surprising amount of pypy downloads of neo4j-embedded (4000 new installs in the last 30 days), so the feedback you are providing here is gonna end up helping a lot of people :) Thank you so much for taking the time to document the issues you are having. /jake ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
Hi, I've also been wondering about this subject. According to the Neo4J design guide ( http://wiki.neo4j.org/content/Design_Guide) the factory node and id generator patterns are the way to implement sequential ID generation. However, according to this thread, it sounds like in a multi-threaded environment I have one of two choices: 1. lock the factory node. However this will force transaction serialization. That's not practical in my app. 2. reducing the granularity of the transactions - in the sense that each transaction should only contain one node creation (of the same type, i.e. uses the same factory node). That's not practical for me either, because in several cases I would like to create more than one node in the same transaction. Since the design guide recommends the factory node pattern, I'm wondering if there's anything I'm missing here, or that I should just avoid this pattern and use some other ID generation mechanism. Thanks, Yaniv On Thu, Nov 3, 2011 at 11:13 AM, Mattias Persson matt...@neotechnology.comwrote: 2011/11/3 Cres cre...@gmail.com This solution would have been ok if I had only one node created from that factory in each transaction. It doesn't matter... after factory.setProperty is run that transaction has got a write lock on that factory node which is held until the transaction committs. The benefit you get from my proposal would be that you make sure you read the correct value. however, as shown in the sample code I posted in the original message, I have multiple nodes created in one transaction, and multiple such transactions in multiple threads. So while the creation of the actual nodes will of course be serialized, one thread's transaction will have to wait for the other thread's transaction to finish completely, and so if the first thread has some processing to do between the creation of the first and second nodes, the other third won't be able to create its two nodes in the meanwhile, because the first thread will have the lock on the factory node until the entire transaction completes. I'm looking for a way to do this having the nodes creation serialized but without having the entire transactions serialized, possibly by somehow releasing the lock on the factory node in mid-transaction, or by any other method. Thanks again, Ran. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Node-Id-generation-deadlock-tp3473118p3476498.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST api get list of all relationship properties (unique)
Hi Peter and Community,
I played a bit with passing a groovy script to neo4j. However the response
seems to be not JSON encoded, did I overlook something?
I'm using 1.5M02 and did the following request (from tcpmon)
http request:
POST /db/data/ext/GremlinPlugin/graphdb/execute_script HTTP/1.1
Content-Type: application/json
post body:
{script:neo4j = g.getRawGraph()}
The result seems to be not JSON encoded and different from the
documentation:
..
Content-Type: application/json
EmbeddedGraphDatabase [...neo4j\\neo4j-community-1.5.M02\\data\\graph.db]
Shouldn't this be a JSON string? Later on grails complains with an
groovyx.net.http.ResponseParseException which is correct, since it's not
JSON, right?
Thanks, Jure
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/REST-api-get-list-of-all-relationship-properties-unique-tp3458852p3476872.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Python embedding: problems with shutdown and threads
That's awesome news. I will check it out when it's ready. I would suggest wrapping up the lifecycle management in a context manager. You're already using the GraphDatabase object's __enter__ and __exit__ to manage transactions, so you'd probably have to add a second object. Would you like me to add a bug somewhere? I could turn that script into a unit test, which should make it easier to evaluate. On Thu, Nov 3, 2011 at 3:08 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: On Thu, Nov 3, 2011 at 6:55 AM, Tim McNamara paperl...@timmcnamara.co.nz wrote: Here are a few notes of my own from using Neo4j with Python embedded. I know that this isn't the right place . Ubuntu doesn't set $JAVA_HOME. I ended up hard coding the location of my JVM. I seem to encounter unclean shutdowns all the time when running with the iPython REPL. Is there a db.shutdown() command that I've been neglecting to use? Yeah, as long as you call db.shutdown(), things should be ok. See here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html We should perhaps set up the db to shut itself down properly automatically if it detects that python is shutting down.. We decided against that before, since we felt like we don't own the application lifecycle. WDYT? Tim On 3 November 2011 17:28, Michael Rene Armida m...@marmida.com wrote: I'm having some weird problems with threading on shutdown. As I previously discovered, using neo4j in a multi-threaded Python app requires complete segregation of neo4j into a single thread. That all works fine for creating a database, however, it appears that shutting one down is a bit of a mystery. I wrote a test program to minimally illustrate the difficulty: http://pastebin.com/gg204kae Threading is always easy to get wrong, so I added a simple switch to remove neo4j from the picture. When I disable neo, the program starts up and shuts down cleanly. When I re-enable neo, some of the print statements never get run, and the app mysteriously exits. I am testing this by saving it into a script and running it on the command line. Any time after the worker thread finishes creating the neo db, I hit ctrl-c, and the two scenarios play out as above. It seems that the embedded Python API is not exactly at the forefront of popular use, and so I understand if nobody knows what's up with this. I just thought I should add something, for the record. I hope my script doesn't have an obvious bug. Thanks, Michael I had a discussion with Tobias this morning about threading with JPype, and am currently writing a fix that will allow full threading support for neo4j-embedded. I'll make sure to run this script as well after that fix is implemented, to make sure it gets fixed as well. As far as the forefront of use goes, there is a surprising amount of pypy downloads of neo4j-embedded (4000 new installs in the last 30 days), so the feedback you are providing here is gonna end up helping a lot of people :) Thank you so much for taking the time to document the issues you are having. /jake ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Python embedding: problems with shutdown and threads
On Thu, Nov 3, 2011 at 3:24 PM, Michael Rene Armida m...@marmida.com wrote: That's awesome news. I will check it out when it's ready. I would suggest wrapping up the lifecycle management in a context manager. You're already using the GraphDatabase object's __enter__ and __exit__ to manage transactions, so you'd probably have to add a second object. Would you like me to add a bug somewhere? I could turn that script into a unit test, which should make it easier to evaluate. It's fine as it is for now, I've written a unit test that checks that threading works (which it seems to do, although there are still a few issues), and I'll add your script to that test suite as well once I get this to pass. Unless I hit any snags, should be pushed later today or tomorrow. There is no need for it now, but if you report bugs in the future, there is a bug tracker active for the project on github. Reporting bugs there is probably preferable to reporting them here, since that makes it easy to keep track of and prioritize work. https://github.com/neo4j/python-embedded /jake On Thu, Nov 3, 2011 at 3:08 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: On Thu, Nov 3, 2011 at 6:55 AM, Tim McNamara paperl...@timmcnamara.co.nz wrote: Here are a few notes of my own from using Neo4j with Python embedded. I know that this isn't the right place . Ubuntu doesn't set $JAVA_HOME. I ended up hard coding the location of my JVM. I seem to encounter unclean shutdowns all the time when running with the iPython REPL. Is there a db.shutdown() command that I've been neglecting to use? Yeah, as long as you call db.shutdown(), things should be ok. See here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html We should perhaps set up the db to shut itself down properly automatically if it detects that python is shutting down.. We decided against that before, since we felt like we don't own the application lifecycle. WDYT? Tim On 3 November 2011 17:28, Michael Rene Armida m...@marmida.com wrote: I'm having some weird problems with threading on shutdown. As I previously discovered, using neo4j in a multi-threaded Python app requires complete segregation of neo4j into a single thread. That all works fine for creating a database, however, it appears that shutting one down is a bit of a mystery. I wrote a test program to minimally illustrate the difficulty: http://pastebin.com/gg204kae Threading is always easy to get wrong, so I added a simple switch to remove neo4j from the picture. When I disable neo, the program starts up and shuts down cleanly. When I re-enable neo, some of the print statements never get run, and the app mysteriously exits. I am testing this by saving it into a script and running it on the command line. Any time after the worker thread finishes creating the neo db, I hit ctrl-c, and the two scenarios play out as above. It seems that the embedded Python API is not exactly at the forefront of popular use, and so I understand if nobody knows what's up with this. I just thought I should add something, for the record. I hope my script doesn't have an obvious bug. Thanks, Michael I had a discussion with Tobias this morning about threading with JPype, and am currently writing a fix that will allow full threading support for neo4j-embedded. I'll make sure to run this script as well after that fix is implemented, to make sure it gets fixed as well. As far as the forefront of use goes, there is a surprising amount of pypy downloads of neo4j-embedded (4000 new installs in the last 30 days), so the feedback you are providing here is gonna end up helping a lot of people :) Thank you so much for taking the time to document the issues you are having. /jake ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST api get list of all relationship properties (unique)
Jure,
if the primitives in the response are not recognized (like a normal
Groovy Object), then the toString() of the result is returned. If you
want, you can fork and add to the representations, see
https://github.com/neo4j/gremlin-plugin/blob/master/src/main/java/org/neo4j/server/plugin/gremlin/GremlinToRepresentationConverter.java
- there we probably should add GraphDatabaseService as one possible
return and convert it to the normal representation you see at
/db/data/ .
You up for it?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
On Thu, Nov 3, 2011 at 4:42 AM, Jure Zakotnik j...@gmx.de wrote:
Hi Peter and Community,
I played a bit with passing a groovy script to neo4j. However the response
seems to be not JSON encoded, did I overlook something?
I'm using 1.5M02 and did the following request (from tcpmon)
http request:
POST /db/data/ext/GremlinPlugin/graphdb/execute_script HTTP/1.1
Content-Type: application/json
post body:
{script:neo4j = g.getRawGraph()}
The result seems to be not JSON encoded and different from the
documentation:
..
Content-Type: application/json
EmbeddedGraphDatabase [...neo4j\\neo4j-community-1.5.M02\\data\\graph.db]
Shouldn't this be a JSON string? Later on grails complains with an
groovyx.net.http.ResponseParseException which is correct, since it's not
JSON, right?
Thanks, Jure
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/REST-api-get-list-of-all-relationship-properties-unique-tp3458852p3476872.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST api get list of all relationship properties (unique)
Shouldn't it be enough to call toString() on your results if they are not valid
representations, which are for a plugin:
* Primitives
* Node
* Relationship
* Path
* Iterables or Collections thereof
* probably also Map but I'm not sure.
please try:
{script:neo4j = g.getRawGraph().toString()}
Michael
Am 03.11.2011 um 15:59 schrieb Peter Neubauer:
Jure,
if the primitives in the response are not recognized (like a normal
Groovy Object), then the toString() of the result is returned. If you
want, you can fork and add to the representations, see
https://github.com/neo4j/gremlin-plugin/blob/master/src/main/java/org/neo4j/server/plugin/gremlin/GremlinToRepresentationConverter.java
- there we probably should add GraphDatabaseService as one possible
return and convert it to the normal representation you see at
/db/data/ .
You up for it?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
On Thu, Nov 3, 2011 at 4:42 AM, Jure Zakotnik j...@gmx.de wrote:
Hi Peter and Community,
I played a bit with passing a groovy script to neo4j. However the response
seems to be not JSON encoded, did I overlook something?
I'm using 1.5M02 and did the following request (from tcpmon)
http request:
POST /db/data/ext/GremlinPlugin/graphdb/execute_script HTTP/1.1
Content-Type: application/json
post body:
{script:neo4j = g.getRawGraph()}
The result seems to be not JSON encoded and different from the
documentation:
..
Content-Type: application/json
EmbeddedGraphDatabase [...neo4j\\neo4j-community-1.5.M02\\data\\graph.db]
Shouldn't this be a JSON string? Later on grails complains with an
groovyx.net.http.ResponseParseException which is correct, since it's not
JSON, right?
Thanks, Jure
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/REST-api-get-list-of-all-relationship-properties-unique-tp3458852p3476872.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] zero fromDepth and toDepth
So, do we fix the implementation to match (see https://github.com/neo4j/community/issues/80) or adjust the JavaDoc? I kinda think it makes sense to change the code, have written tests for it. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 12:21 AM, Mattias Persson matt...@neotechnology.com wrote: Right, the toDepth implementation isn't matching the javadoc and it is a bit confusing. 2011/11/3 Alex a...@auv.name Done: http://neo4jdb.lighthouseapp.com/projects/77609-neo4j-community/tickets/17-consisnte-behavior-of-fromdepth-todepth-and-atdepth there's a typo in the title... time to get some sleep :) Alex -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/zero-fromDepth-and-toDepth-tp3474825p3476080.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Debian/Ubuntu packages for Neo4j?
Hi! Excellent - I'll jump in the repo and have a look! IIRC that project downloads the release package from our mvn repo - but we don't deploy them there any more. So you'll have to modify that part. Regarding launchpad we have a team: https://launchpad.net/~neo4j and a project: https://launchpad.net/neo4j The team is currently me and Jake, you're welcome to join in! /anders Nige On 3 November 2011 02:07, Peter Neubauer peter.neuba...@neotechnology.comwrote: There is the work Jake has been doing at https://github.com/jakewins/neo4j-server-debian . I would love if you could fork and update? Anders has made first attempts to set up a launchpad repo, not sure what the state there is. Welcome to chime in! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Wed, Nov 2, 2011 at 4:40 PM, Nigel Smallni...@nigelsmall.name wrote: Quick question: has anyone done any work on putting together .deb packages for Neo? And if so, has there been any consideration about getting them into Debian or Ubuntu repositories? Nige ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Python embedding: problems with shutdown and threads
On 2011-11-03 11:08, Jacob Hansson wrote: On Thu, Nov 3, 2011 at 6:55 AM, Tim McNamara paperl...@timmcnamara.co.nzwrote: Here are a few notes of my own from using Neo4j with Python embedded. I know that this isn't the right place . Ubuntu doesn't set $JAVA_HOME. I ended up hard coding the location of my JVM. I seem to encounter unclean shutdowns all the time when running with the iPython REPL. Is there a db.shutdown() command that I've been neglecting to use? Yeah, as long as you call db.shutdown(), things should be ok. See here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html We should perhaps set up the db to shut itself down properly automatically if it detects that python is shutting down.. We decided against that before, since we felt like we don't own the application lifecycle. WDYT? I found this snippet to work great for my modules: def _close_db(): try: neo4jdb.shutdown() except NameError: print 'Could not shutdown Neo4j database. Is it open in \ another process?' import atexit atexit.register(_close_db) Tim On 3 November 2011 17:28, Michael Rene Armida m...@marmida.com wrote: I'm having some weird problems with threading on shutdown. As I previously discovered, using neo4j in a multi-threaded Python app requires complete segregation of neo4j into a single thread. That all works fine for creating a database, however, it appears that shutting one down is a bit of a mystery. I wrote a test program to minimally illustrate the difficulty: http://pastebin.com/gg204kae Threading is always easy to get wrong, so I added a simple switch to remove neo4j from the picture. When I disable neo, the program starts up and shuts down cleanly. When I re-enable neo, some of the print statements never get run, and the app mysteriously exits. I am testing this by saving it into a script and running it on the command line. Any time after the worker thread finishes creating the neo db, I hit ctrl-c, and the two scenarios play out as above. It seems that the embedded Python API is not exactly at the forefront of popular use, and so I understand if nobody knows what's up with this. I just thought I should add something, for the record. I hope my script doesn't have an obvious bug. Thanks, Michael I had a discussion with Tobias this morning about threading with JPype, and am currently writing a fix that will allow full threading support for neo4j-embedded. I'll make sure to run this script as well after that fix is implemented, to make sure it gets fixed as well. As far as the forefront of use goes, there is a surprising amount of pypy downloads of neo4j-embedded (4000 new installs in the last 30 days), so the feedback you are providing here is gonna end up helping a lot of people :) Thank you so much for taking the time to document the issues you are having. /jake ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Johan Lundberg NORDUnet A/S lundb...@nordu.net +46730714375 Tulegatan 11 113 53 Stockholm Sweden ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Using the WrappingNeoServerBootstrapper with sbt
Has anybody had success in getting an embedded neo4j-server running with sbt? I had some trouble in getting the dependencies but now it at least compiles and starts up. However the start ultimately failed. Before switching back to maven I wanted to ask whether someone has a working sbt setup and can share the build.sbt? BTW, the error I encounter at the moment (doesn't seem to be caused by a missing dependency): INFO: Starting Neo Server on port [7474] with [80] threads available Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: Logging to org.slf4j.impl.JDK14LoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Using database at /tmp/tmp-neo-test Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log WARNING: No database tuning properties set in the property file, using defaults. Please specify the performance properties file with org.neo4j.server.db.tuning.properties in the server properties file [null]. Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted discovery module at [/] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted REST API at [/db/data] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted management API at [/db/manage] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted webadmin at [/webadmin] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounting static content at [/webadmin] from [webadmin-html] Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: jetty-6.1.25 Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: NO JSP Support for /webadmin, did not find org.apache.jasper.servlet.JspServlet Nov 3, 2011 5:01:17 PM com.sun.jersey.api.core.PackagesResourceConfig init INFO: Scanning for root resource and provider classes in the packages: org.neo4j.server.webadmin.rest Nov 3, 2011 5:01:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses INFO: Root resource classes found: class org.neo4j.server.webadmin.rest.ConsoleService class org.neo4j.server.webadmin.rest.JmxService class org.neo4j.server.webadmin.rest.RootService class org.neo4j.server.webadmin.rest.MonitorService Nov 3, 2011 5:01:17 PM com.sun.jersey.api.core.ScanningResourceConfig init INFO: No provider classes found. Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog warn SEVERE: unavailable com.sun.jersey.api.container.ContainerException: No WebApplication provider is present at com.sun.jersey.spi.container.WebApplicationFactory.createWebApplication(WebApplicationFactory.java:69) at com.sun.jersey.spi.container.servlet.ServletContainer.create(ServletContainer.java:391) at com.sun.jersey.spi.container.servlet.ServletContainer$InternalWebComponent.create(ServletContainer.java:306) at com.sun.jersey.spi.container.servlet.WebComponent.load(WebComponent.java:607) at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:210) at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:373) at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:556) at javax.servlet.GenericServlet.init(GenericServlet.java:241) at org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440) at org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:685) at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) at org.mortbay.jetty.Server.doStart(Server.java:224) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.neo4j.server.web.Jetty6WebServer.startJetty(Jetty6WebServer.java:168) at org.neo4j.server.web.Jetty6WebServer.start(Jetty6WebServer.java:105) at org.neo4j.server.NeoServerWithEmbeddedWebServer.startWebServer(NeoServerWithEmbeddedWebServer.java:252) at org.neo4j.server.NeoServerWithEmbeddedWebServer.start(NeoServerWithEmbeddedWebServer.java:106) at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:87) at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:76) Regards, Markus ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Using the WrappingNeoServerBootstrapper with sbt
Markus, I just spent a number of afternoons getting into the guts of Ivy and Maven classifiers for the Dr. Who manual. To set up the server dependencies in Ivy (which SBT uses under the hood), refer to https://github.com/jimwebber/neo4j-tutorial/blob/master/settings/ivy.xml and https://github.com/jimwebber/neo4j-tutorial/blob/master/settings/ivysettings.xml which make Ivy aware of the fact that there are number of artifacts needed with the same group and artifact id but different classifiers. Does that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 9:03 AM, Markus Joschko markus.josc...@gmail.com wrote: Has anybody had success in getting an embedded neo4j-server running with sbt? I had some trouble in getting the dependencies but now it at least compiles and starts up. However the start ultimately failed. Before switching back to maven I wanted to ask whether someone has a working sbt setup and can share the build.sbt? BTW, the error I encounter at the moment (doesn't seem to be caused by a missing dependency): INFO: Starting Neo Server on port [7474] with [80] threads available Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: Logging to org.slf4j.impl.JDK14LoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Using database at /tmp/tmp-neo-test Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log WARNING: No database tuning properties set in the property file, using defaults. Please specify the performance properties file with org.neo4j.server.db.tuning.properties in the server properties file [null]. Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted discovery module at [/] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted REST API at [/db/data] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted management API at [/db/manage] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted webadmin at [/webadmin] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounting static content at [/webadmin] from [webadmin-html] Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: jetty-6.1.25 Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: NO JSP Support for /webadmin, did not find org.apache.jasper.servlet.JspServlet Nov 3, 2011 5:01:17 PM com.sun.jersey.api.core.PackagesResourceConfig init INFO: Scanning for root resource and provider classes in the packages: org.neo4j.server.webadmin.rest Nov 3, 2011 5:01:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses INFO: Root resource classes found: class org.neo4j.server.webadmin.rest.ConsoleService class org.neo4j.server.webadmin.rest.JmxService class org.neo4j.server.webadmin.rest.RootService class org.neo4j.server.webadmin.rest.MonitorService Nov 3, 2011 5:01:17 PM com.sun.jersey.api.core.ScanningResourceConfig init INFO: No provider classes found. Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog warn SEVERE: unavailable com.sun.jersey.api.container.ContainerException: No WebApplication provider is present at com.sun.jersey.spi.container.WebApplicationFactory.createWebApplication(WebApplicationFactory.java:69) at com.sun.jersey.spi.container.servlet.ServletContainer.create(ServletContainer.java:391) at com.sun.jersey.spi.container.servlet.ServletContainer$InternalWebComponent.create(ServletContainer.java:306) at com.sun.jersey.spi.container.servlet.WebComponent.load(WebComponent.java:607) at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:210) at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:373) at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:556) at javax.servlet.GenericServlet.init(GenericServlet.java:241) at org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440) at org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:685) at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at
Re: [Neo4j] REST api get list of all relationship properties (unique)
Or,
if we are not explicitely catching and converting the representations,
at the end of this switch we could hand it over to the server REST
subsystem in order to convert things. I will raised
https://github.com/neo4j/gremlin-plugin/issues/2
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
On Thu, Nov 3, 2011 at 8:12 AM, Michael Hunger
michael.hun...@neotechnology.com wrote:
Shouldn't it be enough to call toString() on your results if they are not
valid representations, which are for a plugin:
* Primitives
* Node
* Relationship
* Path
* Iterables or Collections thereof
* probably also Map but I'm not sure.
please try:
{script:neo4j = g.getRawGraph().toString()}
Michael
Am 03.11.2011 um 15:59 schrieb Peter Neubauer:
Jure,
if the primitives in the response are not recognized (like a normal
Groovy Object), then the toString() of the result is returned. If you
want, you can fork and add to the representations, see
https://github.com/neo4j/gremlin-plugin/blob/master/src/main/java/org/neo4j/server/plugin/gremlin/GremlinToRepresentationConverter.java
- there we probably should add GraphDatabaseService as one possible
return and convert it to the normal representation you see at
/db/data/ .
You up for it?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
On Thu, Nov 3, 2011 at 4:42 AM, Jure Zakotnik j...@gmx.de wrote:
Hi Peter and Community,
I played a bit with passing a groovy script to neo4j. However the response
seems to be not JSON encoded, did I overlook something?
I'm using 1.5M02 and did the following request (from tcpmon)
http request:
POST /db/data/ext/GremlinPlugin/graphdb/execute_script HTTP/1.1
Content-Type: application/json
post body:
{script:neo4j = g.getRawGraph()}
The result seems to be not JSON encoded and different from the
documentation:
..
Content-Type: application/json
EmbeddedGraphDatabase [...neo4j\\neo4j-community-1.5.M02\\data\\graph.db]
Shouldn't this be a JSON string? Later on grails complains with an
groovyx.net.http.ResponseParseException which is correct, since it's not
JSON, right?
Thanks, Jure
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/REST-api-get-list-of-all-relationship-properties-unique-tp3458852p3476872.html
Sent from the Neo4j Community Discussions mailing list archive at
Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Using the WrappingNeoServerBootstrapper with sbt
Hi Peter, I have seen your posts and the projects. I tried to mimic the setup as close as possible. However I also do my first steps in sbt which doesn't make it easier. My best guess is, that I am missing a jetty dependency. Does the web interface of the server need jsp support, e.g? These lines seem to be the most interesting regarding the error: INFO: NO JSP Support for /webadmin, did not find org.apache.jasper.servlet.JspServlet Nov 3, 2011 5:02:20 PM com.sun.jersey.api.core.ScanningResourceConfig init INFO: No provider classes found. ivyXML := dependencies dependency org=org.neo4j.app name=neo4j-server rev=1.5-SNAPSHOT artifact name=neo4j-server type=jar / artifact name=neo4j-server type=jar m:classifier=static-web/ /dependency dependency org=org.neo4j name=neo4j-community rev=1.5-SNAPSHOT / dependency org=org.neo4j name=neo4j-kernel rev=1.5-SNAPSHOT/ dependency org=org.codehaus.jackson name=jackson-core-asl rev=1.7.5 / dependency org=org.codehaus.jackson name=jackson-mapper-asl rev=1.7.5 / dependency org=org.mortbay.jetty name=jetty rev=6.1.25 / dependency org=org.rrd4j name=rrd4j rev=2.0.7 transitive=false / dependency org=com.sun.jersey name=jersey-client rev=1.6 / dependency org=com.sun.jersey name=jersey-core rev=1.6 / dependency org=com.sun.jersey name=jersey-server rev=1.6 / dependency org=org.aspectj name=aspectjrt rev=1.6.11/ dependency org=org.aspectj name=aspectjtools rev=1.6.11/ /dependencies On Thu, Nov 3, 2011 at 5:09 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Markus, I just spent a number of afternoons getting into the guts of Ivy and Maven classifiers for the Dr. Who manual. To set up the server dependencies in Ivy (which SBT uses under the hood), refer to https://github.com/jimwebber/neo4j-tutorial/blob/master/settings/ivy.xml and https://github.com/jimwebber/neo4j-tutorial/blob/master/settings/ivysettings.xml which make Ivy aware of the fact that there are number of artifacts needed with the same group and artifact id but different classifiers. Does that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 9:03 AM, Markus Joschko markus.josc...@gmail.com wrote: Has anybody had success in getting an embedded neo4j-server running with sbt? I had some trouble in getting the dependencies but now it at least compiles and starts up. However the start ultimately failed. Before switching back to maven I wanted to ask whether someone has a working sbt setup and can share the build.sbt? BTW, the error I encounter at the moment (doesn't seem to be caused by a missing dependency): INFO: Starting Neo Server on port [7474] with [80] threads available Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: Logging to org.slf4j.impl.JDK14LoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Using database at /tmp/tmp-neo-test Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log WARNING: No database tuning properties set in the property file, using defaults. Please specify the performance properties file with org.neo4j.server.db.tuning.properties in the server properties file [null]. Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted discovery module at [/] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted REST API at [/db/data] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted management API at [/db/manage] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted webadmin at [/webadmin] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounting static content at [/webadmin] from [webadmin-html] Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: jetty-6.1.25 Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: NO JSP Support for /webadmin, did not find org.apache.jasper.servlet.JspServlet Nov 3, 2011 5:01:17 PM com.sun.jersey.api.core.PackagesResourceConfig init INFO: Scanning for root resource and provider classes in the packages: org.neo4j.server.webadmin.rest Nov 3, 2011 5:01:17 PM com.sun.jersey.api.core.ScanningResourceConfig
Re: [Neo4j] Using the WrappingNeoServerBootstrapper with sbt
Markus, could you first try to get the tutorial up and running so you have a working baseline? SBT and Ivy is a quirky combination - not sure myself what is going on there at all times. From there, we could try working our way down ... Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 9:27 AM, Markus Joschko markus.josc...@gmail.com wrote: Hi Peter, I have seen your posts and the projects. I tried to mimic the setup as close as possible. However I also do my first steps in sbt which doesn't make it easier. My best guess is, that I am missing a jetty dependency. Does the web interface of the server need jsp support, e.g? These lines seem to be the most interesting regarding the error: INFO: NO JSP Support for /webadmin, did not find org.apache.jasper.servlet.JspServlet Nov 3, 2011 5:02:20 PM com.sun.jersey.api.core.ScanningResourceConfig init INFO: No provider classes found. ivyXML := dependencies dependency org=org.neo4j.app name=neo4j-server rev=1.5-SNAPSHOT artifact name=neo4j-server type=jar / artifact name=neo4j-server type=jar m:classifier=static-web/ /dependency dependency org=org.neo4j name=neo4j-community rev=1.5-SNAPSHOT / dependency org=org.neo4j name=neo4j-kernel rev=1.5-SNAPSHOT/ dependency org=org.codehaus.jackson name=jackson-core-asl rev=1.7.5 / dependency org=org.codehaus.jackson name=jackson-mapper-asl rev=1.7.5 / dependency org=org.mortbay.jetty name=jetty rev=6.1.25 / dependency org=org.rrd4j name=rrd4j rev=2.0.7 transitive=false / dependency org=com.sun.jersey name=jersey-client rev=1.6 / dependency org=com.sun.jersey name=jersey-core rev=1.6 / dependency org=com.sun.jersey name=jersey-server rev=1.6 / dependency org=org.aspectj name=aspectjrt rev=1.6.11/ dependency org=org.aspectj name=aspectjtools rev=1.6.11/ /dependencies On Thu, Nov 3, 2011 at 5:09 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Markus, I just spent a number of afternoons getting into the guts of Ivy and Maven classifiers for the Dr. Who manual. To set up the server dependencies in Ivy (which SBT uses under the hood), refer to https://github.com/jimwebber/neo4j-tutorial/blob/master/settings/ivy.xml and https://github.com/jimwebber/neo4j-tutorial/blob/master/settings/ivysettings.xml which make Ivy aware of the fact that there are number of artifacts needed with the same group and artifact id but different classifiers. Does that work? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 9:03 AM, Markus Joschko markus.josc...@gmail.com wrote: Has anybody had success in getting an embedded neo4j-server running with sbt? I had some trouble in getting the dependencies but now it at least compiles and starts up. However the start ultimately failed. Before switching back to maven I wanted to ask whether someone has a working sbt setup and can share the build.sbt? BTW, the error I encounter at the moment (doesn't seem to be caused by a missing dependency): INFO: Starting Neo Server on port [7474] with [80] threads available Nov 3, 2011 5:01:17 PM org.mortbay.log.Slf4jLog info INFO: Logging to org.slf4j.impl.JDK14LoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Using database at /tmp/tmp-neo-test Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log WARNING: No database tuning properties set in the property file, using defaults. Please specify the performance properties file with org.neo4j.server.db.tuning.properties in the server properties file [null]. Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted discovery module at [/] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted REST API at [/db/data] Nov 3, 2011 5:01:17 PM org.neo4j.server.logging.Logger log INFO: Mounted management API at [/db/manage] Nov
[Neo4j] Activity Streams and Twitter Sample App
Andreas Ronge created a new sample app called kvitter @ https://github.com/andreasronge/kvitter . This got me thinking about the Twitter clone done in Redis @ http://redis.io/topics/twitter-clone If you scroll down 2/3's of the way down you'll read this piece: After we create a post we obtain the post id. We need to LPUSH this post id in every user that's following the author of the post, and of course in the list of posts of the author. This is the bottleneck of the application (if you can call anything on Redis a bottleneck since it's so freaking fast). A tweet by http://twitter.com/#!/APLUSK will have to do 8 Million writes to each of Ashton's Followers. In Neo4j, we shouldn't need to do that since we can express that by relationships: # this should return the tweets of all the people I follow. Person1.outgoing(:follows).outgoing(:tweeted).depth(2).filter(position.length() == 2;) I don't want to get every tweet of every person I follow, just 100 of them. Person1.outgoing(:follows).outgoing(:tweeted).depth(2).filter(position.length() == 2;) .prune(position.returnedNodesCount() 100) But what about ordering them so I get the Latest 100 tweets from all of the tweets a persons followers have tweeted? What are some options here? A) Return all them, and then filter them so only the top 100 are displayed (See getLatestEvents from https://trac.neo4j.org/browser/examples/activity-stream/src/main/java/org/neo4j/examples/activitystream/ActivityStreamExample.java?rev=3888 ) B) A custom Evaluator that adds tweets to an ordered set and trims off the old tweets as it goes based on a tweet date property. (Will have to check every tweet property, like http://sujitpal.blogspot.com/2009/06/custom-traverser-for-neo4j.html ) C) Put the tweet date on the Tweeted Relationship, so we only check at the relationship property level D) Claim defeat and use a tweeted_recently relationship added from every new tweet to every follower (with trimming) sort of like Redis E) Claim defeat and use an index to store the latest tweets like Redis F) (assuming no id reuse) A custom Breadth First Traverser that evaluates just the last 100 Tweeted relationships ordered by id in conjunction with B. (is this possible?) Other ideas? How scalable are these solutions? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Activity-Streams-and-Twitter-Sample-App-tp3477669p3477669.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Activity Streams and Twitter Sample App
Came up with another possibility: G) Store the Latest 100 tweeted relationship ids with dates as a property of the User Node, and a custom Breadth First Traversal that evaluates the list of every follower's latest 100 tweets before deciding which relationships to follow. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Activity-Streams-and-Twitter-Sample-App-tp3477669p3477693.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Batch uploading to REST server
Hi, I was benchmarking some batch creation jobs to the REST server. I tried to load ~15k nodes, 15k relationships between them, and indexing all the nodes. This took in the range of 1 - 2 hours which to me seems to be very slow. Is there anything I can do in server configuration or the like to speed up the process? The 623KB file was POSTed to /db/data/batch. Server is community 1.5.M02. Thanks, Alex -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Batch-uploading-to-REST-server-tp340p340.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Comparing Lucene index lookup performance to lookup by node id
This is probably not news to anyone, but I might as well post about it in case new users are wondering about performance between index based lookups and lookups by node ids. I have a test database of 750,000 nodes of type A. The db also contains 90,000 nodes of types B and C, and roughly 4M relationships between A-B and A-C (so two different relationship types). The size on disk is 4.7GB, of which the Lucene index takes 2.3GB or so. Node of type A has three properties, one fulltext indexed ones and an id type property indexed with type exact index (type of property is a string). Let's call the property name as guid. The relationships and other types of nodes also have indexed properties, which are all indexed in their own indexes. There are about 14M properties in the db. To test the performance I generate a list of all node IDs and guid property values, and perform 400,000 lookups using random entries from those lists, and record the execution time of the 400,000 lookups. This is on a box with 8GB of RAM, and the performance runs are nowhere near using all that memory. I'm using SDN 2.0.0 M1 to access the data. The node id lookups are done with the findOne(Long id) method in the CRUDRepository class and the guid property lookups are done with the findByPropertyValue(String indexName, String property, Object value) method in the NamedIndexRepository class. Using default settings for the graph db. The node id lookups run in about 12,700ms The index based guid property id lookups run in about 123,000ms. So roughly a 10x performance difference. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] GEOFF server plugin reappeared
Hi all,
following Nigels great work, I created a separate GEOFF plugin for the
Neo4j Server, https://github.com/neo4j/geoff-plugin (depending on
https://github.com/nigelsmall/neo4j-geoff-plugin, hoping to get both
building on CI soon) . It lets you submit GEOFF in JSON like
POST http://localhost:7474/db/data/ext/GeoffPlugin/graphdb/insert
Accept: application/json
Content-Type: application/json
{geoff:{(Joe):{name:Joe}}}
Which results in the creation of a node with name=Joe. For more
cool examples, see http://py2neo.org/geoff/
Thanks a lot Nigel for pushing this!
Now, I am not sure how to handle the ordering of things, like adding
indexes needs to be done after creating nodes, but it's a good start
:)
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Rent Low-Cost High Memory Servers on a Monthly Basis
I was reading a thread earlier where the OP was worried about the Sizing Recommendations given by Neo4j specially for testing purposes. I recommend renting a server to explore performance characteristics of your application and data size before shelling out 6 figures a server. About two years ago, I found a hosting provider out of LA called WebNX that rents high end servers for very reasonable prices. Their website is: http://webnx.com and they list their standard setups. They have clearance boxes at : http://www.webnx.com/servers/clearance.html ... and you can find even more specials at http://www.webhostingtalk.com/forumdisplay.php?f=36 (a forum for dedicated server offers) For example: $159 a month: Intel Core i7 870 (4 / 8HT cores at 2.93ghz) 16gb RAM, 1x 120GB SSD w/ 500GB HDD $259 a month: Intel Core i7 870(4 / 8HT cores at 2.93ghz) 32gb RAM, 4x 300gb 15k sas-2 w/ hardware raid 10 $349 a month: AMD(2x 8 core)16 core 64gb RAM, 4x 450gb 15k rpm sas w/ hardware raid 10 $499 a month: Intel Xeon X5550 - 8 cores at 2.66ghz, 96gb RAM, 4x 146gb 15k sas w/ raid 10 10TB BW $899 a month: Intel 12 core 2x 5645(24HT Cores), 144gb RAM, 8x 300gb 15k sas w/ hardware raid 10 You can find these: http://www.webhostingtalk.com/showthread.php?t=1093819highlight=webnx I don't have any financial connection with WEBNX, I am just a customer. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Rent-Low-Cost-High-Memory-Servers-on-a-Monthly-Basis-tp3478120p3478120.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] zero fromDepth and toDepth
Sure, but implementations using it will break then. Maybe not this close to release? hard decision. 2011/11/3 Peter Neubauer peter.neuba...@neotechnology.com So, do we fix the implementation to match (see https://github.com/neo4j/community/issues/80) or adjust the JavaDoc? I kinda think it makes sense to change the code, have written tests for it. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 12:21 AM, Mattias Persson matt...@neotechnology.com wrote: Right, the toDepth implementation isn't matching the javadoc and it is a bit confusing. 2011/11/3 Alex a...@auv.name Done: http://neo4jdb.lighthouseapp.com/projects/77609-neo4j-community/tickets/17-consisnte-behavior-of-fromdepth-todepth-and-atdepth there's a typo in the title... time to get some sleep :) Alex -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/zero-fromDepth-and-toDepth-tp3474825p3476080.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] GEOFF server plugin reappeared
If the data submitted exists purely in a single CompositeDescriptor, the
add method of the GEOFFLoader class handles things in the correct order
already.
Nige
On 3 November 2011 18:22, Peter Neubauer
peter.neuba...@neotechnology.comwrote:
Hi all,
following Nigels great work, I created a separate GEOFF plugin for the
Neo4j Server, https://github.com/neo4j/geoff-plugin (depending on
https://github.com/nigelsmall/neo4j-geoff-plugin, hoping to get both
building on CI soon) . It lets you submit GEOFF in JSON like
POST http://localhost:7474/db/data/ext/GeoffPlugin/graphdb/insert
Accept: application/json
Content-Type: application/json
{geoff:{(Joe):{name:Joe}}}
Which results in the creation of a node with name=Joe. For more
cool examples, see http://py2neo.org/geoff/
Thanks a lot Nigel for pushing this!
Now, I am not sure how to handle the ordering of things, like adding
indexes needs to be done after creating nodes, but it's a good start
:)
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Comparing Lucene index lookup performance to lookup by node id
Indexes, while fast they are still an indirection and way slower than a direct access of something. So this is quite expected. 2011/11/3 Tero Paananen tpp.paana...@gmail.com This is probably not news to anyone, but I might as well post about it in case new users are wondering about performance between index based lookups and lookups by node ids. I have a test database of 750,000 nodes of type A. The db also contains 90,000 nodes of types B and C, and roughly 4M relationships between A-B and A-C (so two different relationship types). The size on disk is 4.7GB, of which the Lucene index takes 2.3GB or so. Node of type A has three properties, one fulltext indexed ones and an id type property indexed with type exact index (type of property is a string). Let's call the property name as guid. The relationships and other types of nodes also have indexed properties, which are all indexed in their own indexes. There are about 14M properties in the db. To test the performance I generate a list of all node IDs and guid property values, and perform 400,000 lookups using random entries from those lists, and record the execution time of the 400,000 lookups. This is on a box with 8GB of RAM, and the performance runs are nowhere near using all that memory. I'm using SDN 2.0.0 M1 to access the data. The node id lookups are done with the findOne(Long id) method in the CRUDRepository class and the guid property lookups are done with the findByPropertyValue(String indexName, String property, Object value) method in the NamedIndexRepository class. Using default settings for the graph db. The node id lookups run in about 12,700ms The index based guid property id lookups run in about 123,000ms. So roughly a 10x performance difference. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Activity Streams and Twitter Sample App
one more solution. set a sampling_ratio, say 10: Person1.outgoing(:follows).outgoing(:tweeted).depth(2).filter(position.length()== 2;) .prune(position.returnedNodesCount() 100 * sampling_ratio) then do a sort based on timestamp. the goal is not to get the perfect result but *good enough* ones depends on your experience. On Thu, Nov 3, 2011 at 4:48 PM, maxdemarzi maxdema...@gmail.com wrote: Came up with another possibility: G) Store the Latest 100 tweeted relationship ids with dates as a property of the User Node, and a custom Breadth First Traversal that evaluates the list of every follower's latest 100 tweets before deciding which relationships to follow. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Activity-Streams-and-Twitter-Sample-App-tp3477669p3477693.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Best wishes, Linan Wang Architect, Programmer, PhD ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Cypher where regular expressions - case insensitive regexs?
re: http://docs.neo4j.org/chunked/snapshot/query-where.html#where-regular-expressions Is there a way to use case insensitive regular expressions in Cypher where clauses? node.property =~ /foo.*/i does not appear to work. This would be a great addition to the language, if it's currently unsupported. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Comparing Lucene index lookup performance to lookup by node id
Indexes, while fast they are still an indirection and way slower than a direct access of something. So this is quite expected. Agreed. I wanted to run the performance tests to find out how much slower the index lookups are. Would they have been 2x - 4x slower, it would've probably still been acceptable for my particular use case. At 10x, I need to think about alternatives. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher where regular expressions - case insensitive regexs?
Great suggestion. I'll add it to the backlog. https://github.com/neo4j/community/issues/82 Andrés On Thu, Nov 3, 2011 at 9:31 PM, Tero Paananen tpp.paana...@gmail.comwrote: re: http://docs.neo4j.org/chunked/snapshot/query-where.html#where-regular-expressions Is there a way to use case insensitive regular expressions in Cypher where clauses? node.property =~ /foo.*/i does not appear to work. This would be a great addition to the language, if it's currently unsupported. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] zero fromDepth and toDepth
We can do this after 1.5. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 12:28 PM, Mattias Persson matt...@neotechnology.com wrote: Sure, but implementations using it will break then. Maybe not this close to release? hard decision. 2011/11/3 Peter Neubauer peter.neuba...@neotechnology.com So, do we fix the implementation to match (see https://github.com/neo4j/community/issues/80) or adjust the JavaDoc? I kinda think it makes sense to change the code, have written tests for it. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 12:21 AM, Mattias Persson matt...@neotechnology.com wrote: Right, the toDepth implementation isn't matching the javadoc and it is a bit confusing. 2011/11/3 Alex a...@auv.name Done: http://neo4jdb.lighthouseapp.com/projects/77609-neo4j-community/tickets/17-consisnte-behavior-of-fromdepth-todepth-and-atdepth there's a typo in the title... time to get some sleep :) Alex -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/zero-fromDepth-and-toDepth-tp3474825p3476080.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Rent Low-Cost High Memory Servers on a Monthly Basis
Very cool, thanks for the hint! /peter On Thu, Nov 3, 2011 at 12:14 PM, maxdemarzi maxdema...@gmail.com wrote: I was reading a thread earlier where the OP was worried about the Sizing Recommendations given by Neo4j specially for testing purposes. I recommend renting a server to explore performance characteristics of your application and data size before shelling out 6 figures a server. About two years ago, I found a hosting provider out of LA called WebNX that rents high end servers for very reasonable prices. Their website is: http://webnx.com and they list their standard setups. They have clearance boxes at : http://www.webnx.com/servers/clearance.html ... and you can find even more specials at http://www.webhostingtalk.com/forumdisplay.php?f=36 (a forum for dedicated server offers) For example: $159 a month: Intel Core i7 870 (4 / 8HT cores at 2.93ghz) 16gb RAM, 1x 120GB SSD w/ 500GB HDD $259 a month: Intel Core i7 870(4 / 8HT cores at 2.93ghz) 32gb RAM, 4x 300gb 15k sas-2 w/ hardware raid 10 $349 a month: AMD(2x 8 core)16 core 64gb RAM, 4x 450gb 15k rpm sas w/ hardware raid 10 $499 a month: Intel Xeon X5550 - 8 cores at 2.66ghz, 96gb RAM, 4x 146gb 15k sas w/ raid 10 10TB BW $899 a month: Intel 12 core 2x 5645(24HT Cores), 144gb RAM, 8x 300gb 15k sas w/ hardware raid 10 You can find these: http://www.webhostingtalk.com/showthread.php?t=1093819highlight=webnx I don't have any financial connection with WEBNX, I am just a customer. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Rent-Low-Cost-High-Memory-Servers-on-a-Monthly-Basis-tp3478120p3478120.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Activity Streams and Twitter Sample App
I had not considered imperfect solutions, and in some activity stream scenarios a sampling of the last few messages could work. The sample would have to be taken from all Person nodes because if we sample from the Tweets in general and we encounter a chatty person node early on, it would take up all the sample space. Person1 - Person 2 - 10 tweets Person1 - Person3 - 1000 tweets = Sample Size reached, traversal stops Person1 - Person4 - 10 tweets Person1 - PersonX - 10 tweets Person3 would prevent Person4 to PersonX's tweets from ever making it to the sample. Some applications in domains like Financials or Network Monitoring may require the last known status and sampling might not be acceptable. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Activity-Streams-and-Twitter-Sample-App-tp3477669p3478477.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationships stored order
Hi Evgeny, Could you maybe describe the use case behind this requirement a bit more? Thanks, David On Sun, Oct 30, 2011 at 4:01 PM, Evgeny Gazdovsky gazdov...@gmail.comwrote: PS We don't need a traverse through relationships in stored order, only iterations for start or end node. -- Evgeny ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- David Montag david.mon...@neotechnology.com Neo Technology, www.neotechnology.com Cell: 650.556.4411 Skype: ddmontag ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] 1.5.M02 - JVM BootStrapper Issues
Hi Guys, I have downloaded the latest Community edition and notice these three custom lines of code causing the Neo4J server not to start up: wrapper.java.additional.1=-d64 wrapper.java.additional.1=-server wrapper.java.additional.1=-Xss2048k With the previous version, they worked fine. They live at the bottom of the neo4j-wrapper.conf Error I get is. java.lang.IllegalArgumentException at java.lang.ProcessImpl.init(Unknown Source) at java.lang.ProcessImpl.start(Unknown Source) at java.lang.ProcessBuilder.start(Unknown Source) at java.lang.Runtime.exec(Unknown Source) at org.neo4j.wrapper.ServerProcessConsole.doStart(ServerProcessConsole.j ava:39) at org.neo4j.wrapper.ServerProcess.init(ServerProcess.java:116) at org.neo4j.wrapper.ServerProcessConsole.init(ServerProcessConsole.ja va:29) at org.neo4j.wrapper.NeoServiceWrapper.launchAsConsoleApp(NeoServiceWrap per.java:48) at org.neo4j.wrapper.NeoServiceWrapper.main(NeoServiceWrapper.java:35) Is this expected, since I am sure the JVM should support adding these additional parameters? I want these in here, so I can always ensure it is 64bit, server mode with a stack size of 2048k ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Exception when converting older graph
Hi Guys,
Is it possible to fix this in future release or not, this means in the event of
a unclean shutdown, a regression is needed or is it possible to had detection
if the old db was not shutdown cleanly and improve the error message?
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Mattias Persson
Sent: Thursday, 27 October 2011 2:12 AM
To: Neo4j user discussions
Subject: Re: [Neo4j] Exception when converting older graph
Hi,
did you open it after a clean shutdown of the 1.4.M05 store? Because it will
auto upgrade the store to a 1.5 format, but there's a problem of detecting
store version in 1.5.M02 if you try to open it after a non-clean shutdown.
2011/10/26 Paul A. Jackson paul.jack...@pb.com
I have a graph that was created with 1.4.M05 that I am trying to open
with 1.5.M02. Is this supported?
I get this exception:
Caused by: org.neo4j.graphdb.TransactionFailureException: Could not
create data source [nioneodb], see nested exception for cause of error
at
org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:153)
at
org.neo4j.kernel.GraphDbInstance.start(GraphDbInstance.java:112)
at
org.neo4j.kernel.EmbeddedGraphDbImpl.init(EmbeddedGraphDbImpl.java:190)
at
org.neo4j.kernel.EmbeddedGraphDatabase.init(EmbeddedGraphDatabase.java:80)
at com.g1.dcg.graph.neo4j.NeoGraph.init(NeoGraph.java:124)
... 42 more
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:249)
at
org.neo4j.kernel.impl.nioneo.xa.Command.readDynamicRecord(Command.java:253)
at
org.neo4j.kernel.impl.nioneo.xa.Command$RelationshipTypeCommand.readCommand(Command.java:957)
at
org.neo4j.kernel.impl.nioneo.xa.Command.readCommand(Command.java:1004)
at
org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource$CommandFactory.readCommand(NeoStoreXaDataSource.java:302)
at
org.neo4j.kernel.impl.transaction.xaframework.LogIoUtils.readTxCommandEntry(LogIoUtils.java:157)
at
org.neo4j.kernel.impl.transaction.xaframework.LogIoUtils.readLogEntry(LogIoUtils.java:99)
at
org.neo4j.kernel.impl.transaction.xaframework.LogIoUtils.readEntry(LogIoUtils.java:76)
at
org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.readEntry(XaLogicalLog.java:866)
at
org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.doInternalRecovery(XaLogicalLog.java:796)
at
org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.open(XaLogicalLog.java:238)
at
org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.open(XaLogicalLog.java:192)
at
org.neo4j.kernel.impl.transaction.xaframework.XaContainer.openLogicalLog(XaContainer.java:97)
at
org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.init(NeoStoreXaDataSource.java:147)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.neo4j.kernel.impl.transaction.XaDataSourceManager.create(XaDataSourceManager.java:75)
at
org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:147)
... 46 more
The values in the readDynamicRecord method at the time of the call are:
static = org.neo4j.kernel.impl.nioneo.xa.Command
byteChannel = {org.neo4j.kernel.impl.util.BufferedFileChannel@53535}
buffer =
{java.nio.DirectByteBuffer@33560}java.nio.DirectByteBuffer[pos=12
lim=12 cap=713]
id = 1
type = 0
inUseFlag = 1
inUse = true
record = {org.neo4j.kernel.impl.nioneo.store.DynamicRecord@63952
}DynamicRecord[1,true,isLight,-1]
nrOfBytes = -1
nextBlock = -4294967280
Thanks.
Paul Jackson, Principal Software Engineer Pitney Bowes Business
Insight
4200 Parliament Place | Suite 600 | Lanham, MD 20706-1844 USA
O: 301.918.0850 | M: 703.862.0120 | www.pb.com paul.jack...@pb.com
Every connection is a new opportunity(tm)
Please consider the environment before printing or forwarding this email.
If you do print this email, please recycle the paper.
This email message may contain confidential, proprietary and/or
privileged information. It is intended only for the use of the intended
recipient(s).
If you have received it in error, please immediately advise the sender
by reply email and then delete this email message. Any disclosure,
copying, distribution or use of the information contained in this
email message to or by anyone other than
Re: [Neo4j] Relationships stored order
2011/11/4 David Montag david.mon...@neotechnology.com Hi Evgeny, Could you maybe describe the use case behind this requirement a bit more? We use the neo as persistent memory in the new age programming language. Every expression on this language is compiled into graph structure. So we need a graph with sorted relationships. And sort order is equal to the order in which relationships are created (stored). -- Evgeny ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 1.5.M02 - JVM BootStrapper Issues
Romiko, could you give the parameters different numbers, like in https://github.com/neo4j/packaging/blob/master/standalone/src/main/distribution/text/community/conf/neo4j-wrapper.conf#L5 and try again? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 4:07 PM, Romiko Derbynew romiko.derby...@readify.net wrote: Hi Guys, I have downloaded the latest Community edition and notice these three custom lines of code causing the Neo4J server not to start up: wrapper.java.additional.1=-d64 wrapper.java.additional.1=-server wrapper.java.additional.1=-Xss2048k With the previous version, they worked fine. They live at the bottom of the neo4j-wrapper.conf Error I get is. java.lang.IllegalArgumentException at java.lang.ProcessImpl.init(Unknown Source) at java.lang.ProcessImpl.start(Unknown Source) at java.lang.ProcessBuilder.start(Unknown Source) at java.lang.Runtime.exec(Unknown Source) at org.neo4j.wrapper.ServerProcessConsole.doStart(ServerProcessConsole.j ava:39) at org.neo4j.wrapper.ServerProcess.init(ServerProcess.java:116) at org.neo4j.wrapper.ServerProcessConsole.init(ServerProcessConsole.ja va:29) at org.neo4j.wrapper.NeoServiceWrapper.launchAsConsoleApp(NeoServiceWrap per.java:48) at org.neo4j.wrapper.NeoServiceWrapper.main(NeoServiceWrapper.java:35) Is this expected, since I am sure the JVM should support adding these additional parameters? I want these in here, so I can always ensure it is 64bit, server mode with a stack size of 2048k ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationships stored order
Do they need to be sorted because of some arbitrary reason, or because you're storing data structures like lists that you want to preserve the order of? David On Thu, Nov 3, 2011 at 4:44 PM, Evgeny Gazdovsky gazdov...@gmail.comwrote: 2011/11/4 David Montag david.mon...@neotechnology.com Hi Evgeny, Could you maybe describe the use case behind this requirement a bit more? We use the neo as persistent memory in the new age programming language. Every expression on this language is compiled into graph structure. So we need a graph with sorted relationships. And sort order is equal to the order in which relationships are created (stored). -- Evgeny ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- David Montag david.mon...@neotechnology.com Neo Technology, www.neotechnology.com Cell: 650.556.4411 Skype: ddmontag ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 1.5.M02 - JVM BootStrapper Issues
Hi Peter, I tried that as well, but to no avail, here is how to reproduce it. # # JVM Parameters # wrapper.java.additional.1=-Dorg.neo4j.server.properties=conf/neo4j-server.properties wrapper.java.additional.2=-Djava.util.logging.config.file=conf/logging.properties # Uncomment the following line to enable garbage collection logging #wrapper.java.additional.3=-Xloggc:data/log/neo4j-gc.log #These three lines are behaving badly. wrapper.java.additional.4=-d64 wrapper.java.additional.5=-server wrapper.java.additional.6=-Xss2048k # Initial Java Heap Size (in MB) wrapper.java.initmemory=512 # Maximum Java Heap Size (in MB) wrapper.java.maxmemory=1024 # # Wrapper settings # -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Peter Neubauer Sent: Friday, 4 November 2011 10:48 AM To: Neo4j user discussions Cc: Tatham Oddie Subject: Re: [Neo4j] 1.5.M02 - JVM BootStrapper Issues Romiko, could you give the parameters different numbers, like in https://github.com/neo4j/packaging/blob/master/standalone/src/main/distribution/text/community/conf/neo4j-wrapper.conf#L5 and try again? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 4:07 PM, Romiko Derbynew romiko.derby...@readify.net wrote: Hi Guys, I have downloaded the latest Community edition and notice these three custom lines of code causing the Neo4J server not to start up: wrapper.java.additional.1=-d64 wrapper.java.additional.1=-server wrapper.java.additional.1=-Xss2048k With the previous version, they worked fine. They live at the bottom of the neo4j-wrapper.conf Error I get is. java.lang.IllegalArgumentException at java.lang.ProcessImpl.init(Unknown Source) at java.lang.ProcessImpl.start(Unknown Source) at java.lang.ProcessBuilder.start(Unknown Source) at java.lang.Runtime.exec(Unknown Source) at org.neo4j.wrapper.ServerProcessConsole.doStart(ServerProcessConsole.j ava:39) at org.neo4j.wrapper.ServerProcess.init(ServerProcess.java:116) at org.neo4j.wrapper.ServerProcessConsole.init(ServerProcessConsole.ja va:29) at org.neo4j.wrapper.NeoServiceWrapper.launchAsConsoleApp(NeoServiceWrap per.java:48) at org.neo4j.wrapper.NeoServiceWrapper.main(NeoServiceWrapper.java:35) Is this expected, since I am sure the JVM should support adding these additional parameters? I want these in here, so I can always ensure it is 64bit, server mode with a stack size of 2048k ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Comparing Lucene index lookup performance to lookup by node id
Hi Tero, thanks for the valueable feedback. Please note that SDN is not yet optimized in all places. So I'd love your input by profiling your use-case. It is probably something in between. Can you use visualvm or yourkit or another profiler to figure out the hotspot methods where the most time is spent for the index-lookup. I would also love to pair with you on this. (Or get your data-generator and use-cases and profile it myself). Could you please try to run the same test with the raw neo4j API to see the difference. Another note: SDN was never intended to be a tool for pulling mass data from the graph into memory as it adds some overhead for management and object creation. The most important use-cases for SDN is that it gives you an easy way to work with the graph in terms of your different domain models and also eases the integration of other libraries (e.g. mvc) that require domain POJOS. For mass-data handling it might be sensible to drop down to the core neo4j API (by getting the index used explicitely and pull the nodes with index.get(key,value)). Cheers Michael Am 03.11.2011 um 18:32 schrieb Tero Paananen: This is probably not news to anyone, but I might as well post about it in case new users are wondering about performance between index based lookups and lookups by node ids. I have a test database of 750,000 nodes of type A. The db also contains 90,000 nodes of types B and C, and roughly 4M relationships between A-B and A-C (so two different relationship types). The size on disk is 4.7GB, of which the Lucene index takes 2.3GB or so. Node of type A has three properties, one fulltext indexed ones and an id type property indexed with type exact index (type of property is a string). Let's call the property name as guid. The relationships and other types of nodes also have indexed properties, which are all indexed in their own indexes. There are about 14M properties in the db. To test the performance I generate a list of all node IDs and guid property values, and perform 400,000 lookups using random entries from those lists, and record the execution time of the 400,000 lookups. This is on a box with 8GB of RAM, and the performance runs are nowhere near using all that memory. I'm using SDN 2.0.0 M1 to access the data. The node id lookups are done with the findOne(Long id) method in the CRUDRepository class and the guid property lookups are done with the findByPropertyValue(String indexName, String property, Object value) method in the NamedIndexRepository class. Using default settings for the graph db. The node id lookups run in about 12,700ms The index based guid property id lookups run in about 123,000ms. So roughly a 10x performance difference. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 1.5.M02 - JVM BootStrapper Issues
Romiko, is this on Windows or Linux? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 5:22 PM, Romiko Derbynew romiko.derby...@readify.net wrote: Hi Peter, I tried that as well, but to no avail, here is how to reproduce it. # # JVM Parameters # wrapper.java.additional.1=-Dorg.neo4j.server.properties=conf/neo4j-server.properties wrapper.java.additional.2=-Djava.util.logging.config.file=conf/logging.properties # Uncomment the following line to enable garbage collection logging #wrapper.java.additional.3=-Xloggc:data/log/neo4j-gc.log #These three lines are behaving badly. wrapper.java.additional.4=-d64 wrapper.java.additional.5=-server wrapper.java.additional.6=-Xss2048k # Initial Java Heap Size (in MB) wrapper.java.initmemory=512 # Maximum Java Heap Size (in MB) wrapper.java.maxmemory=1024 # # Wrapper settings # -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Peter Neubauer Sent: Friday, 4 November 2011 10:48 AM To: Neo4j user discussions Cc: Tatham Oddie Subject: Re: [Neo4j] 1.5.M02 - JVM BootStrapper Issues Romiko, could you give the parameters different numbers, like in https://github.com/neo4j/packaging/blob/master/standalone/src/main/distribution/text/community/conf/neo4j-wrapper.conf#L5 and try again? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 4:07 PM, Romiko Derbynew romiko.derby...@readify.net wrote: Hi Guys, I have downloaded the latest Community edition and notice these three custom lines of code causing the Neo4J server not to start up: wrapper.java.additional.1=-d64 wrapper.java.additional.1=-server wrapper.java.additional.1=-Xss2048k With the previous version, they worked fine. They live at the bottom of the neo4j-wrapper.conf Error I get is. java.lang.IllegalArgumentException at java.lang.ProcessImpl.init(Unknown Source) at java.lang.ProcessImpl.start(Unknown Source) at java.lang.ProcessBuilder.start(Unknown Source) at java.lang.Runtime.exec(Unknown Source) at org.neo4j.wrapper.ServerProcessConsole.doStart(ServerProcessConsole.j ava:39) at org.neo4j.wrapper.ServerProcess.init(ServerProcess.java:116) at org.neo4j.wrapper.ServerProcessConsole.init(ServerProcessConsole.ja va:29) at org.neo4j.wrapper.NeoServiceWrapper.launchAsConsoleApp(NeoServiceWrap per.java:48) at org.neo4j.wrapper.NeoServiceWrapper.main(NeoServiceWrapper.java:35) Is this expected, since I am sure the JVM should support adding these additional parameters? I want these in here, so I can always ensure it is 64bit, server mode with a stack size of 2048k ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] GeoSpatial LayerNodeIndex
org.neo4j.gis.spatial.indexprovider.LayerNodeIndex#add doesn't seem to conform to the org.neo4j.graphdb.index.Index#add interface specifications: Adds a key/value pair for entity to the index. If that key/value pair for the entity is already in the index it's up to the implementation to make it so that such an operation is idempotent. In my tests, calling LayerNodeIndex#add N number of times for the same Node resulted in N number of nodes being added to the corresponding Spatial Layer for that index. https://github.com/neo4j/spatial/blob/master/src/main/java/org/neo4j/gis/spatial/indexprovider/LayerNodeIndex.java Any thoughts? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] GeoSpatial LayerNodeIndex
Mmh, yes that is correct, thanks for spotting it. Could you raise an issue and do you care to fork and fix (https://github.com/neo4j/spatial/issues) ? Otherwise, we might get to it first after next week ... Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 6:29 PM, b m bm3...@gmail.com wrote: org.neo4j.gis.spatial.indexprovider.LayerNodeIndex#add doesn't seem to conform to the org.neo4j.graphdb.index.Index#add interface specifications: Adds a key/value pair for entity to the index. If that key/value pair for the entity is already in the index it's up to the implementation to make it so that such an operation is idempotent. In my tests, calling LayerNodeIndex#add N number of times for the same Node resulted in N number of nodes being added to the corresponding Spatial Layer for that index. https://github.com/neo4j/spatial/blob/master/src/main/java/org/neo4j/gis/spatial/indexprovider/LayerNodeIndex.java Any thoughts? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Comparing Lucene index lookup performance to lookup by node id
So I'd love your input by profiling your use-case. It is probably something in between. Can you use visualvm or yourkit or another profiler to figure out the hotspot methods where the most time is spent for the index-lookup. I would also love to pair with you on this. (Or get your data-generator and use-cases and profile it myself). Could you please try to run the same test with the raw neo4j API to see the difference. Sure thing. I'll try and do this tomorrow, but it might have to wait until next week. Another note: SDN was never intended to be a tool for pulling mass data from the graph into memory as it adds some overhead for management and object creation. The most important use-cases for SDN is that it gives you an easy way to work with the graph in terms of your different domain models and also eases the integration of other libraries (e.g. mvc) that require domain POJOS. For mass-data handling it might be sensible to drop down to the core neo4j API (by getting the index used explicitely and pull the nodes with index.get(key,value)). Absolutely, and this is exactly why I'm using SDN. The performance tests I ran are by no means supposed to simulate normal uses. The repetitions were there to make sure I could get meaningful run times for the micro benchmark. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Relationships stored order
Hello, Storing order required because it's fundamental structure property. If I do type get (title) (book) that always should be title from book never book from title because meaning are different (but second expression is also valid). As you can see both expressions are valid but have different evaluation results. Graph created for that expressions is this (simplified): node(get) - link - node(title)node(get) - link - node(book) , note that node(get) is same node. Now, if query node(get) for children order should be counted: 1 relationship to node(title), 2 relationship to node(book) This is down order requirement, but there also up order one. The graph is compressing on fly, so that mean that get (title) (book) get (author) (book) sharing node(book). Same time from node(book) it should be possible to go back to node(get). Remember that there are two node(get) and up order procedure must find correct one. For now we using lucene and preparing berkeley db index, but interesting to search for better performance too. On Fri, Nov 4, 2011 at 4:56 AM, David Montag david.mon...@neotechnology.com wrote: Do they need to be sorted because of some arbitrary reason, or because you're storing data structures like lists that you want to preserve the order of? David On Thu, Nov 3, 2011 at 4:44 PM, Evgeny Gazdovsky gazdov...@gmail.com wrote: 2011/11/4 David Montag david.mon...@neotechnology.com Hi Evgeny, Could you maybe describe the use case behind this requirement a bit more? We use the neo as persistent memory in the new age programming language. Every expression on this language is compiled into graph structure. So we need a graph with sorted relationships. And sort order is equal to the order in which relationships are created (stored). -- Dmitriy Shabanov ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Cypher DSL - updates, QueryDSL, mapping
Hi guys,
I have continued to work on the Cypher DSL. For reference, have a look here:
https://github.com/neo4j/cypher-dsl/blob/master/src/test/java/org/neo4j/cypherdsl/CypherReferenceTest.java
The main DSL is starting to stabilize, and it should now be reasonably
easy to use. For the design decisions that I posted before, the results are:
1) Statics vs initialization block: both are supported! Examples:
assertEquals( START john=node(0) RETURN john, new CypherQuery()
{{
starts( node( john, 0 ) ).returns( nodes( john ) );
}}.toString() );
is the same as:
starts(node(john, 0)).returns(nodes(john));
In first version there are no static imports, so it's very
tab-completion friendly. Second is more compact, but requires static
imports for alot.
2) Path syntax. To declare a path for the MATCH clause, it looks like so:
.match(path(p).from(x).as(r).optional().out(KNOWS).hops(1,3).to(y)
-
MATCH p=(x)-[r?:KNOWS*1..3]-(y)
So that's that! Have a look at the GitHub class above for more details.
I have also started integrating with QueryDSL for expressions and object
mapping. I'll post separately about that to make it easier to follow any
discussions.
/Rickard
--
Rickard Öberg
Developer
Neo Technology
Twitter: @rickardoberg, Skype: rickardoberg
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Cypher DSL and QueryDSL
Hi, I have made a first stab at integrating the new Cypher DSL with QueryDSL(.com). The first feature is simply being able to use QueryDSL expressions in the WHERE clause. Example: QPerson person = QPerson.person; Assert.assertEquals( START person=node(1,2,3) WHERE person.firstName=\P\ and person.age25 RETURN person, CypherQueryDSL.start( node( person, 1, 2, 3 ) ) .where( person.firstName.eq( P ).and( person.age.gt( 25 ) ) ) .returns( nodes( person ) ) .toString() ); --- QPerson is a generated class from QueryDSL, and comes from a simple Java class Person that has two fields. With that I can then construct the WHERE expression quite easily, as is shown above. And that's about it! Very straightforward. Look at the Cypher DSL in GitHub for example on how to set it up, and what Maven dependencies you need to use it. /Rickard -- Rickard Öberg Developer Neo Technology Twitter: @rickardoberg, Skype: rickardoberg ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Cypher DSL and result mapping
Hi, I've briefly started to look at how to map Cypher query results into DTO's. I'm using the QueryDSL helper classes to do it, but instead of putting the projection methods into the query object (as QueryDSL does it), I have a separate Projection class for it. The reason is that I don't want to require DSL users to have access to the execution engine, which will obviously not be the case if you are creating queries on the client which are to be sent to the server. Here's an example of what I want it to look like when used: Projection projection = new Projection(engine); IterableFriend friends = projection.iterable( start( node( john, john ) ) .match( path().from( john ).out( friend ) .link().out( friend ).to( fof ) ) .returns( properties( john.name, fof.name ) ), Friend.class ); System.out.println( friends ); --- So the Projection encapsulates the reference to the query execution, which means that you don't need to have a reference to it when executing the queries to do the projection. The iterable method takes a DSL query and the Java class you want the results to be projected into. Pretty straightforward. For now the code assumes that the fields in Friend is the same as the order of the return results. This is broken, and will be fixed later to use proper mapping, preferably with aliases in the return statement as hints. What are the projection methods that you would like to see? What are common usecases? /Rickard -- Rickard Öberg Developer Neo Technology Twitter: @rickardoberg, Skype: rickardoberg ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
