Re: [Neo4j] Arnoldi iteration

2010-06-02 Thread Peter Neubauer
Lorenzo, On Mon, May 31, 2010 at 2:33 PM, Lorenzo Livi lorenz.l...@gmail.com wrote: I'll make a try with the 0.6-snapshot version and I'll let you know. yes, please get back with feedback! Tobias and Mattias have been looking closer at some of the algos in the graph-algo package but not all are

Re: [Neo4j] Node creation limit

2010-06-02 Thread Mattias Persson
Exactly, the problem is most likely that you try to insert all your stuff in one transaction. All data for a transaction is kept in memory until committed so for really big transactions it can fill your entire heap. Try to group 10k operations or so for big insertions or use the batch inserter.

Re: [Neo4j] Node creation limit

2010-06-02 Thread Mattias Persson
Exactly, the problem is most likely that you try to insert all your stuff in one transaction. All data for a transaction is kept in memory until committed so for really big transactions it can fill your entire heap. Try to group 10k operations or so for big insertions or use the batch inserter.

[Neo4j] Compacting files?

2010-06-02 Thread Alex Averbuch
Hey, Is there a way to compact the data stores (relationships, nodes, properties) in Neo4j? I don't mind if its a manual operation. I have some datasets that have had a lot of relationships removed from them but the file is still the same size, so I'm guessing there are a lot of holes in this

Re: [Neo4j] PHP REST API client

2010-06-02 Thread Anders Nawroth
Hi! This is awesome! I tried it out and have a suggestion: to make the semantics for storing NULLs consistent you could change the PropertyContainer::__set method to remove the property if it exists when trying to set it to NULL. This will make sure NULL is returned when you try to read the

Re: [Neo4j] [Neo] TransactionEventHandler and Spring transaction handling

2010-06-02 Thread Johan Svensson
Antonis, Just committed some bug fixes in the event framework and hopefully this also solves the problem you experienced when using Spring. Could you please try the latest neo4j-kernel 1.1-SNAPSHOT to see if it works? To answer your other question the handler is called in the same thread and you

[Neo4j] Traversals in REST Service vs. Traversals in neo4j.py

2010-06-02 Thread Javier de la Rosa
I'm developing the support to traversals for Python REST Client. The underlying idea for me is to mantain the compatibility with neo4j.py (a really hard issue), but the traversals made me to think about some questions: 1. How can I implement support to isStopNode or isReturnable in REST Service? I

Re: [Neo4j] Traversals in REST Service vs. Traversals in neo4j.py

2010-06-02 Thread Mattias Persson
2010/6/2 Javier de la Rosa ver...@gmail.com: I'm developing the support to traversals for Python REST Client. The underlying idea for me is to mantain the compatibility with neo4j.py (a really hard issue), but the traversals made me to think about some questions: 1. How can I implement

Re: [Neo4j] Compacting files?

2010-06-02 Thread Johan Svensson
Alex, You are correct about the holes in the store file and I would suggest you export the data and then re-import it again. Neo4j is not optimized for the use case were more data is removed than added over time. It would be possible to write a compacting utility but since this is not a very

Re: [Neo4j] Compacting files?

2010-06-02 Thread Alex Averbuch
Hi Johan, Do you mean a utility that creates a new Neo4j instance and copies all entities into it from an old Neo4j instance? That's definitely no problem. I've written a bit of import/export code in my graph_gen_utils branch. I have a GraphReader interface which is generic and only contains

Re: [Neo4j] Traversals in REST Service vs. Traversals in neo4j.py

2010-06-02 Thread Javier de la Rosa
Thank you for your clarification. On 2 June 2010 13:31, Mattias Persson matt...@neotechnology.com wrote: return filter: { language: javascript, body: position.node().getProperty( 'name' ).equals( 'Javier' ) } Will we see language: python in the near future? -- Javier de la Rosa

Re: [Neo4j] Traversals in REST Service vs. Traversals in neo4j.py

2010-06-02 Thread Javier de la Rosa
And one more question, what's the meaning of uniqueness: node path parameter? What values does it support? Which is the equivalent en neo4j.py? -- Javier de la Rosa ___ Neo4j mailing list User@lists.neo4j.org

Re: [Neo4j] PHP REST API client

2010-06-02 Thread Alastair James
Hi! I tried it out and have a suggestion: to make the semantics for storing NULLs consistent you could change the PropertyContainer::__set method to remove the property if it exists when trying to set it to NULL. Excellent idea! I will add ASAP. For some reason calling Node::save twice

Re: [Neo4j] Traversals in REST Service vs. Traversals in neo4j.py

2010-06-02 Thread Mattias Persson
I don't think the python bindings (or any other binding) has caught up to the new traversal framework. Uniqueness is all about when to visit a node and when not to. If the uniqueness would be NODE_GLOBAL a node wouldn't be visited more than once in a traversal. NODE_PATH means that a node won't be

Re: [Neo4j] Traversals in REST Service vs. Traversals in neo4j.py

2010-06-02 Thread Mattias Persson
2010/6/2 Javier de la Rosa ver...@gmail.com: Thank you for your clarification. On 2 June 2010 13:31, Mattias Persson matt...@neotechnology.com wrote: return filter: {    language: javascript,    body: position.node().getProperty( 'name' ).equals( 'Javier' ) } Will we see language:

[Neo4j] neo4j-utils

2010-06-02 Thread Mattias Persson
Is there someone out there using neo4j-utils component, http://components.neo4j.org/neo4j-utils/ ? I'm the one responsible for creating the (somewhat messy) utilities in there. Something just hit me when looking at it: most of the public methods in the code (although not all) which does some write

Re: [Neo4j] Compacting files?

2010-06-02 Thread Craig Taverner
I've thought about this briefly, and somehow it actually seems easier (to me) to consider a compacting (defragmenting) algorithm than a generic import/export. The problem is that in both cases you have to deal with the same issue, the node/relationship ID's are changed. For the import/export this

Re: [Neo4j] Traversals in REST Service vs. Traversals in neo4j.py

2010-06-02 Thread Javier de la Rosa
On 2 June 2010 16:21, Mattias Persson matt...@neotechnology.com wrote: I don't think the python bindings (or any other binding) has caught up to the new traversal framework. Uniqueness is all about when to visit a node and when not to. If the uniqueness would be NODE_GLOBAL a node wouldn't be

Re: [Neo4j] Compacting files?

2010-06-02 Thread Alex Averbuch
Hi Craig, Just a quick note about needing to keep all IDs in memory during an import/export operation. The way I'm doing it at the moment it's not necessary to do so. When exporting: Write IDs to the exported format (this could be JSON, XML, GML, GraphML, etc) When importing: First import all

Re: [Neo4j] Compacting files?

2010-06-02 Thread Craig Taverner
Yes. I guess you cannot escape an old-new ID map (or in your case ID-GID). I think it is possible to maintain that outside the database: - In memory, as I suggested, but only valid under some circumstances - On disk, and lucene is a good idea here. Why not index with lucene, but without

Re: [Neo4j] Compacting files?

2010-06-02 Thread Alex Averbuch
- On disk, and lucene is a good idea here. Why not index with lucene, but without storing the property to the node? I like it! This sounds like a cleaner approach than my current one, and (I'm not sure about how to do this either) may be no more complex than the way I'm doing it. Like you say,

[Neo4j] Tell neo to not reuse ID's

2010-06-02 Thread Martin Neumann
Hej, Is it somehow possible to tell Neo4j not to reuse id's at all? Im running some experiments on Neo4j and I want to add and delete the nodes and relationships. To make sure that I can repeat the same experiment I create a log containing the ID's of the nodes i want to delete. To make sure

Re: [Neo4j] Node creation limit

2010-06-02 Thread Biren Gandhi
Thanks. Big transactions were indeed problematic. Splitting them down into smaller chunks did the trick. I'm still disappointed by the on-disk size of a minimal node without any relationships or attributes. For 500K nodes, it is taking 80MB space (160 byes/node) and for 1M objects it is consuming

Re: [Neo4j] Node creation limit

2010-06-02 Thread Mattias Persson
Only 4,4mb out of those 80 is consumed by nodes so you must be storing some properties somewhere. Would you mind sharing your code so that it would be easier to get a better insight into your problem? 2010/6/2, Biren Gandhi biren.gan...@gmail.com: Thanks. Big transactions were indeed

Re: [Neo4j] Node creation limit

2010-06-02 Thread Biren Gandhi
There is only 1 property - n (to store name of the node) - used as follows: Node node = graphDb.createNode(); node.setProperty( NAME_KEY, username ); And the values of username are Node-1, Node-2 etc. On Wed, Jun 2, 2010 at 3:14 PM, Mattias Persson

Re: [Neo4j] Node creation limit

2010-06-02 Thread Biren Gandhi
Here is some content from neostore.propertystore.db.strings - another huge file. What are the max number of nodes/relationships that people have tried with Neo4j so far? Can someone share disk space usage characteristics? od -N 1000 -x -c neostore.propertystore.db.strings 000 8500

Re: [Neo4j] Tell neo to not reuse ID's

2010-06-02 Thread Craig Taverner
Here is a crazy idea that probably only works for nodes. Don't actually delete the nodes, just the relationships and the node properties. The skeleton node will retain the id in the table preventing re-use. If these orphans are not relevant to your tests, this should have the effect you are

Re: [Neo4j] Compacting files?

2010-06-02 Thread Thomas Sant'ana
On Wed, Jun 2, 2010 at 9:30 AM, Johan Svensson jo...@neotechnology.comwrote: Alex, You are correct about the holes in the store file and I would suggest you export the data and then re-import it again. Neo4j is not optimized for the use case were more data is removed than added over time.