Re: [Neo4j] Neo4j for a CMS backend
Hi Peter, seems like you confused CMDB (Configuration Management Database) with CMS (Content Management System)? While Neo4j is perfect for building both, the use cases don't have very much in common ... Greetings Axel Am 07.12.2011 09:26, schrieb Peter Neubauer: Hi folks, thought this might be interesting for some of you ... http://www.dzone.com/links/r/why_im_pretty_excited_about_using_neo4j_for_a_cmd.html Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer brew install neo4j neo4j start heroku addons:add neo4j ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Record not in use problem and solution
We remove nodes from the index with index.remove(node). The index is initialized at startup with IndexNode index = graphDb.index().forNodes(fulltextAllNodes, MapUtil.stringMap(provider, lucene, type, fulltext)); Removing the node from the index before executing node.delete() avoids the exception. Isolating the testcase is difficult, and I don't have the time right now. But I can send you a database archive containing a zombie node if you like. Am 17.11.2011 10:35, schrieb Mattias Persson: Removing from index: is that done by looking at properties and removing them from the index for every property. If that's the case then this is expected since the node has been deleted just before. 2011/11/17 Peter Neubauerpeter.neuba...@neotechnology.com Axel, Can you isolate this into a test, and maybe send the messages.log off list for us to look at? On Nov 17, 2011 12:09 AM, Axel Morgnera...@morgner.de wrote: Sorry to say, but I can reproduce the behaviour with 1.5, too. Without deleting the node from the index manually, I get still this org.neo4j.kernel.impl.nioneo.store.InvalidRecordException: 17.11.2011 00:01:48 org.structr.core.node.TransactionCommand execute WARNUNG: Transaction failure org.neo4j.kernel.impl.nioneo.store.InvalidRecordException: Record[46] not in use at org.neo4j.kernel.impl.nioneo.store.NodeStore.getRecord(NodeStore.java:215) at org.neo4j.kernel.impl.nioneo.store.NodeStore.getRecord(NodeStore.java:96) at org.neo4j.kernel.impl.nioneo.xa.WriteTransaction.nodeDelete(WriteTransaction.java:683) at org.neo4j.kernel.impl.persistence.PersistenceManager.nodeDelete(PersistenceManager.java:136) at org.neo4j.kernel.impl.core.NodeManager.deleteNode(NodeManager.java:847) at org.neo4j.kernel.impl.core.NodeImpl.delete(NodeImpl.java:266) at org.neo4j.kernel.impl.core.NodeProxy.delete(NodeProxy.java:57) at org.structr.core.entity.AbstractNode.delete(AbstractNode.java:1358) at org.structr.rest.constraint.ResourceConstraint$2.execute(ResourceConstraint.java:153) at org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:61) at org.structr.rest.constraint.ResourceConstraint.doDelete(ResourceConstraint.java:129) at org.structr.rest.servlet.JsonRestServlet.doDelete(JsonRestServlet.java:188) at javax.servlet.http.HttpServlet.service(HttpServlet.java:647) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:403) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:301) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:162) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:140) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Interestingly, all node properties (besides id) are empty. Greetings Axel Sorry, I meant we had this with 1.4.2, but nevertheless I will double-check 1.5 now .. Am 16.11.2011 22:28, schrieb Peter Neubauer: Axel, please do, as we have found and squashed a number of potential problems in QA for 1.5 after 1.4.1 Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Wed, Nov 16, 2011 at 10:27 PM, Axel Morgnera...@morgner.de wrote: Using neo4j 1.4.1 we had some exceptions like 15.11.2011 18:16:11 org.structr.core.entity.AbstractNode setProperty WARNUNG: Exception in setProperty
Re: [Neo4j] Record not in use problem and solution
Does this problem occur with automatic node indexing? We're not using auto indexing, so I can't say. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Record not in use problem and solution
It cannot be only that. By the looks of it some part of your code uses Node#hasProperty after Node#delete, which is illegal and that's what's causing it. Yes, that's right. The exception is expected behaviour. But why the node's still in the index? I think it's just a timing problem in our code. It's happening only under race conditions when creating and deleting nodes with multiple threads. Maybe it's because after node.delete() the deleting thread doesn't immediately remove it from the index while another thread accesses the node via index lookup, and of course fails to use the node as it has already the notInUse flag persisted. Seems that the node is then kind of locked. I don't know ... It's a bit weird that it's so easy to bring the database into an inconsistant state with a node still being in the index, but our fix is working for us so far, and I understood what we did wrong. Sorry that I don't have more time to sort it out. But if you really think that it's worth to have a deeper look into it, I could setup a small webapp using the libraries we're using here (it's all available at Github [1]). [1] https://github.com/structr/structr/blob/develop/structr/structr-core/src/main/java/org/structr/rest/constraint/ResourceConstraint.java Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Record not in use problem and solution
Using neo4j 1.4.1 we had some exceptions like
15.11.2011 18:16:11 org.structr.core.entity.AbstractNode setProperty
WARNUNG: Exception in setProperty
org.neo4j.kernel.impl.nioneo.store.InvalidRecordException: Record[22]
not in use
at
org.neo4j.kernel.impl.nioneo.store.NodeStore.getRecord(NodeStore.java:178)
at org.neo4j.kernel.impl.nioneo.store.NodeStore.getRecord(NodeStore.java:95)
at
org.neo4j.kernel.impl.nioneo.xa.WriteTransaction.nodeLoadProperties(WriteTransaction.java:1043)
at
org.neo4j.kernel.impl.persistence.PersistenceManager.loadNodeProperties(PersistenceManager.java:114)
at
org.neo4j.kernel.impl.core.NodeManager.loadProperties(NodeManager.java:675)
at org.neo4j.kernel.impl.core.NodeImpl.loadProperties(NodeImpl.java:115)
at
org.neo4j.kernel.impl.core.Primitive.ensureFullProperties(Primitive.java:648)
at org.neo4j.kernel.impl.core.Primitive.hasProperty(Primitive.java:283)
at org.neo4j.kernel.impl.core.NodeProxy.hasProperty(NodeProxy.java:150)
The exception was thrown when our new REST API was under heavy load. In
our testcase, we deleted nodes in a while-true-loop while creating nodes
in another. After some concurrent requests, the nodes were not cleanly
auto-removed from the (manual) index. So some nodes were still returned
by an index search, but the above exception was thrown.
Before fixing, our code for removing looked like this (AbstractNode and
StructrRelationship are just wrapper classes for Neo's Node and
Relationship).
[...]
// 1: delete relationships
if(obj instanceof AbstractNode) {
ListStructrRelationship rels =
((AbstractNode)obj).getRelationships();
for(StructrRelationship rel : rels) {
success = rel.delete();
}
}
// 2: delete object
success = obj.delete();
[...]
Removing the node from the index before deleting it resolved the exception.
[...]
// 1: remove node from index
Services.command(securityContext,
RemoveNodeFromIndex.class).execute(obj);
// 2: delete relationships
if(obj instanceof AbstractNode) {
ListStructrRelationship rels =
((AbstractNode)obj).getRelationships();
for(StructrRelationship rel : rels) {
success = rel.delete();
}
}
// 3: delete object
success = obj.delete();
[...]
Maybe this helps someone.
Tonight, I will do a test with neo4j 1.5 and report back.
Greetings
Axel
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Record not in use problem and solution
Sorry, I meant we had this with 1.4.2, but nevertheless I will
double-check 1.5 now ..
Am 16.11.2011 22:28, schrieb Peter Neubauer:
Axel,
please do, as we have found and squashed a number of potential
problems in QA for 1.5 after 1.4.1
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
On Wed, Nov 16, 2011 at 10:27 PM, Axel Morgnera...@morgner.de wrote:
Using neo4j 1.4.1 we had some exceptions like
15.11.2011 18:16:11 org.structr.core.entity.AbstractNode setProperty
WARNUNG: Exception in setProperty
org.neo4j.kernel.impl.nioneo.store.InvalidRecordException: Record[22]
not in use
at
org.neo4j.kernel.impl.nioneo.store.NodeStore.getRecord(NodeStore.java:178)
at org.neo4j.kernel.impl.nioneo.store.NodeStore.getRecord(NodeStore.java:95)
at
org.neo4j.kernel.impl.nioneo.xa.WriteTransaction.nodeLoadProperties(WriteTransaction.java:1043)
at
org.neo4j.kernel.impl.persistence.PersistenceManager.loadNodeProperties(PersistenceManager.java:114)
at
org.neo4j.kernel.impl.core.NodeManager.loadProperties(NodeManager.java:675)
at org.neo4j.kernel.impl.core.NodeImpl.loadProperties(NodeImpl.java:115)
at
org.neo4j.kernel.impl.core.Primitive.ensureFullProperties(Primitive.java:648)
at org.neo4j.kernel.impl.core.Primitive.hasProperty(Primitive.java:283)
at org.neo4j.kernel.impl.core.NodeProxy.hasProperty(NodeProxy.java:150)
The exception was thrown when our new REST API was under heavy load. In
our testcase, we deleted nodes in a while-true-loop while creating nodes
in another. After some concurrent requests, the nodes were not cleanly
auto-removed from the (manual) index. So some nodes were still returned
by an index search, but the above exception was thrown.
Before fixing, our code for removing looked like this (AbstractNode and
StructrRelationship are just wrapper classes for Neo's Node and
Relationship).
[...]
// 1: delete relationships
if(obj instanceof AbstractNode) {
ListStructrRelationship rels =
((AbstractNode)obj).getRelationships();
for(StructrRelationship rel : rels) {
success= rel.delete();
}
}
// 2: delete object
success= obj.delete();
[...]
Removing the node from the index before deleting it resolved the exception.
[...]
// 1: remove node from index
Services.command(securityContext,
RemoveNodeFromIndex.class).execute(obj);
// 2: delete relationships
if(obj instanceof AbstractNode) {
ListStructrRelationship rels =
((AbstractNode)obj).getRelationships();
for(StructrRelationship rel : rels) {
success= rel.delete();
}
}
// 3: delete object
success= obj.delete();
[...]
Maybe this helps someone.
Tonight, I will do a test with neo4j 1.5 and report back.
Greetings
Axel
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Axel Morgner
Morgner UG
Hanauer Landstr. 291a
60314 Frankfurt
Germany
Phone +49 151 40522060
Skype axel.morgner
E-mail a...@morgner.de
Web http://www.morgner.de
Twitter @amorgner
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Record not in use problem and solution
not cleanly
auto-removed from the (manual) index. So some nodes were still returned
by an index search, but the above exception was thrown.
Before fixing, our code for removing looked like this (AbstractNode and
StructrRelationship are just wrapper classes for Neo's Node and
Relationship).
[...]
// 1: delete relationships
if(obj instanceof AbstractNode) {
ListStructrRelationship rels =
((AbstractNode)obj).getRelationships();
for(StructrRelationship rel : rels) {
success= rel.delete();
}
}
// 2: delete object
success= obj.delete();
[...]
Removing the node from the index before deleting it resolved the exception.
[...]
// 1: remove node from index
Services.command(securityContext,
RemoveNodeFromIndex.class).execute(obj);
// 2: delete relationships
if(obj instanceof AbstractNode) {
ListStructrRelationship rels =
((AbstractNode)obj).getRelationships();
for(StructrRelationship rel : rels) {
success= rel.delete();
}
}
// 3: delete object
success= obj.delete();
[...]
Maybe this helps someone.
Tonight, I will do a test with neo4j 1.5 and report back.
Greetings
Axel
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Axel Morgner
Morgner UG
Hanauer Landstr. 291a
60314 Frankfurt
Germany
Phone +49 151 40522060
Skype axel.morgner
E-mail a...@morgner.de
Web http://www.morgner.de
Twitter @amorgner
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Native UUID support?
Hi, is there any news on implementing native UUID in Neo4j? Back in May 2010, there was a discussion about UUID in Neo: http://lists.neo4j.org/pipermail/user/2010-May/003625.html Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher-Pickle?
People already familiar with graphs seem to love the original Cypher syntax while newbies may find it helpful to start with something they already know. What about letting both coexist peacefully? Axel Am 05.11.2011 14:29, schrieb Andres Taylor: On Nov 5, 2011 1:51 PM, Jim Webberj...@neotechnology.com wrote: I really don't want Cypher to pander to SQL. Cypher is about graph matching and should be awesome at it PQL isn't any different in this aspect. Mattias' ascii-art is still the way to describe your pattern. Cypher is already very like SQL in many ways - PQL is a way to acknowledge these similarities and turn them into a selling point instead. Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher-Pickle?
Hi, like this, people familiar with SQL (which are many) don't have to think, they just read and understand. +1 Axel Am 04.11.2011 18:58, schrieb Peter Neubauer: Hi all, moving this discussion to the community, it is very important getting your feedback! Including Andres' original post on top again ... 2011/11/4 Andres Taylorandres.tay...@neotechnology.com Good afternoon, gentlemen. I've been involved in a sales thingie, and had to translate SQL to Cypher. At around the same time, Andreas suggested that we should call Cypher PQL (pickle) instead - Pattern Query Language. After sleeping on it, I realized the obviousness of the truth - if we want to hook Joe Corp Java-coder, having something that is similar to SQL is a Good Thing (TM). So this morning I changed Cypher (and the 228 breaking tests) to look like SQL. Example: Cypher -=-=-=-=- START lucy = node(1000) MATCH lucy-[:ACTS_IN]-movie-[:ACTS_IN]-co_actor RETURN movie.title, count(*) ORDER BY count(*) DESC PQL -=-=- SELECT movie.title, count(*) FROM node(1000) as lucy PATTERN lucy-[:ACTS_IN]-movie-[:ACTS_IN]-co_actor ORDER BY count(*) DESC The code is in place now (the branch is PQL). What's left to do is take care of cypher-plugin, check that I haven't missed anything in the manual, and make sure SDN doesn't mind. Michael assured me that that would not be a problem. The question is - should we make an effort to ship this with 1.5? I think we should. The change is purely syntactic - and that makes me feel comfortable that no new bugs have been introduced. Opinions? Andrés (Peter - happy now? You win!) On Fri, Nov 4, 2011 at 10:45 AM, David Montag david.mon...@neotechnology.com wrote: Sorta agree with Mattias. OTOH the syntax is pretty cool too. Could we ask the community? Could this discussion be had publicly? Also, I like the fact that Cypher flows forward and SQL flows backward. David 2011/11/4 Mattias Perssonmatt...@neotechnology.com Going down the SQL-like path will certainly please many, but also anger others where it doesn't function exactly like SQL... that I don't think we'd get with that differentiated syntax. Also we'd have to put constraints on evolving the syntax with no, we can't do that because that wouldn't be SQL like and I'd really really dislike that; it'd make us followers instead of leaders. Den 4 november 2011 17:50 skrev Michael Hunger michael.hun...@neotechnology.com: I'll take it to be the devils advocate. Do we really want to be perceived as a SQL-like database (aka first association == RDBMS) also Pickle sounds like nitpicking and also like PL/SQL. What level of boringness do we want to afford. These are the gut feelings. What about all our users that already got comfortable with Cypher (which is a cooler name btw.) and are writing a lot of queries in it. Also our screencasts, documentation and third party libs (also provided by the community have to be updated, or at least checked and their docs updated). Putting select at the beginning breaks the flow somehow, the return at the end felt more natural, because I decided, what I want to project as one of the last things, LINQ did it better too. Are our node and relationship-sets in the start clause really like tables? (Aka sets of similar rows) ? Probably Shoudn't we get rid then of the iconographic syntax as well? And default to something like FROM nodes() as x [LEFT OUTER|…] JOIN nodes() as y via r ON (r.type = 'KNOWS' and … ) I know you're only taking on a role here, but I'd like to say: are you mad? etc. Having said that all, let's go for it. Michael Am 04.11.2011 um 17:14 schrieb Björn Granvik: I like it....a lot. Could someone be the devil's advocate and point to possible problems? Tech, ux, or whatever. Or is it truly just a good thing? /Björn 4 nov 2011 kl. 17:01 skrev Ian Robinsonian.robin...@neotechnology.com: I've got to say, that suddenly looks very good. Interesting moment at the NOSQL Exchange this week with someone struggling to 'see' the ASCII art representation of a graph : Cypher is perfectly comprehensible to me - it's only when you come across someone else's misunderstandings and difficulties that you see the benefits of conforming to expectations. I'd say: let's try get this into 1.5. The sooner we can start showing off something that devs already expect, the more we'll hook them, I'm sure. ian ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Article: The Coming SQL Collapse
My 2 cents: The Neo4j API is clean, open, and sort-of low level by intention. It is neither ugly, smelly, nor it does it violate anything. Neo4j in general is very stable. But, of course, if you try the latest snapshot, it may have bugs (as any software has). Since May 2010, we're developing a CMS based on Neo4j (structr) and do some graph-related projects. Coming from the Oracle world, I can only say that working with Neo4j is a revelation. Axel Am 14.10.2011 14:48, schrieb Tobias Ivarsson: We had an interesting discussion about this internally at Neo Technology today. We thought it might be of interest to the broader community. I don't think the discussion is over, so it would be interesting to continue it on the public mailing list. It regards the initial paragraphs of an article posted to dzone recently: http://www.dzone.com/links/rss/the_coming_sql_collapse.html It mentions Neo4j and how the author dislikes a common way of using Neo4j for building applications. It would be interesting to hear suggestions on how to improve this. Forwarded conversation follows: On Fri, Oct 14, 2011 at 10:13 AM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: I found this while reading feeds in bed last night: *The Coming SQL Collapse* http://www.dzone.com/links/rss/the_coming_sql_collapse.html (Sent from Flipboardhttp://flipboard.com) The things he say about SQL vs NOSQL is not very interesting, but I'd like to raise what he says about Neo4j: I looked at neo4j briefly the other day, and quite predictably thought ‘wow, this looks like a serious tinkertoy: it‘s basically a bunch of nodes where you just blob your attributes.‘ Worse than that, to wrap objects around it, you have to have them explicitly incorporate their node class, which is ugly, smelly, violates every law of separation of concerns and logical vs. physical models. On the plus side, as I started to look at it more, I realized that it was the perfect way to implement a backend for a bayesian inference engine (more on that later). Why? Because inference doesn‘t care particularly about all the droll requirements that are settled for you by SQL, and there are no real set operations to speak of. He attacks our pattern of building domain models with Neo4j, calling it ugly, smelly and in violation of every law of separation of concerns and logical vs. physical models. Is he right? My feeling is that he is brain washed with too many so called best practices, but Neo4j has been my main model for a long time now, my perspective is likely skewed. I'd like to hear your thoughts. -- Tobias Ivarssontobias.ivars...@neotechnology.com On Fri, Oct 14, 2011 at 10:32 AM, Rickard Öberg rickard.ob...@neotechnology.com wrote: Well, I'd tend to agree with the author. Mixing persistence details with the domain model itself is really a bad idea. Infrastructure details should not pollute the domain logic as it does with the currently suggested usage of Neo4j. But I think both Spring Data Graph and the Qi4j usage model fixes this, as it allows you to keep many of those things outside of the domain code. /Rickard On Fri, Oct 14, 2011 at 11:45 AM, Tobias Ivarssontobias.ivarsson@ neotechnology.com wrote: On Fri, Oct 14, 2011 at 11:21 AM, Rickard Öberg rickard.ob...@neotechnology.com wrote: On 10/14/11 17:16 , Tobias Ivarsson wrote: I was hoping for a bit more elaboration, of why it is a bad idea. Spring Data Neo4j operates mainly in the same way (at least it did when I was part of the design process), it just hides the details of it. The model we suggest is not to mix infrastructure details (nodes, relationships, traversals) with the domain logic. We suggest the domain logic be a separate layer, acting on domain data objects (defined as a set of interfaces). What we do suggest though is for those domain data objects to be implemented as wrappers of nodes and relationships. That sounds like transaction script+anemic domain model, which is an anti-pattern as well. I'm guessing the anemic domain model is what you are pointing out as the anti-pattern. I can see how transaction scripts are and ADM usually come together though. References: http://martinfowler.com/eaaCatalog/transactionScript.html http://martinfowler.com/bliki/AnemicDomainModel.html Domain logic should be in the domain objects, and so splitting them into several layers confuse things more than it helps. I agree with you, SDN solves this, and does so well. So is the bad part of it just the part of having the implementation of your domain data objects deal with the Neo4j API, instead of having DTOs and DAOs that you load from the database, operate on in memory, then store back to the db at explicit points? Or are there deeper issues than that? It relates to testability (you should be able to test domain objects without infrastructure), code cohesion (put domain logic in domain objects), and as the
Re: [Neo4j] neo4j-spatial: geotools maven repo problem
Don't know since when, but http://download.osgeo.org/ is up again. Could you please monitor this? If it persists, we can change the repo URL. Cheers, /peter neubauer On Fri, Sep 9, 2011 at 2:52 PM, Axel Morgnera...@morgner.de wrote: Hi, today maven could not download dependencies for neo4j-spatial. It seemed that the repo repository idosgeo/id nameOpen Source Geospatial Foundation Repository/name urlhttp://download.osgeo.org/webdav/geotools//url /repository ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] neo4j-spatial: geotools maven repo problem
Hi, today maven could not download dependencies for neo4j-spatial. It seemed that the repo repository idosgeo/id nameOpen Source Geospatial Foundation Repository/name urlhttp://download.osgeo.org/webdav/geotools//url /repository is not accessible. If used repository idosgeo/id nameOpen Source Geospatial Foundation Repository/name urlhttp://repo.opengeo.org/url /repository in pom.xml instead, it worked. Does anyone know whether this is temporary? Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Community - count me in!
Hi Peter! That's great news! I'm absolutely convinced that an open community that is so well supported like Neo4j's will make the difference in many projects. Greetings Axel Am 06.09.2011 um 16:32 schrieb Peter Neubauer: Hi all, Over the last year, the graph community has seen a fantastic surge of interest and activity. We feel that we can and must support all of you more seriously, so I am stepping up as the first of the Neo Technology community team, dedicating more of my time to explicit community support. http://blog.neo4j.org/2011/09/neo4j-community-team-announcement.html Good days to come everyone, it's an honor to work with you! Cheers, /peter neubauer ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] IMPORTANT: Corruption Bug in 1.4 M05
Hi Jim, thanks for clarification. Yesterday, we set some instances back to M04, no issue occured so far. Axel On 03.07.2011 14:36, Jim Webber wrote: Hi Axel, - Is 1.4 M04 safe? Yes. This bug was introduced in M05. - How can we detect whether corruption occured, and is there a way recover from that? Your database will likely be missing nodes. Though I believe if you perform an abrupt unclean shutdown of M05 and bring up the database (and recover) in M04 you should be OK. If anyone from the kernel team knows better then please correct me here :-) - Is this bug contained in the current SNAPSHOT? Yes, but it has been eliminated from the current source code, so doing a build for yourself might be in order if you don't want to got back to M04. This categorically will not be in the GA release! Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] IMPORTANT: Corruption Bug in 1.4 M05
Hi Jim, thanks for reporting anyway. We run some structr instances on the latest milestone, so I'd be interesed in some more details: - Is 1.4 M04 safe? - How can we detect whether corruption occured, and is there a way recover from that? - Is this bug contained in the current SNAPSHOT? Thanks and best regards Axel -- http://structr.org On 02.07.2011 20:46, Jim Webber wrote: Hi fellow graphistas, In testing our upcoming 1.4 GA release, we discovered a subtle bug in our 1.4 M05 release that can result in database corruption. Under certain circumstances with clean shutdowns of the database it's possible that freed IDs will be recycled more than once, meaning that on restart, multiple nodes may receive the same node ID, corrupting the graph. It's still fine to develop against 1.4 M05 to become familiar with the new features and APIs which will be preserved in the GA release. However we do not recommend, as with milestones in general, moving into production on the 1.4 M05 release since data corruption of data is possible. We're now busy making sure that the 1.4 GA release is a completely solid - thanks to you all for being such an awesome and supportive community. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j -- Can it be embedded in Android?
Yes, structr has its own CMS-oriented graph domain, but is not restrictred to that. structr has in its core a wrapper around Neo4j that provides filesystem-like functionality. The serialization/deserialization part is done by kryonet itself. If you get Neo4j to run on Android, you could use the structr API (structr-core) to build your own Android app, and then sync (push/pull) subtrees between a server instance and the mobile app. But ... do you need real offline persistence on the mobile device? Or would it be sufficient to store the equation graph on a server and just hold a cached view of it on the mobile device, using f.e. the offline capabilities of HTML5? Just a thought .. On 25.06.2011 20:33, Sidharth Kshatriya wrote: In structr, I notice that you have defined your own tree data structure and algorithms. Is there a nice document oriented / graph oriented *lightweight *library we can use with Kryonet? Obviously it would be important to be able to serialize part of the tree (subtree) and not all of it... when we are communicating changes across the network... On Fri, Jun 24, 2011 at 10:11 PM, Axel Morgnera...@morgner.de wrote: In structr [1], we use kryonet [2] to push/pull subtrees including binary files between instances (source code: [3]). We also thought about an Android client which holds a subset of the structr graph. Neo4j running on Android (and iOS as well) would be great. [1] http://structr.org/ [2] http://code.google.com/p/kryonet/ [3] https://github.com/structr/structr/tree/release-0.4-SNAPSHOT/structr/structr-core/src/main/java/org/structr/core/cloud On 24.06.2011 15:35, Rick Bullotta wrote: I'd probably just use a serialized JSON or XML object in that case, if it is truly a tree (and not a graph). -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Sidharth Kshatriya Sent: Friday, June 24, 2011 9:18 AM To: Neo4j user discussions Subject: Re: [Neo4j] Neo4j -- Can it be embedded in Android? The present round of android tablets have 512MB - 1 GB Ram...which is likely to increase in the future... Essentially I want to use a graph database to represent a complicated math document to be shown a mobile / tablet. Each equation is nothing but a tree so graph databases lend themselves well to the problem. The other option is to use an embedded object database like db4o which runs on android and implement a custom tree structure within it... or mobile couchbase (available for android and ios). Any better ideas? On Fri, Jun 24, 2011 at 6:26 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: I think the limited capabilities of the Android device(s) (RAM, primarily) limit the usefulness of Neo4J versus alternatives since the datasets are usually small and simple in mobile apps. If we need any heavy-duty graph work for a mobile app, we'd do it on the server. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org ] On Behalf Of Sidharth Kshatriya Sent: Friday, June 24, 2011 8:53 AM To: Neo4j user discussions Subject: Re: [Neo4j] Neo4j -- Can it be embedded in Android? Yes, I saw that on the mailing list archives too. I would have though there would be some interest in using this on android -- but there seems to be no news about it since... On Fri, Jun 24, 2011 at 6:13 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: I remember something like that, too. The main issue is probably the non-traditional file system that Android exposes. -Original Message- From: user-boun...@lists.neo4j.org [mailto: user-boun...@lists.neo4j.org] On Behalf Of Craig Taverner Sent: Friday, June 24, 2011 8:37 AM To: Neo4j user discussions Subject: Re: [Neo4j] Neo4j -- Can it be embedded in Android? I heard that Peter Neubauer made a port of neo4j to android a few years ago, but that nothing has been done since and no version since then would work. So my understanding is that it does not work on android, but that it is possible to make it work (with some work ;-). Peter is away, but I expect he would have a better answer than me. On Fri, Jun 24, 2011 at 1:33 PM, Sidharth Kshatriya sid.kshatr...@gmail.com wrote: Dear All, I have googled for this on the web and did not arrive at a satisfactory answer. *Question: Is it possible to run Neo4j on Android? * Thanks, Sidharth -- Sidharth Kshatriya www.sidk.info ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUI UX Design - Project Management Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 E-mail a...@morgner.de Web http
Re: [Neo4j] Neo4j -- Can it be embedded in Android?
In structr [1], we use kryonet [2] to push/pull subtrees including binary files between instances (source code: [3]). We also thought about an Android client which holds a subset of the structr graph. Neo4j running on Android (and iOS as well) would be great. [1] http://structr.org/ [2] http://code.google.com/p/kryonet/ [3] https://github.com/structr/structr/tree/release-0.4-SNAPSHOT/structr/structr-core/src/main/java/org/structr/core/cloud On 24.06.2011 15:35, Rick Bullotta wrote: I'd probably just use a serialized JSON or XML object in that case, if it is truly a tree (and not a graph). -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Sidharth Kshatriya Sent: Friday, June 24, 2011 9:18 AM To: Neo4j user discussions Subject: Re: [Neo4j] Neo4j -- Can it be embedded in Android? The present round of android tablets have 512MB - 1 GB Ram...which is likely to increase in the future... Essentially I want to use a graph database to represent a complicated math document to be shown a mobile / tablet. Each equation is nothing but a tree so graph databases lend themselves well to the problem. The other option is to use an embedded object database like db4o which runs on android and implement a custom tree structure within it... or mobile couchbase (available for android and ios). Any better ideas? On Fri, Jun 24, 2011 at 6:26 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: I think the limited capabilities of the Android device(s) (RAM, primarily) limit the usefulness of Neo4J versus alternatives since the datasets are usually small and simple in mobile apps. If we need any heavy-duty graph work for a mobile app, we'd do it on the server. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Sidharth Kshatriya Sent: Friday, June 24, 2011 8:53 AM To: Neo4j user discussions Subject: Re: [Neo4j] Neo4j -- Can it be embedded in Android? Yes, I saw that on the mailing list archives too. I would have though there would be some interest in using this on android -- but there seems to be no news about it since... On Fri, Jun 24, 2011 at 6:13 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: I remember something like that, too. The main issue is probably the non-traditional file system that Android exposes. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Craig Taverner Sent: Friday, June 24, 2011 8:37 AM To: Neo4j user discussions Subject: Re: [Neo4j] Neo4j -- Can it be embedded in Android? I heard that Peter Neubauer made a port of neo4j to android a few years ago, but that nothing has been done since and no version since then would work. So my understanding is that it does not work on android, but that it is possible to make it work (with some work ;-). Peter is away, but I expect he would have a better answer than me. On Fri, Jun 24, 2011 at 1:33 PM, Sidharth Kshatriya sid.kshatr...@gmail.com wrote: Dear All, I have googled for this on the web and did not arrive at a satisfactory answer. *Question: Is it possible to run Neo4j on Android? * Thanks, Sidharth -- Sidharth Kshatriya www.sidk.info ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] neo4j 1.4 for structr 0.4 and self relationships
Hi Neo4j-Team! Just wanted to say 'thank you'. With neo4j 1.4, we can solve a problem we had before with group permissions in structr [1]. Without self relationships, users were unable to see their own group they're in, because the permission isReadable was determined by a SECURITY relationship between the object (the group) and the principal (with group-wise permissions, the principal is the group itself). After upgrading structr to neo4j-1.4-SNAPSHOT (in branch 'release-0.4-SNAPSHOT' [2]) , we can create SECURITY self-relationships for each group, so it just works. :-) Greetings from Frankfurt a.M. Axel [1] http://structr.org/ [2] https://github.com/structr/structr/tree/release-0.4-SNAPSHOT ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Announcing structr 0.3
Dear Neo4j creators, users and supporters, tonight, we released a first public version (v0.3) of structr. Basically, structr is a CMS based on Neo4j. Source code is available at github [1]. With structr, you can build web sites and interactive, database-backed web applications. The principle idea is to render a subgraph starting with a node addressed by a URL. We've put together a roadmap with some more features we've in mind [4]. Currently, there's a lack of detail information. We'll try to get as much documenation online during the next days and weeks. so please be patient. Some basics, a roadmap and some screenshots are available now [3]. To stay informed, follow @structr on Twitter [2]. Thank you all for your support during the last months. The responsiveness and quality of this list is extraordinary, and it was one of the main reasons to go with Neo4j for this project. We're looking forward to further developing structr and add more and more features. Working with a graph database is really exciting! Best wishes from Frankfurt am Main, Germany Axel the structr team [1] https://github.com/structr/structr [2] http://twitter.com/#!/structr http://twitter.com/#%21/structr [3] http://structr.org/ [4] http://structr.org/roadmap ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Color suggestions for the Self-Relationship bike shed
+1 for 1 too Am 16.05.2011 14:11, schrieb Tobias Ivarsson: What do you think? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Finding orphaned nodes...
Maybe slow, but easy and and working:
for (Node n : graphDb.getAllNodes()) {
if (!n.hasRelationship()) {
n.delete();
}
}
On 12.04.2011 20:19, Rick Bullotta wrote:
Is there any recommended technique for scanning a Neo database for orphaned
nodes (e.g. nodes with no relationships)?
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Lucene index: Too many open files
Using 1.3-M03 here. I also try to reproduce it with an isolated test.
Will keep you informed about the result.
Axel
On 01.04.2011 11:01, Mattias Persson wrote:
2011/4/1 Michael Hungermichael.hun...@neotechnology.com
What about the line in the if that doesn't execute the block, if
index-hits .hasNext() returns false?
Does this close the underlying structures ?
if (hits != null hits.iterator().hasNext()) {
As of some milestone back it closes the iterator if there are no results in
it. Which version of neo4j are you running Alex?
and here 2 iterators are opened? perhaps this might be a problem, one for
the if, the second for the for-each loop
Actually IndexHits is cheating where #iterator() {return this;} so there's
only one iterator.
As a note I can say that I'm currently running a test with 30 threads,
randomly modifying/querying an index where some queries are looped through
and some just a bit and then closed manually. Also getSingle() is used
randomly. And as far as I can see, after 10 minutes of such a test running
the number of open files is kept it a steady low. I just cannot reproduce
this on the latest milestone.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Lucene index: Too many open files
On 31.03.2011 09:29, Mattias Persson wrote:
There are two things here:
The error you're getting is because you probably forget to close some
IndexHits somewhere, IndexHits that you haven't iterated all the way
through... could you check your code for such mistakes?
Hmm ... I checked the code. Iteration always goes through all hits I think:
IndexHits hits = index.query(new QueryContext(textualQueryString));
ListAbstractNode result = nodeFactory.createNodes(hits);
The createNodes-Method looks like that:
public ListAbstractNode createNodes(final IterableNode hits) {
ListAbstractNode nodes = new ArrayListAbstractNode();
if (hits != null hits.iterator().hasNext()) {
for (Node node : hits) {
AbstractNode n = createNode(node);
nodes.add(n);
}
}
return nodes;
}
Maybe the IndexHits are not closed if thread gets interrupted?
Now I added a hits.close() after the createNodes line. More stress tests
will follow and reveal all flaws in my code ... ;-)
The other one that doCommit always seems to be invoked I just tried and
cannot reproduce, so may be something on your end? could you provide a code
snippet to reproduce this?
Debugged it, but was not able to isolate or reproduce it. Will keep an
eye on it. I'm quite sure it has to do with my asynchronous logging
service, but it seems that it is kinda hard to track down ...
Thanks for your help, and best regards
Axel
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Lucene index: Too many open files
Hi, during stress tests, I got a Too many open files message (see detailed stack trace below). Server is a Debian Linux 2.6.26 x86_64 w/ 24 GB RAM, Core i7 which handles the load with ease. cat /proc/sys/fs/file-max 2399285 The parameter fs.file-max was already set very high, but I'll increase it further and see if it will happen again. fs.file-max = 6815744 fs.aio-max-nr = 1048576 Are there recommendations of linux kernel parameters? Or/and would you recommend other things to circumvent that, f.e. using multiple indexes instead of one? And it seems to me that finishing a Neo4j transaction will always trigger a LuceneTransaction.doCommit, even if there was no update on index (I try to avoid for the log nodes). Can this be switched off? Thanks and greetings Axel [1] Stack trace java.lang.RuntimeException: java.io.FileNotFoundException: /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too many open files) at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) at org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) at org.structr.core.log.LogService.flushQueue(LogService.java:101) at org.structr.core.log.LogService.run(LogService.java:73) Caused by: java.io.FileNotFoundException: /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too many open files) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java:233) at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.init(SimpleFSDirectory.java:180) at org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) at org.apache.lucene.index.FieldsWriter.init(FieldsWriter.java:86) at org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66) at org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144) at org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:193) at org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1460) at org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1479) at org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1099) at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:777) at org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:752) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1932) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1906) at org.neo4j.index.impl.lucene.LuceneTransaction.applyDocuments(LuceneTransaction.java:299) at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:260) ... 12 more javax.transaction.xa.XAException: Unknown xid[GlobalId[NEOKERNL|5131995998687892543|61442], BranchId[ 52 49 52 49 52 49 ]] at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.rollback(XaResourceManager.java:470) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.rollback(XaResourceHelpImpl.java:111) at org.neo4j.kernel.impl.transaction.TransactionImpl.doRollback(TransactionImpl.java:533) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:648) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) at org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) at org.structr.core.log.LogService.flushQueue(LogService.java:101) at org.structr.core.log.LogService.run(LogService.java:73) Mar 30, 2011 6:31:53 PM org.neo4j.kernel.impl.transaction.TxManager commit SEVERE: Unable to rollback transaction. Some resources may be commited others not. Neo4j kernel should be SHUTDOWN for resource maintance
Re: [Neo4j] Close with running transaction
Seems that another thread is still running when I close the database, so nothing wrong with Neo4j, but with my code. :-( Can I somehow force all running transactions (even when started from other threads) to finish? Am 29.03.2011 10:23, schrieb Mattias Persson: I can't reproduce this. I tried: GraphDatabaseService db = new EmbeddedGraphDatabase( mydb ); IndexNode index = db.index().forNodes( myindex ); Thread.sleep( 5000 ); db.shutdown(); and no such message or recovery is displayed. Are you doing something else in your simple example? 2011/3/28 Axel Morgnera...@morgner.de Good evening, lately I'm experiencing the following log message when closing the database, even if no write transaction was there between startup and shutdown. In my test, I even wait 5s before closing the database and another 5s before stopping the thread - same result. INFO: Close invoked with 1 running transaction(s) This always leads to an unclean shutdown, causing a long recovery period at next startup. Not sure, but I think it shows up since I upgraded to 1.3-M03, or using the new index framework (can't say exactly). I do shutdown the database with graphDb.shutdown() since I can think. Am I missing something? How can I avoid this behaviour? Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Close with running transaction
Good evening, lately I'm experiencing the following log message when closing the database, even if no write transaction was there between startup and shutdown. In my test, I even wait 5s before closing the database and another 5s before stopping the thread - same result. INFO: Close invoked with 1 running transaction(s) This always leads to an unclean shutdown, causing a long recovery period at next startup. Not sure, but I think it shows up since I upgraded to 1.3-M03, or using the new index framework (can't say exactly). I do shutdown the database with graphDb.shutdown() since I can think. Am I missing something? How can I avoid this behaviour? Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] neo4j-spatial pulled from neo4j maven repo?
Hi, is it true that the current neo4j-spatial jarfiles are not available on http://m2.neo4j.org/org/neo4j/neo4j-spatial/ anymore? I cannot find it at http://m2.neo4j.org/snapshots/org/neo4j/ either. On https://repo.neo4j.org/ I only find 0.1-SNAPSHOT. https://repo.neo4j.org/content/repositories/snapshots/org/neo4j/neo4j-spatial/0.5-SNAPSHOT/neo4j-spatial-0.5-SNAPSHOT.jar Where is the correct maven repo? Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j-spatial pulled from neo4j maven repo?
Yes, thanks a lot!! Am 24.03.2011 10:42, schrieb Anders Nawroth: Hi! The neo4j-spatial snapshots are now back. /anders On 03/24/2011 10:18 AM, Axel Morgner wrote: Hi, is it true that the current neo4j-spatial jarfiles are not available on http://m2.neo4j.org/org/neo4j/neo4j-spatial/ anymore? I cannot find it at http://m2.neo4j.org/snapshots/org/neo4j/ either. On https://repo.neo4j.org/ I only find 0.1-SNAPSHOT. https://repo.neo4j.org/content/repositories/snapshots/org/neo4j/neo4j-spatial/0.5-SNAPSHOT/neo4j-spatial-0.5-SNAPSHOT.jar Where is the correct maven repo? Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Apache Chemistry + Neo4j
Kyle, CMIS support is on our roadmap for the 1.0 release of structr [1]. At the moment, we're planning to release a first public beta (v0.3) of structr (CMS/ECM system based on Neo4j) for end of May 2011. Greetings Axel [1] http://structr.org ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Issue with lucene index
That would be great! Or Frankfurt? It's a little more central (but Berlin is ok for me, too) BTW: We should change the subject (it sounds so negative) ;-) Since I created the original post, I'll start a new thread. On 14.03.2011 13:02, Peter Neubauer wrote: Berlin sounds great. Last year a couple of guys met up at StudiVZ, and suddenly we were 30 people. Go for it, there is a LOT of good vibe in Beerlin! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Mar 14, 2011 at 12:37 PM, Michael Hunger michael.hun...@neotechnology.com wrote: They guys could create at least one in Malmö? Isn't Andreas there as well, and certainly some more fine folks? We can do one locally here in Gemany, perhaps Berlin (perhaps we can combine that with our monthly flight to CPH). Cheers Michael Am 14.03.2011 um 11:50 schrieb Jim Webber: Hey Rick, It was a pleasure to meet you too. And this got me thinking - it would be great to meet more folks from this list, or to form user groups, or generally just get a beer and talk Neo4j graphs. Is there, for example, a strong London contingent on this list? I only know me and Nat Pryce so far. Anyone else care to get together in London? Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Beer and Talk
Just thought that beer and talk are importend enough to have its own wiki page: http://wiki.neo4j.org/content/User_Meetings Maybe we can put additional information like dates etc. onto this page? On 14.03.2011 14:17, Andreas Kollegger wrote: Love the idea of an app. Perhaps a topic for the first get-togethers? Some brainstorming on how to self-organize using a graph. For now: email, a spreadsheet and maybe a map? On Mar 14, 2011, at 2:04 PM, Alfredas Chmieliauskas wrote: Great! I think thats a great idea! A On Mon, Mar 14, 2011 at 2:02 PM, Michael Hunger michael.hun...@neotechnology.com wrote: I would, I already have extensive plans for that. I will share them with you :) Cheers Michael Am 14.03.2011 um 13:50 schrieb Alfredas Chmieliauskas: Who would like to start a social networking site for developers (on top of neo4j technology and community)? I'm in. A On Mon, Mar 14, 2011 at 1:45 PM, bhargav gundabhargav@gmail.com wrote: Stockholm, Sweden On Mon, Mar 14, 2011 at 1:41 PM, Alfredas Chmieliauskas al.fre...@gmail.com wrote: Amsterdam On Mon, Mar 14, 2011 at 1:15 PM, Axel Morgnera...@morgner.de wrote: Hi everybody, as said, here's a new thread for the idea of having beer and talk meetings. Possible locations so far: Malmö London Berlin Frankfurt Looking forward to seeing more Neo4j people in personal! Greetings Axel On 14.03.2011 13:02, Peter Neubauer wrote: Berlin sounds great. Last year a couple of guys met up at StudiVZ, and suddenly we were 30 people. Go for it, there is a LOT of good vibe in Beerlin! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Mar 14, 2011 at 12:37 PM, Michael Hunger michael.hun...@neotechnology.com wrote: They guys could create at least one in Malmö? Isn't Andreas there as well, and certainly some more fine folks? We can do one locally here in Gemany, perhaps Berlin (perhaps we can combine that with our monthly flight to CPH). Cheers Michael Am 14.03.2011 um 11:50 schrieb Jim Webber: Hey Rick, It was a pleasure to meet you too. And this got me thinking - it would be great to meet more folks from this list, or to form user groups, or generally just get a beer and talk Neo4j graphs. Is there, for example, a strong London contingent on this list? I only know me and Nat Pryce so far. Anyone else care to get together in London? Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Beer and Talk
To come back to the talk part - how about a round of Lightning Talk (or Pecha Kucha [1])-style presentations as part of the meetings? Peter, to your question: How does immer der, der fragt translate to English. ;-) But seriously: I'd like to help, but before the release date of our project, my time is very limited. Of course I will help organize local things here in Frankfurt or support the people in Berlin. Greetings Axel [1] http://en.wikipedia.org/wiki/Pecha_Kucha On 14.03.2011 14:47, Peter Neubauer wrote: Ok, great stuff! Now, could we try to get everyone to have a meetup during the next 2 months, so we can collect some feedback on what is interesting and how to do things? And, it would be great to have a meetup manager to coordinate things, if anyone likes to step up? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Mar 14, 2011 at 2:23 PM, Axel Morgnera...@morgner.de wrote: Just thought that beer and talk are importend enough to have its own wiki page: http://wiki.neo4j.org/content/User_Meetings Maybe we can put additional information like dates etc. onto this page? On 14.03.2011 14:17, Andreas Kollegger wrote: Love the idea of an app. Perhaps a topic for the first get-togethers? Some brainstorming on how to self-organize using a graph. For now: email, a spreadsheet and maybe a map? On Mar 14, 2011, at 2:04 PM, Alfredas Chmieliauskas wrote: Great! I think thats a great idea! A On Mon, Mar 14, 2011 at 2:02 PM, Michael Hunger michael.hun...@neotechnology.comwrote: I would, I already have extensive plans for that. I will share them with you :) Cheers Michael Am 14.03.2011 um 13:50 schrieb Alfredas Chmieliauskas: Who would like to start a social networking site for developers (on top of neo4j technology and community)? I'm in. A On Mon, Mar 14, 2011 at 1:45 PM, bhargav gundabhargav@gmail.com wrote: Stockholm, Sweden On Mon, Mar 14, 2011 at 1:41 PM, Alfredas Chmieliauskas al.fre...@gmail.comwrote: Amsterdam On Mon, Mar 14, 2011 at 1:15 PM, Axel Morgnera...@morgner.de wrote: Hi everybody, as said, here's a new thread for the idea of having beer and talk meetings. Possible locations so far: Malmö London Berlin Frankfurt Looking forward to seeing more Neo4j people in personal! Greetings Axel On 14.03.2011 13:02, Peter Neubauer wrote: Berlin sounds great. Last year a couple of guys met up at StudiVZ, and suddenly we were 30 people. Go for it, there is a LOT of good vibe in Beerlin! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Mar 14, 2011 at 12:37 PM, Michael Hunger michael.hun...@neotechnology.comwrote: They guys could create at least one in Malmö? Isn't Andreas there as well, and certainly some more fine folks? We can do one locally here in Gemany, perhaps Berlin (perhaps we can combine that with our monthly flight to CPH). Cheers Michael Am 14.03.2011 um 11:50 schrieb Jim Webber: Hey Rick, It was a pleasure to meet you too. And this got me thinking - it would be great to meet more folks from this list, or to form user groups, or generally just get a beer and talk Neo4j graphs. Is there, for example, a strong London contingent on this list? I only know me and Nat Pryce so far. Anyone else care to get together in London? Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUI UX Design - Project Management c/o inxire GmbH Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 E-mail a...@morgner.de Web http://www.morgner.de ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] InvalidRecordException
$RelationshipEventConsumerImpl.getMoreRelationships(NeoStoreXaConnection.java:304) at org.neo4j.kernel.impl.nioneo.xa.NioNeoDbPersistenceSource$NioNeoDbResourceConnection.getMoreRelationships(NioNeoDbPersistenceSource.java:465) at org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:100) at org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:585) at org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:358) at org.neo4j.kernel.impl.core.IntArrayIterator.hasNext(IntArrayIterator.java:115) Do you think the DB is getting corrupted? -- Massimo http://meridio.blogspot.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUI UX Design - Project Management c/o inxire GmbH Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 E-mail a...@morgner.de Web http://www.morgner.de ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] InvalidRecordException
Hi Tobias, I sent you more detailed information off the list. Greetings Axel On 11.03.2011 10:49, Tobias Ivarsson wrote: Please tell me that you have the database file in the state they were in when this happened. That you have not tried to repair the database in any way. If you do, could you please send me those database files off-list (dropbox or similar, or contact me directly to discuss how to transfer it). I will of course treat your data with utmost privacy, but I would love to get to examine a store that exhibits this behavior to find out what could have caused it. Massimo: If you can produce a test case that reliably (or at least semi-reliably) reproduces this, that would of course be even better, but it doesn't hurt if I could look at the store files and try to deduce the problem in parallell. Cheers, ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] InvalidRecordException
Hi,
I'm getting an InvalidRecordException
org.neo4j.kernel.impl.nioneo.store.InvalidRecordException: Node[5] is
neither firstNode[37781] nor secondNode[37782] for Relationship[188125]
at
org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getMoreRelationships(ReadTransaction.java:131)
at
org.neo4j.kernel.impl.nioneo.xa.NioNeoDbPersistenceSource$ReadOnlyResourceConnection.getMoreRelationships(NioNeoDbPersistenceSource.java:280)
at
org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:100)
at
org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:585)
at
org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:358)
at
org.neo4j.kernel.impl.core.IntArrayIterator.hasNext(IntArrayIterator.java:115)
when iterating through the relationships of a certain node:
Node node = graphDb.getNodeById(sNode.getId());
IterableRelationship rels =
node.getRelationships(relType, dir);
for (Relationship r : rels) { - here the expeption
occurs
...
}
I'm using 1.3.M03.
Seems that the database is in an inconsitant state. Don't know how this
could happen ...
Greetings
Axel
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Issue with lucene index
Upgraded to 1.3.M03, issue is gone. Thanks a lot! Axel Am 09.03.2011 20:24, schrieb Mattias Persson: Den onsdagen den 9:e mars 2011 skrev David Montagdavid.mon...@neotechnology.com: Alex, That is caused by a bug. The bug has been fixed after 1.2. If you upgrade to the latest milestone, you will no longer get this exception. Also, the latest milestone will recover the store to a clean state. Your data will be safe. Correct, and go with the M03 milestone first because M04 will introduce changes which requires an upgrade from a cleanly shut down database. David On Wed, Mar 9, 2011 at 10:26 AM, Axel Morgnera...@morgner.de wrote: Hi out there, is there a caveat when concurrently accessing a lucene index for writing? From time to time, in rare cases, I'm getting an exception when committing and indexing a series of nodes to Neo4j (1.2). The transaction is triggered by a log service thread while the main thread (web app) concurrently writes updates to the index, too. After such an exception, the system is unable to recover unless I manually remove the lucene, lucene-fulltext and index folders from the neo4j directory and start over. Then the system comes up again, and I can sucessfully rebuild the index. Do I have to take care of locking by myself? Greetings Axel [1] java.lang.NullPointerException at org.neo4j.index.impl.lucene.IndexType$CustomType.removeFromDocument(IndexType.java:165) at org.neo4j.index.impl.lucene.LuceneCommand$RemoveCommand.perform(LuceneCommand.java:228) at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:231) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:319) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:447) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:65) at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:517) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:623) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:586) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:105) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:86) at org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:968) at org.structr.core.log.LogService.run(LogService.java:85) javax.transaction.xa.XAException: Unknown xid[GlobalId[NEOKERNL|-7500914275797900644|1974], BranchId[ 52 49 52 49 52 49 ]] at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.rollback(XaResourceManager.java:471) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.rollback(XaResourceHelpImpl.java:112) at org.neo4j.kernel.impl.transaction.TransactionImpl.doRollback(TransactionImpl.java:534) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:650) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:586) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:105) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:86) at org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:968) at org.structr.core.log.LogService.run(LogService.java:85) 09.03.2011 18:27:07 org.neo4j.kernel.impl.transaction.TxManager commit SCHWERWIEGEND: Unable to rollback transaction. Some resources may be commited others not. Neo4j kernel should be SHUTDOWN for resource maintance and transaction recovery java.lang.NullPointerException at org.neo4j.index.impl.lucene.IndexType$CustomType.removeFromD-- David Montag Neo Technology, www.neotechnology.com Cell: 650.556.4411 david.mon...@neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to copy a complete database?
Hi Balazs,
you have to manually copy all nodes and relationships from one to the
other database. I have written a tool for that and posted some code here
[1].
But take care, in the posted version in [1], relationship properties
were not copied. To fix that, you have to replace
s.createRelationshipTo(e, r.getType());
by
Relationship newRelationship =
s.createRelationshipTo(e, r.getType());
for (String key : r.getPropertyKeys()) {
newRelationship.setProperty(key, r.getProperty(key));
}
I hope this is useful for you.
Greetings
Axel
[1] http://lists.neo4j.org/pipermail/user/2010-November/005478.html
On 03.03.2011 11:35, Balazs E. Pataki wrote:
Hi,
I have a big database based on Neo4J 1.2. Now, if I would like to use
the short strings feature of Neo4j 1.3 M03 I should regenerate my full
database, that is all strings should be reset so that it may or may not
be stored according to the new short strings policy.
It seems to me that the easiest way to do this would be to somehow be
able to copy the full 1.2 database to a newly created 1.3 M03 database
by traversing the 1.2 database. But there maybe a simpler (neo4j
builtin) way to do this. Any hints about this?
Thanks,
---
balazs
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to copy a complete database?
+1 On 03.03.2011 12:25, Peter Neubauer wrote: Mmh, maybe this could be in some Utilities class? Seems a good thing to be able to clone a graph ... ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Exception in indexer framework when commiting transaction
Hi, after transition to the new indexer framework, I tried to rebuild the index on a small test database with 11,000 nodes. It failed with the following exception: java.lang.IllegalArgumentException: name and value cannot both be empty at org.apache.lucene.document.Field.init(Field.java:398) at org.apache.lucene.document.Field.init(Field.java:371) at org.apache.lucene.document.Field.init(Field.java:350) at org.neo4j.index.impl.lucene.IndexType.instantiateField(IndexType.java:331) at org.neo4j.index.impl.lucene.IndexType$CustomType.addToDocument(IndexType.java:159) at org.neo4j.index.impl.lucene.LuceneCommand$AddCommand.perform(LuceneCommand.java:187) at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:231) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:319) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:447) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:65) at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:517) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:623) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:586) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:105) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:86) Thu Feb 24 14:21:56 CET 2011: --- CONFIGURATION START --- Thu Feb 24 14:21:56 CET 2011: Physical mem: 7999MB, Heap size: 5333MB Thu Feb 24 14:21:56 CET 2011: Kernel version: Neo4j - Graph Database Kernel 1.2-SNAPSHOT (revision: 8112) Thu Feb 24 14:21:56 CET 2011: Operating System: Linux; version: 2.6.35-25-generic; arch: amd64; cpus: 8 Thu Feb 24 14:21:56 CET 2011: VM Name: OpenJDK 64-Bit Server VM Thu Feb 24 14:21:56 CET 2011: VM Vendor: Sun Microsystems Inc. Thu Feb 24 14:21:56 CET 2011: VM Version: 19.0-b09 I made sure that I only add non-null values to the index. Does anyone have an idea what's going wrong? Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Exception in indexer framework when commiting transaction
Thanks Matthias, yes, I had one empty key/value property. After adding checks, it now works fine. Greetings Axel Am 24.02.2011 15:28, schrieb Mattias Persson: Use the source, Luke. Consulting the lucene source I see that you're trying to add something where both key and value are empty. Not null, but empty strings that is. You should have checks for that as well in your code. 2011/2/24 Axel Morgnera...@morgner.de: Hi, after transition to the new indexer framework, I tried to rebuild the index on a small test database with 11,000 nodes. It failed with the following exception: java.lang.IllegalArgumentException: name and value cannot both be empty at org.apache.lucene.document.Field.init(Field.java:398) at org.apache.lucene.document.Field.init(Field.java:371) at org.apache.lucene.document.Field.init(Field.java:350) at org.neo4j.index.impl.lucene.IndexType.instantiateField(IndexType.java:331) at org.neo4j.index.impl.lucene.IndexType$CustomType.addToDocument(IndexType.java:159) at org.neo4j.index.impl.lucene.LuceneCommand$AddCommand.perform(LuceneCommand.java:187) at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:231) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:319) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:447) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:65) at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:517) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:623) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:586) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:105) at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:86) Thu Feb 24 14:21:56 CET 2011: --- CONFIGURATION START --- Thu Feb 24 14:21:56 CET 2011: Physical mem: 7999MB, Heap size: 5333MB Thu Feb 24 14:21:56 CET 2011: Kernel version: Neo4j - Graph Database Kernel 1.2-SNAPSHOT (revision: 8112) Thu Feb 24 14:21:56 CET 2011: Operating System: Linux; version: 2.6.35-25-generic; arch: amd64; cpus: 8 Thu Feb 24 14:21:56 CET 2011: VM Name: OpenJDK 64-Bit Server VM Thu Feb 24 14:21:56 CET 2011: VM Vendor: Sun Microsystems Inc. Thu Feb 24 14:21:56 CET 2011: VM Version: 19.0-b09 I made sure that I only add non-null values to the index. Does anyone have an idea what's going wrong? Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUI UX Design - Project Management c/o inxire GmbH Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 e-maila...@morgner.de Webhttp://www.morgner.de ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help us make Neo4j better at handling YOUR data
Tobias, here are some results from a mid-size CMS database: Computing character frequencies for 205910 string records 30% ... 40% ... 50% ... 60% ... 70% ... 80% ... 90% ...100% Matching potential encodings for 205910 string records 30% ... 40% ... 50% ... 60% ... 70% ... 80% ... 90% ...100% STRING STORE STATISTICS = 4 bit frequencies = e: 14651 t: 12769 m: 10505 q: 9569 r: 7346 o: 7304 l: 6977 0: 6883 s: 6491 a: 6315 i: 5609 1: 4876 H: 4851 U: 3977 2: 3661 n: 3655 : 3325 S: 3160 R: 3062 D: 2773 f: 2638 C: 2618 y: 2447 9: 2376 u: 2302 M: 2100 c: 2088 E: 2086 A: 2080 p: 1767 3: 1664 8: 1660 = 5 bit frequencies = e: 2697 a: 2326 n: 2246 : 2125 t: 1835 r: 1630 o: 1590 i: 1290 m: 769 l: 739 d: 734 s: 718 I: 589 p: 578 g: 577 C: 570 c: 554 A: 534 u: 531 y: 498 M: 416 .: 405 h: 393 b: 326 S: 294 1: 280 G: 264 0: 263 -: 237 O: 215 2: 214 P: 199 B: 197 j: 197 D: 181 R: 179 6: 177 5: 173 3: 161 4: 159 T: 149 v: 141 J: 140 9: 137 k: 136 8: 132 L: 131 w: 123 +: 120 H: 119 ,: 94 N: 92 _: 90 f: 90 K: 81 7: 80 W: 75 E: 65 /: 58 V: 52 z: 50 F: 46 Q: 38 x: 32 = 6 bit frequencies = : 8770 0: 4902 a: 4646 1: 4115 e: 4073 2: 4057 n: 3753 3: 3496 o: 3313 4: 3268 t: 3117 6: 2936 +: 2913 8: 2886 r: 2813 5: 2724 7: 2484 9: 2449 i: 2343 s: 2097 l: 1657 c: 1652 C: 1006 P: 1003 d: 996 u: 944 h: 832 m: 570 g: 538 M: 494 S: 471 .: 466 p: 410 ,: 409 y: 380 b: 336 R: 317 H: 307 f: 306 B: 296 A: 294 k: 290 v: 271 L: 249 T: 226 j: 210 w: 201 O: 185 K: 182 I: 173 V: 156 -: 143 D: 143 G: 142 z: 137 N: 135 /: 118 W: 114 @: 104 J: 103 F: 94 E: 62 x: 61 =: 50 : 37 Z: 36 U: 33 : 32 Y: 31 q: 30 ß: 30 : 22 _: 20 ': 19 ö: 17 Q: 13 ü: 12 ’: 10 X: 10 ;: 7 ä: 7 é: 6 ): 5 ?: 5 á: 5 ó: 5 #: 4 (: 3 í: 3 ô: 3 !: 2 °: 2 Ö: 2 è: 2 : 1 :: 1 [: 1 }: 1 º: 1 ï: 1 â: 1 à: 1 ú: 1 ù: 1 ò: 1 27513 strings with category bitmask 0b0 137 strings with category bitmask 0b1 1 strings with category bitmask 0b10 3041 strings with category bitmask 0b1000 39 strings with category bitmask 0b1 6775 strings with category bitmask 0b10001 4 strings with category bitmask 0b101110 7516 strings with category bitmask 0b1010001 949 strings with category bitmask 0b1000 2 strings with category bitmask 0b11101110 45 strings with category bitmask 0b1 104 strings with category bitmask 0b10001 3 strings with category bitmask 0b11000 3 strings with category bitmask 0b11001 872 strings with category bitmask 0b10001 6375 strings with category bitmask 0b100010001 4 strings with category bitmask 0b100010011 152 strings with category bitmask 0b100010101 16 strings with category bitmask 0b100011000 167 strings with category bitmask 0b100011001 1 strings with category bitmask 0b100111001 5133 strings with category bitmask 0b10011 3338 strings with category bitmask 0b101010001 149 strings with category bitmask 0b11001 6517 strings with category bitmask 0b110010001 2 strings with category bitmask 0b110010011 54 strings with category bitmask 0b110011001 1745 strings with category bitmask 0b111010001 1 strings with category bitmask 0b11101 1297 strings with category bitmask 0b1 = Category index = 0. NineSevenBitAscii 1. LowerCaseHexadecimal 2. UpperCaseHexadecimal 3. PunctuatedNumerical 4. AlphaNumericalName 5. Numerical 6. FrequencyBased:4bit 7. FrequencyBased:5bit 8. FrequencyBased:6bit /STRING STORE STATISTICS On 05.02.2011 11:05, Tobias Ivarsson wrote: Damn. That one place assumes that you don't have any empty strings. I've uploaded a patched version. Same location: https://github.com/downloads/thobe/neo4j-admin-store/stringstat.jar -tobias On Fri, Feb 4, 2011 at 6:52 PM, Rick Bullotta rick.bullo...@burningskysoftware.com wrote: Same here. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Axel Morgner Sent: Friday, February 04, 2011 12:29 PM To: user@lists.neo4j.org Subject: Re: [Neo4j] Help us make Neo4j better at handling YOUR data Hi Tobias, just ran the utility, but got an exception: Computing character frequencies for 205895 string records 30% ... 40% ... 50% ... 60% ... 70% ... 80% ... 90% ...100% Matching potential encodings for 205895 string records ...Exception in thread main java.lang.StringIndexOutOfBoundsException: String index out of range: 0 at java.lang.String.charAt(String.java:694) at org.neo4j.admin.tool.stringstat.Numerical.matches(Numerical.java:30) at org.neo4j.admin.tool.stringstat.TryAssumptions.process(TryAssumptions.java:4 6) at org.neo4j.admin.tool.stringstat.Main.main(Main.java:55) Greetings Axel I have written
Re: [Neo4j] Some brainstorming ideas - 1. Use Neo4j as Document Store
That's a very interesting approach, very graph-like. And it would open the door to distributed/parallel blob access/storage. On 06.02.2011 22:29, Michael Hunger wrote: What about the partition blobs into evenly sized shards and distribute them over more nodes. Then you can adapt this to the block size your neo4j-store is optimized for and the cache penalties should be that bad. Just link the nodes that contain the block data, but also make them all accessible from a blob-root e.g. via numbered relationships. And then retrieve the whole binary data via a traverser. Cheers Michael Am 06.02.2011 um 22:22 schrieb Rick Bullotta: Axel/Michael: The real issue to consider is that Neo4J is not well suited for storing large strings/blobs of data. It is very easy to do, but not really an optimal use of Neo4J. The transaction logs and in-memory caches will quickly get clogged with the large property values. It is probably better to create an abstraction between the raw document storage and the access veneer, the latter of which Neo is a fine choice. You could use any K-V store, document store, or even the file system itself, and get the best of both worlds. What I do think, however, is that Neo is not the best choice for the document storage itself (at present). One other approach that might work is to extend Neo to be able to tell the Neo engine not to cache specific properties and to not place certain actions in the transaction log. This way, Neo itself might be used for storage, without the transaction clog and cache poisoning. ;-) Just a thought... Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Michael Hunger Sent: Sunday, February 06, 2011 4:15 PM To: Neo4j user discussions Subject: Re: [Neo4j] Some brainstorming ideas - 1. Use Neo4j as Document Store Axel, Actually you could do both. I collapsed the file-system-structure to a simple notion of collection + key as in mongodb but of course you could use the filesystem metaphor to model the b-tree that resolves a key that represents a path to the document node. Michael Am 06.02.2011 um 22:09 schrieb Axel Morgner: Hi Michael, just for clarification: Do you refer to how to store the internal structure of BSON documents, or how to store documents in a filesystem-like tree structure? Axel On 06.02.2011 21:50, Michael Hunger wrote: Hi there, this weekend I was away from a computer so I spent some time brainstorming things. I would be interested in any feedback on those thoughts and would like to broaden the audience beyond the devteam. My question: Has anyone ever tried to use Neo4j as a document store ala MongoDB. The idea. From the root node have relationships to a number of buckets or collections. From each collection root have a b-tree to all key-nodes of the system. Each key-node is just connected by some specfic relationships (like contains or link or subdocument) to its data. (i.e. nodes, relationships and their properties) not necessarily to other parts of the graph. So for each key node one uses just a default traverser (depth-first, end of graph, limited to outgoing relationships of the mentioned types) to collect the whole document (or custom traversers to retrieve only parts of the documents). That could be an interesting experiment to run. WDYT? Michael ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Some brainstorming ideas - 1. Use Neo4j as Document Store
Ok, sounds great. Currently we're using a rather classical approach (as Rick I think) which is, storing binary content in the file system. In our CMS use case, we have a typical properties/binary ratio of 10/90 percent, so the Neo store remains small and handy. Another benefit is that direct file access is easy and matured. But it's outside Neo. Axel On 06.02.2011 22:14, Michael Hunger wrote: Axel, Actually you could do both. I collapsed the file-system-structure to a simple notion of collection + key as in mongodb but of course you could use the filesystem metaphor to model the b-tree that resolves a key that represents a path to the document node. Michael Am 06.02.2011 um 22:09 schrieb Axel Morgner: Hi Michael, just for clarification: Do you refer to how to store the internal structure of BSON documents, or how to store documents in a filesystem-like tree structure? Axel On 06.02.2011 21:50, Michael Hunger wrote: Hi there, this weekend I was away from a computer so I spent some time brainstorming things. I would be interested in any feedback on those thoughts and would like to broaden the audience beyond the devteam. My question: Has anyone ever tried to use Neo4j as a document store ala MongoDB. The idea. From the root node have relationships to a number of buckets or collections. From each collection root have a b-tree to all key-nodes of the system. Each key-node is just connected by some specfic relationships (like contains or link or subdocument) to its data. (i.e. nodes, relationships and their properties) not necessarily to other parts of the graph. So for each key node one uses just a default traverser (depth-first, end of graph, limited to outgoing relationships of the mentioned types) to collect the whole document (or custom traversers to retrieve only parts of the documents). That could be an interesting experiment to run. WDYT? Michael ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Help us make Neo4j better at handling YOUR data
Hi Tobias, just ran the utility, but got an exception: Computing character frequencies for 205895 string records 30% ... 40% ... 50% ... 60% ... 70% ... 80% ... 90% ...100% Matching potential encodings for 205895 string records ...Exception in thread main java.lang.StringIndexOutOfBoundsException: String index out of range: 0 at java.lang.String.charAt(String.java:694) at org.neo4j.admin.tool.stringstat.Numerical.matches(Numerical.java:30) at org.neo4j.admin.tool.stringstat.TryAssumptions.process(TryAssumptions.java:46) at org.neo4j.admin.tool.stringstat.Main.main(Main.java:55) Greetings Axel I have written a small utility that analyzes the string properties stored by Neo4j and computes some statistics about them. If I could get as many of you to run this tool on your stores and send those statistics to me as possible, that would be great. This tool is available for download here: https://github.com/downloads/thobe/neo4j-admin-store/stringstat.jar To run it, all you need to do is: java -jar stringstat.jar /path/to/your/neo4j/store/dir ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neos4j and ECM
The content tree can be made out of CHILD relationships, content items can be mapped to nodes of a certain type (e.g. folder, file, image, pages, etc.), content item properties like name, date and type-specific data could be mapped to node properties. File operations (add, move, copy, delete) alter relationships and create/delete nodes. Access control can be realized with the ACL pattern [1] as described in the Neo4j wiki). IMHO, every ECM use case can be easily modeled with a graph DB. Axel [1] http://wiki.neo4j.org/content/ACL Am 20.01.2011 20:50, schrieb kyle adams: How would you translate a content model and the processes around that content model to the graph DB design? On Jan 20, 2011 2:46 PM, Axel Morgnera...@morgner.de wrote: Hi Kyle, currently I'm working on a ECM solution based on Neo4j which will be released as open source soon. Do you have any specific questions? Greetings Axel Am 20.01.2011 19:38, schrieb kyle adams: Neos4j is very intriguing, but I'm not sure how it would fit into existing (or new for that matter) ECM use cases. Any ideas? -- *Kyle * ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUI UX Design - Project Management c/o inxire GmbH Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 E-mail a...@morgner.de Web http://www.morgner.de ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neos4j and ECM
Hi Rick, I'm storing binary content in the file system and keep a pointer to the file (url) as a node property. Greetings Axel On 21.01.2011 00:00, Rick Bullotta wrote: Axel, are you storing the actual content in Neo or in the file system directly (or some other datastore)? I've been told not to use Neo for blob/large text storage. Curious what approach you're taking. Best, Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Axel Morgner Sent: Thursday, January 20, 2011 3:15 PM To: user@lists.neo4j.org Subject: Re: [Neo4j] Neos4j and ECM The content tree can be made out of CHILD relationships, content items can be mapped to nodes of a certain type (e.g. folder, file, image, pages, etc.), content item properties like name, date and type-specific data could be mapped to node properties. File operations (add, move, copy, delete) alter relationships and create/delete nodes. Access control can be realized with the ACL pattern [1] as described in the Neo4j wiki). IMHO, every ECM use case can be easily modeled with a graph DB. Axel [1] http://wiki.neo4j.org/content/ACL Am 20.01.2011 20:50, schrieb kyle adams: How would you translate a content model and the processes around that content model to the graph DB design? On Jan 20, 2011 2:46 PM, Axel Morgnera...@morgner.de wrote: Hi Kyle, currently I'm working on a ECM solution based on Neo4j which will be released as open source soon. Do you have any specific questions? Greetings Axel Am 20.01.2011 19:38, schrieb kyle adams: Neos4j is very intriguing, but I'm not sure how it would fit into existing (or new for that matter) ECM use cases. Any ideas? -- *Kyle * ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Loading MySQL dump
Hello Abhi,
you may have a look at the SQL Importer [1].
If you need more control over the import mechanism, and you have MySQL
tables as XML (f.e. using mysqladmin export), you may use dom4j, Commons
StringUtils and Java to parse it.
Within a Neo4j transaction, read the XML document, e.g. like this:
SAXReader xmlReader = new SAXReader();
try {
Document xmlDoc = xmlReader.read(filename);
Element root = xmlDoc.getRootElement();
ListElement rootChildren = root.elements();
// Then iterate through child nodes
for (Element e : rootChildren) {
// Create a node
Node node = graphDb.createNode();
String p = e.elementTextTrim(tagName);
if (StringUtils.isNotBlank(p)) { // optional
node.setProperty(propertyKey, p);
}
[...]
} catch (DocumentException ex) {
[...]
}
Depending of your data model (start here: [2], you may create a node
structure with some major nodes (= tables) before adding sub nodes (= rows).
If you got large dumps to import, you may use the Neo4j BatchInserter [3].
Greetings
Axel
[1] http://wiki.neo4j.org/content/SQL_Importer
[2] http://wiki.neo4j.org/content/Domain_Modeling_Gallery
[3] http://wiki.neo4j.org/content/Batch_Insert
On 18.12.2010 05:22, Abhi wrote:
Hello,
I have a huge corpus of MySQL dump files of life sciences data which I would
like to load into Neo4j. Is there an importer available for this? If not,
what is the standard way to go about this problem? Is there any wiki or
links for guidance?
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] NoClassDefFoundError: org/neo4j/kernel/impl/index/IndexStore
Yes, that's I figured out just a second ago. :-) I'll try both options. Thanks On 26.11.2010 10:31, Mattias Persson wrote: It may be that neo4j-spatial 0.3-SNAPSHOT depends on a later kernel/index version (probably 1.2-SNAPSHOT, i.e. latest trunk). In that case there's a conflict. Could you instead depend on a stable spatial version or upgrade to latest kernel/index versions? 2010/11/26 Axel Morgnera...@morgner.de Hi, since yesterday late evening I'm getting an exception when doing GraphDatabaseServicegraphDb = new EmbeddedGraphDatabase(dbPath): Could not create data source [lucene-index], see nested exception for cause of error java.lang.NoClassDefFoundError: org/neo4j/kernel/impl/index/IndexStore This is my neo4j part of pom.xml: dependency groupIdorg.neo4j/groupId artifactIdneo4j-kernel/artifactId version1.1/version /dependency dependency groupIdorg.neo4j/groupId artifactIdneo4j-index/artifactId version1.1/version typejar/type optionalfalse/optional /dependency dependency groupIdorg.neo4j/groupId artifactIdneo4j-spatial/artifactId version0.3-SNAPSHOT/version /dependency Was there an API change lately? Regards Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] NoClassDefFoundError: org/neo4j/kernel/impl/index/IndexStore
Ok, got everything working again, running on 1.2-SNAPSHOT now. Quite faster than 1.1, BTW. :-) Just in case: Is there a neo4j-spatial version which works well with neo4j-1.1? Thanks Axel On 26.11.2010 10:40, Anders Nawroth wrote: Hi! 2010-11-26 10:31, Mattias Persson skrev: It may be that neo4j-spatial 0.3-SNAPSHOT depends on a later kernel/index version (probably 1.2-SNAPSHOT, i.e. latest trunk). That's exactly how it is according to the pom.xml: https://github.com/neo4j/neo4j-spatial/blob/master/pom.xml /anders In that case there's a conflict. Could you instead depend on a stable spatial version or upgrade to latest kernel/index versions? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Spatial question
Currently, I'm still busy with a customer project, but when open sourcing the software end of the year, I'll document it. Am 16.11.2010 00:01, schrieb Peter Neubauer: Axel, nice it worked out for you! Would be great if you could write up a small blog or so on this, and give us some feedback on what to improve to make the component more usable. WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Nov 15, 2010 at 10:03 PM, Axel Morgnera...@morgner.de wrote: On 02.11.2010 14:52, Craig Taverner wrote: Adding neo4j spatial to the picture should enable the CMS handling geo data objects without adding to much complexity (I hope). OK. I think I'm beginning to get an idea of what you want. You have a CMS, and some of the data is geolocated, in the sense that you know what countries it is associated with, and perhaps you even have relationships to 'country' nodes (or other nodes) representing locations. Now you want to use Neo4j Spatial to perform simple spatial queries on this. Your analysis is correct and brilliant. :-) In my case, the CMS payload data itself (like HTML, Images, Videos, CSS, JS, Blogs etc.) is stored in neo4j nodes. But logging data, too, of course. I could imagine something like: 'give me all countries in the specified region (bounding box) and include the number of pages as an attribute of the result set'bute? Right now I'm using the SearchIntersectWindow class to determine which countries are needed to display within a certain global region. The countries data is loaded as a set of GeoJSON files via Javascript (using Polymaps), so neo4j spatial does only provide the country names. I think you should either use the original shapefiles and the existing ShapefileImporter or move to OSM data models for connected topologies. A GeoJSON importer is a nice idea, but not worth the effort if you already have shapefiles. Indeed! I ended up with using the shapefile importer, using neo4j spatial to serve geodata nodes which are then connected on-the-fly with geo-tagged CMS data. Your comments and suggestions helped me a lot, so I just wanted to say thanks again. Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Spatial question
On 02.11.2010 14:52, Craig Taverner wrote: Adding neo4j spatial to the picture should enable the CMS handling geo data objects without adding to much complexity (I hope). OK. I think I'm beginning to get an idea of what you want. You have a CMS, and some of the data is geolocated, in the sense that you know what countries it is associated with, and perhaps you even have relationships to 'country' nodes (or other nodes) representing locations. Now you want to use Neo4j Spatial to perform simple spatial queries on this. Your analysis is correct and brilliant. :-) In my case, the CMS payload data itself (like HTML, Images, Videos, CSS, JS, Blogs etc.) is stored in neo4j nodes. But logging data, too, of course. I could imagine something like: 'give me all countries in the specified region (bounding box) and include the number of pages as an attribute of the result set'bute? Right now I'm using the SearchIntersectWindow class to determine which countries are needed to display within a certain global region. The countries data is loaded as a set of GeoJSON files via Javascript (using Polymaps), so neo4j spatial does only provide the country names. I think you should either use the original shapefiles and the existing ShapefileImporter or move to OSM data models for connected topologies. A GeoJSON importer is a nice idea, but not worth the effort if you already have shapefiles. Indeed! I ended up with using the shapefile importer, using neo4j spatial to serve geodata nodes which are then connected on-the-fly with geo-tagged CMS data. Your comments and suggestions helped me a lot, so I just wanted to say thanks again. Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] database file size after removal of many nodes
Hi Andrés,
In regards to your database size problem, unfortunately, I can't offer a
clean and easy solution. The store files grow as more data is added, but
doesn't shrink when you remove data. There are technical reasons for this,
but for you, it means that you either have to live with a database that is
larger than it needs to be, or you have to manually copy the data to a new
database, and re-index everything.
to cope with the size problem, I recently wrote a little database copy
tool that I like to share [1]. It's not tested thoroughly (please
anybody handle with care .. make backups etc.) and I don't have any
long-term experience with the resulting database copy.
So there's pretty much room for improvement (f.e. copy nodes and
relationships while traversing, supporting partial graphs), but for me
it does the job of shrinking the database. Maybe this is useful for
somebody else, too.
Note: The code doesn't re-index anything, this has to be added based on
anybody's individual indexing strategy.
Could you think of adding such a thing to neo4j? That would f.e. remove
the restriction of copying all the database files to a new location
before the first online backup.
Some oddity I stumbled upon: I was able to create a node and set the id
to 0, so that this node hid the reference node which was isolated then,
resulting in a non-functional database. If I remember correctly my
tests, the respective node had two id properties, one named Id (value
1) and one with id (value 0). It is reproducable with my code, if you
remove the test for sourceId == 0L.
Greetings
Axel
P.S.: Sorry to post the code here, but our svn repos is not public yet.
[1] Code snipplet: Copy all nodes and relationships from a neo4j source
database located at sourceDbPath to a target database at targetDbPath
GraphDatabaseService sourceDb = new
EmbeddedReadOnlyGraphDatabase(sourceDbPath);
GraphDatabaseService targetDb = new
EmbeddedGraphDatabase(targetDbPath);
Transaction targetTx = targetDb.beginTx();
long counter = 0L;
// List to store all relationships of source database
ListRelationship allRels = new ArrayListRelationship();
// Mapping target node id (vale) to source node id (value)
MapLong, Long idMap = new HashMapLong, Long();
// Iterate over all nodes of the source database
IterableNode allNodes = sourceDb.getAllNodes();
try {
for (Node sourceNode : allNodes) {
// Store any outgoing relationship
for (Relationship r :
sourceNode.getRelationships(Direction.OUTGOING)) {
allRels.add(r);
}
long sourceId = sourceNode.getId();
Node targetNode = null;
// Don't create a node with id 0 in the target database!
// If you do, the reference node of the target database will be
hidden.
if (sourceId == 0L) {
targetNode = targetDb.getReferenceNode();
} else {
targetNode = targetDb.createNode();
}
long targetId = targetNode.getId();
// Store target node id in map with source node id as key
idMap.put(sourceId, targetId);
// Copy properties
for (String key : sourceNode.getPropertyKeys()) {
targetNode.setProperty(key, sourceNode.getProperty(key));
}
counter++;
// Commit to disk each 1,000 nodes
if (counter % 1000 == 0) {
targetTx.success();
targetTx.finish();
targetTx = targetDb.beginTx();
}
}
targetTx.success();
} finally {
targetTx.finish();
}
// Now copy all relationships
targetTx = targetDb.beginTx();
long relCounter = 0L;
try {
for (Relationship r : allRels) {
// Create relationship in target database
long sourceStartNodeId = r.getStartNode().getId();
long sourceEndNodeId = r.getEndNode().getId();
long targetStartNodeId = (long)
idMap.get(sourceStartNodeId);
long targetEndNodeId = (long) idMap.get(sourceEndNodeId);
Node s = targetDb.getNodeById(targetStartNodeId);
Node e = targetDb.getNodeById(targetEndNodeId);
s.createRelationshipTo(e, r.getType());
relCounter++;
// Commit to disk each 1,000 relationships
if (relCounter % 1000 == 0) {
targetTx.success();
targetTx.finish();
targetTx = targetDb.beginTx();
}
}
targetTx.success();
} finally {
targetTx.finish();
}
targetDb.shutdown();
sourceDb.shutdown();
Re: [Neo4j] Neo4j Spatial question
Hi Peter, thanks a lot, yes, I'm currently working on it and I'll give feedback as soon as I've accomlished something useful. Axel Am 01.11.2010 15:52, schrieb Peter Neubauer: Axel, I was pointed to http://svn.osgeo.org/geotools/trunk/modules/unsupported/geojson/src/test/java/org/geotools/geojson/FeatureJSONTest.java which has an example of a FeatureCollection reading. Does that help you? /peter On Wed, Oct 27, 2010 at 8:59 PM, Axel Morgnera...@morgner.de wrote: Hi, a part of my current project is about rendering an abstract world map (countries filled with a color gradient). I'm using Polypmaps on a 10 MB geoJSON file with high-res vector data, containing 246 features defining all the world's countries as multi-polygons (about 1.5 million points overall). Now even when the map is zoomed in, the hole file is processed instead of a small set of countries. Polymaps renders SVG path elements for each request, and rendering the whole map is by far too slow (4-5 s even on a fast machine). My idea is to store all features in Neo4j and determine only a subset of individual countries to process for a given map section programmatically. There's already a neo4j instance containing all CMS data, so this approach seems natural to me. With Polymaps I can easily get the coordinates (lat,lon) of the box corners. Now I only have to get all points within that box and get their corresponding country to be loaded and rendered. Can Neo4j spatial support me here? F.e. is there a geoJSON importer? And I saw different search classes in org.neo4j.gis.spatial.query, could that help? Or is there a better approach at all? Thank you in advance Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUI UX Design - Project Management c/o inxire GmbH Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 E-mail a...@morgner.de Web http://www.morgner.de ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Spatial question
Hi Craig, first, thanks a lot for your comprehensive and helpful answer with a number of useful thoughts in it. Your original description says two things of interest, there is a GeoJSON document for all countries (as multi-polygons) and there is an existing neo4j database of all CMS data. I'm not sure what you mean by CMS data? Is this an unrelated content management system, and so you are only saying that you have exerience with neo4j in general, or is this data you wish to use in a spatial context (together with the world map)? The CMS part is the core of the project containing pages, images and other entities needed for etc.. The content tree is stored as nodes and relationships in neo4j, so yes, there's some experience with neo4j. The CMS uses Freemarker as template engine and XPath queries to access nodes. Adding neo4j spatial to the picture should enable the CMS handling geo data objects without adding to much complexity (I hope). In the former case, I guess we are only talking about a GeoJSON importer, in the later we are talking about using Neo4j Spatial to expose spatial information in your existing database, which is a good use case for Neo4j Spatial, so would be good to do. Either way, I think the job is not too big. For the GeoJSON importer, you could write a generic importer supporting the full GeoJSON spec, or you could write one specific to your case of only processing multi-polygons for countries. For the generic importer I'm not sure that my GIS knowledge is sufficient, so probably I'll start with the multi-polygon. Another factor to consider, if you model your importer on the code in the shapefile importer, you will create a database structure somewhat reminiscent of postgis, with each geometry saved as a WKB blob in a node of the database. In this case the primary use of Neo4j is for the RTree index. However, I personally think it would be nicer (better) to store the world-map as a single connected graph, so that countries that share boarders actually share the nodes along those boarders. Yes, I prefer a connected graph, too. So something like country1.isNeighboursOf(country2) or getNeighbours(country) would be easy. Our OSMImporter for Open Street Map maintains the OSM graph in Neo4j, as well as exposes individual geometries for those tools that require them (eg. renderers). You could do a similar thing if you had a data source that contained that topology. Normal GeoJSON, however, does not, since it describes each feature or geometry as a separate object, with no spatial relationship to any other object. Is GeoJSON the main data source, or can you get a more topologically connected data source from polymaps? The GeoJSON files are generated with the OGR Layer Converter, and the original data is a shapefile from http://thematicmapping.org/downloads/world_borders.php (TM_WORLD_BORDERS-0.3.zip http://thematicmapping.org/downloads/TM_WORLD_BORDERS-0.3.zip). Does this file contain topologically connected structures? So, to summarize the options: - Write a plain GeoJSON importer that imports to individual WKB geometries - very simple, could even base on a raw JSON library and write a small mapping layer for your Geometries using JTS, or you could use a GeoJSON library like the GeoTools one already suggested - Write your own new data model, maintaining connectivity to your other data (if that exists) and a 'better' world map graph. - If you do not have access to a fully connected world map of countries, perhaps using OSM data for this is an alternative to your current data source. These all are good suggestions that I will consider. Maybe it's another option to use the original shapefile for import and just build a GeoJSON exporter. I hope I'll figure it out this week .. Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Spatial question
Thanks a lot, I'll give it a try soon. Am 29.10.2010 08:58, schrieb Jan Boonen: Hi Axel, The FeatureJSON class of GeoTools (http://docs.geotools.org/latest/javadocs/org/geotools/geojson/feature/FeatureJSON.html) should be able to do the work. Look at the code example in the Javadocs to see if this is what you need. In order to connect it to neo4j, you could use the code from the Shapefile importer. Cheers, Jan On 28-10-2010 17:24, Peter Neubauer wrote: some example on reading in a FeatureColllection from GeoJSON, but the ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUI UX Design - Project Management c/o inxire GmbH Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 E-mail a...@morgner.de Web http://morgner.de ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Moving a DB from one machine to another...
Hi Chris, moving a neo4j db as an archived directory between different machines is fine. Normally, there's no need to remove any files from within the neo4j directory. Just make sure to archive the db while it's completely shut down. And take care of the access rights the files and the containing directory have. I had some issues with that when moving between a Windows and a Linux machine. Personally, I use 7-zip [1] on Windows and p7zip [2] on Unix as it has the highest compression rates. In addition, there might be other ways to import/export data, f.e. the KML export contained in the webadmin tool. Axel [1] http://www.7-zip.org/ [2] http://p7zip.sourceforge.net/ On Fri, 29 Oct 2010 12:02:02 -0700, Chris Diehl cpdi...@gmail.com wrote: Hi All, I have a neo4j DB on my laptop that I need to share copies of with others. As a first shot I tar'ed and ziped up the contents of the DB directory and gave it to my colleague (MacBook Pro - Windows laptop). Initially he got errors when trying to access it. So on a whim, he deleted all the files with 'log' in their name. Then, after some complaints about needing to recover, all seemed well. I'm wondering if there is a more principled way of moving a DB from one machine to another short of this hack or going through a GraphML export/import sequence. Any suggestions? Chris ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Moving a DB from one machine to another...
GraphML, not KMK, of course. Maybe I wrote this because I'm looking for KML and geoJSON export/import for neo4j data myself. On Fri, 29 Oct 2010 21:49:04 +0200, Peter Neubauer peter.neuba...@neotechnology.com wrote: Hi there, yes, the Neo4j Admin tool (we hope to package it in the next milestone) includes GraphML export/import. Besides, Alex is right, file copy on a clean shut down DB works just fine. For hot production use, you can choose Online Backup (http://wiki.neo4j.org/content/Online_Backup) or the upcoming HA to several slaves (part of the next milestone, too, still not totally finished testing). Would anything of this work for you? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Oct 29, 2010 at 9:18 PM, Axel Morgner a...@morgner.de wrote: Hi Chris, moving a neo4j db as an archived directory between different machines is fine. Normally, there's no need to remove any files from within the neo4j directory. Just make sure to archive the db while it's completely shut down. And take care of the access rights the files and the containing directory have. I had some issues with that when moving between a Windows and a Linux machine. Personally, I use 7-zip [1] on Windows and p7zip [2] on Unix as it has the highest compression rates. In addition, there might be other ways to import/export data, f.e. the KML export contained in the webadmin tool. Axel [1] http://www.7-zip.org/ [2] http://p7zip.sourceforge.net/ On Fri, 29 Oct 2010 12:02:02 -0700, Chris Diehl cpdi...@gmail.com wrote: Hi All, I have a neo4j DB on my laptop that I need to share copies of with others. As a first shot I tar'ed and ziped up the contents of the DB directory and gave it to my colleague (MacBook Pro - Windows laptop). Initially he got errors when trying to access it. So on a whim, he deleted all the files with 'log' in their name. Then, after some complaints about needing to recover, all seemed well. I'm wondering if there is a more principled way of moving a DB from one machine to another short of this hack or going through a GraphML export/import sequence. Any suggestions? Chris ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Good Linux/Windows/OS X install procedure?
+1 for an Ubuntu personal package archive: https://launchpad.net/ubuntu/+ppas Am 28.10.2010 14:49, schrieb Richard Jones: On Ubunto I would love to see an apt-get install procedure. I am almost ready to decide on our DB and this would makie it easy to get up and running and trial the technology rather than learn how it fits together. Cheers Rich On Thu, Oct 28, 2010 at 11:27 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Hi folks, while thinking about getting a first Neo4j server package out, is there anyone who has good experience with installers? Basically, the question is if we should just zip together the distribution, so you can run the startup scripts etc yourself, or do an installer that will let you choose where to put the files, maybe install a few daemon scripts etc. Any hints and opinions on what tools to use and how this should look like? Have you seen any other project that you think has a great installing experience? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4j Spatial question
Hi, a part of my current project is about rendering an abstract world map (countries filled with a color gradient). I'm using Polypmaps on a 10 MB geoJSON file with high-res vector data, containing 246 features defining all the world's countries as multi-polygons (about 1.5 million points overall). Now even when the map is zoomed in, the hole file is processed instead of a small set of countries. Polymaps renders SVG path elements for each request, and rendering the whole map is by far too slow (4-5 s even on a fast machine). My idea is to store all features in Neo4j and determine only a subset of individual countries to process for a given map section programmatically. There's already a neo4j instance containing all CMS data, so this approach seems natural to me. With Polymaps I can easily get the coordinates (lat,lon) of the box corners. Now I only have to get all points within that box and get their corresponding country to be loaded and rendered. Can Neo4j spatial support me here? F.e. is there a geoJSON importer? And I saw different search classes in org.neo4j.gis.spatial.query, could that help? Or is there a better approach at all? Thank you in advance Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Spatial question
All I found is this, but didn't tested any of them: http://jira.codehaus.org/browse/GEOT-1924 http://www.mapfish.org/svn/mapfish/contribs/java-geojson/trunk/src/main/java/org/mapfish/geo/ Am 27.10.2010 23:29, schrieb Peter Neubauer: Alex, I can't find any good code example to parse the GeoJSON with GeoTools, asking on the lists. After that, it should be much the same as the ShapefileImporter, http://github.com/neo4j/neo4j-spatial/blob/master/src/main/java/org/neo4j/gis/spatial/ShapefileImporter.java . Just need some advice. Craig, Davide - any skillz with that? Cheers, /peter neubauer VP Product Management, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Oct 27, 2010 at 9:39 PM, Axel Morgnera...@morgner.de wrote: Sure, I attached an archive with some countries. But ..attachments are filtered by the list, aren't they? Am 27.10.2010 21:27, schrieb Peter Neubauer: Axel, do you have a small sample of that data? It might be very useful to write a small GeoJSON importer and corresponding test for importing stuff into Neo4j Spatial, much like the Shapefile importer. I am thinking of using something like https://svn.osgeo.org/geotools/trunk/modules/unsupported/geojson/src/test/java/org/geotools/geojson/GeometryJSONTest.java if you have some sample data? Cheers, /peter neubauer VP Product Management, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Oct 27, 2010 at 8:59 PM, Axel Morgnera...@morgner.dewrote: Hi, a part of my current project is about rendering an abstract world map (countries filled with a color gradient). I'm using Polypmaps on a 10 MB geoJSON file with high-res vector data, containing 246 features defining all the world's countries as multi-polygons (about 1.5 million points overall). Now even when the map is zoomed in, the hole file is processed instead of a small set of countries. Polymaps renders SVG path elements for each request, and rendering the whole map is by far too slow (4-5 s even on a fast machine). My idea is to store all features in Neo4j and determine only a subset of individual countries to process for a given map section programmatically. There's already a neo4j instance containing all CMS data, so this approach seems natural to me. With Polymaps I can easily get the coordinates (lat,lon) of the box corners. Now I only have to get all points within that box and get their corresponding country to be loaded and rendered. Can Neo4j spatial support me here? F.e. is there a geoJSON importer? And I saw different search classes in org.neo4j.gis.spatial.query, could that help? Or is there a better approach at all? Thank you in advance Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUIUX Design - Project Management c/o inxire GmbH Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 E-mail a...@morgner.de Web http://www.morgner.de ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Axel Morgner Creative Solutions - Software Engineering GUI UX Design - Project Management c/o inxire GmbH Hanauer Landstr. 293a 60314 Frankfurt Germany Phone +49 151 40522060 E-mail a...@morgner.de Web http://www.morgner.de ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j tutorial
You forgot tx.success() in the try{} block, see
http://wiki.neo4j.org/content/Transactions#Controlling_success.
Greetings
Axel
Am 26.10.2010 11:19, schrieb Roman Uhlig:
Hi,
I'm completely new to Neo4j and got an issue while trying the getting started
guide.
Everytime I close the database (GraphDatabaseService.shutdown()) and open it
again later, all data is gone (or was never saved). I did exactly as
described in the guide, so maybe someone can point me in the right direction?
Probably just some stupid mistake by me. ;)
My tiny test case (Windows, Java 1.6):
Writing data:
GraphDatabaseService db = new EmbeddedGraphDatabase(/temp/neo4j/test1);
try {
org.neo4j.graphdb.Transaction tx = db.beginTx();
try {
Node rootNode = db.getReferenceNode();
Node firstNode = db.createNode();
rootNode.createRelationshipTo(firstNode,
IdabaRelTypes.ROOT_TYPES);
} finally {
tx.finish();
}
} finally {
db.shutdown();
}
Reading data:
GraphDatabaseService db = new EmbeddedGraphDatabase(/temp/neo4j/test1);
try {
org.neo4j.graphdb.Transaction tx = db.beginTx();
try {
Node rootNode = db.getReferenceNode();
for (Relationship rel : rootNode.getRelationships()) {
// output test data, but nothing in it
}
} finally {
tx.finish();
}
} finally {
db.shutdown();
}
Thanks in advance,
Roman
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Axel Morgner
Creative Solutions - Software Engineering
GUI UX Design - Project Management
c/o inxire GmbH
Hanauer Landstr. 293a
60314 Frankfurt
Germany
Phone +49 151 40522060
E-mail a...@morgner.de
Web http://morgner.de
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] database file size after removal of many nodes
Hi Andrés, [...] or you have to manually copy the data to a new database, and re-index everything. ok, that's fine for me. Webadmin is lying - it's telling you how many node-id's have been allocated, not how many are in use. Understood. In use means nodes that are connected with the reference node? The number of active nodes is something that the database should be able to answer, but right now, it can not. It's not something planned to do right now, but that could change. Is it a critical feature for you? No, it isn't critical. In my use case, I have rather thousands than billions of nodes, so I can use getAllNodes(). Best regards Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] database file size after removal of many nodes
Hi, first of all, thank you Neo Technology for providing the great neo4j database, and of course you guys here on the list. After having found neo4j as the database to fit our project's requirements best, all the communication here helped me a lot. Now that our neo4j-based project comes closer to something useful, it's time to leave read-only mode and join the active discussion. Right now, we're still in stealth mode, but in about a few weeks we hope to release an application to the Open Source space. It will be sort of a CMS with some special features, using neo4j as embedded persistence store via the Java API. So here are my questions: Is there something like an automatic clean-up mechanism to shrink the database files after deletion of many nodes/rels/attributes? I tried to remove any orphaned nodes (nodes without any relationship and those nodes that aren't connected to the root node anymore), but I'm not sure about the success. And how comes that the neo4j webadmin tools still shows much more nodes than graphDb.getAllNodes()? To give numbers, the neo4j storage folder has 650 MBytes, while graphDb.getAllNodes() returns a size of 225 nodes and the webadmin tool showing something like 560.000 nodes? If anyone can give me a hint, I'd be thankful. Greetings Axel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
