Re: [Neo] Unable to memory map
Hi, I've had some time to look into this issue and it seems that when using the ReadOnly versions of the classes, I get the memory mapping warnings and when using the Writable versions of the classes, the warning does not occur (I'm assuming memory mapping gets enabled). I'm not against using the writable versions of the classes; my only concern is performance. Are the readonly versions faster that the writable versions? And if they are; then if memory mapping is not enabled, are they faster that the writable versions with memory mapping? I'll run some tests, but I guess I would like an expert opinion. Regards, Marc On Mon, Mar 22, 2010 at 10:24 AM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: Hi, We have seen this message before emitted as a warning from Neo4j. Are you seing this as a warning as well, or are you getting an exception thrown to your application code? It's hard to deal with these errors since nio only throws IOException, and not any more semantic information than that, I believe we deal with all cases by issuing a warning and then falling back to another method of performing the same operation, but if you are getting exceptions we need to resolve it. If you are indeed getting exceptions, some code that triggers it would be very helpful. Cheers, Tobias On Wed, Mar 17, 2010 at 1:47 PM, Marc Preddie mpred...@gmail.com wrote: Hi, I've look at the mailing list and found 1 similar situation, but no real solution. So I was hoping someone could shed some light on this. I seem to have an issue with neo4j being able to use memory mapped files. I've run my service on Win XP 64bit, Mac OSX Snow Leopard 10.6.2 and Centos 5.x 64bit and always get the same error when launching. I'm using APOC 1.0 and have a DB of approx 600M. In my neo config I allocate about 5M more for each type of file than the actual file size (I've tried multiple different settings). On each machine I also leave at least 1.5G for the OS and have at least 2.5G heap for the Java process. I'm also using the classes EmbeddedReadOnlyGraphDatabase and LuceneReadOnlyIndexService to access and browse DB. Neo config neostore.nodestore.db.mapped_memory=10M neostore.relationshipstore.db.mapped_memory=110M neostore.propertystore.db.mapped_memory=85M neostore.propertystore.db.index.mapped_memory=10M neostore.propertystore.db.index.keys.mapped_memory=10M neostore.propertystore.db.strings.mapped_memory=320M neostore.propertystore.db.arrays.mapped_memory=10M Here is the error org.neo4j.kernel.impl.nioneo.store.MappedMemException: Unable to map pos=3005872 recordSize=33 totalSize=1153416 at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.init(MappedPersistenceWindow.java:59) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.allocateNewWindow(PersistenceWindowPool.java:530) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.refreshBricks(PersistenceWindowPool.java:430) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:122) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:459) at org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:248) at org.neo4j.kernel.impl.nioneo.xa.NeoReadTransaction.getMoreRelationships(NeoReadTransaction.java:103) at org.neo4j.kernel.impl.nioneo.xa.NioNeoDbPersistenceSource$ReadOnlyResourceConnection.getMoreRelationships(NioNeoDbPersistenceSource.java:275) at org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:93) at org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:585) at org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:332) at org.neo4j.kernel.impl.core.NodeImpl.ensureFullRelationships(NodeImpl.java:320) at org.neo4j.kernel.impl.core.NodeImpl.getAllRelationshipsOfType(NodeImpl.java:129) at org.neo4j.kernel.impl.core.NodeImpl.getSingleRelationship(NodeImpl.java:179) at org.neo4j.kernel.impl.core.NodeProxy.getSingleRelationship(NodeProxy.java:98) Caused by: java.io.IOException: Access is denied at sun.nio.ch.FileChannelImpl.truncate0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:728) at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.init(MappedPersistenceWindow.java:53) ... 78 more Regards Marc ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] Unable to memory map
Hi, Previous to the 1.0 release, there was just a warning message, now with the 1.0 release there is a warning message, along with an exception describing the warning. Everything still works, but we would really like and need the performance boost that memory mapping may provide. I'll try to get a small sample (code and db) to you in the next day or so. Regards, Marc On Mon, Mar 22, 2010 at 10:24 AM, Tobias Ivarsson tobias.ivars...@neotechnology.com wrote: Hi, We have seen this message before emitted as a warning from Neo4j. Are you seing this as a warning as well, or are you getting an exception thrown to your application code? It's hard to deal with these errors since nio only throws IOException, and not any more semantic information than that, I believe we deal with all cases by issuing a warning and then falling back to another method of performing the same operation, but if you are getting exceptions we need to resolve it. If you are indeed getting exceptions, some code that triggers it would be very helpful. Cheers, Tobias On Wed, Mar 17, 2010 at 1:47 PM, Marc Preddie mpred...@gmail.com wrote: Hi, I've look at the mailing list and found 1 similar situation, but no real solution. So I was hoping someone could shed some light on this. I seem to have an issue with neo4j being able to use memory mapped files. I've run my service on Win XP 64bit, Mac OSX Snow Leopard 10.6.2 and Centos 5.x 64bit and always get the same error when launching. I'm using APOC 1.0 and have a DB of approx 600M. In my neo config I allocate about 5M more for each type of file than the actual file size (I've tried multiple different settings). On each machine I also leave at least 1.5G for the OS and have at least 2.5G heap for the Java process. I'm also using the classes EmbeddedReadOnlyGraphDatabase and LuceneReadOnlyIndexService to access and browse DB. Neo config neostore.nodestore.db.mapped_memory=10M neostore.relationshipstore.db.mapped_memory=110M neostore.propertystore.db.mapped_memory=85M neostore.propertystore.db.index.mapped_memory=10M neostore.propertystore.db.index.keys.mapped_memory=10M neostore.propertystore.db.strings.mapped_memory=320M neostore.propertystore.db.arrays.mapped_memory=10M Here is the error org.neo4j.kernel.impl.nioneo.store.MappedMemException: Unable to map pos=3005872 recordSize=33 totalSize=1153416 at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.init(MappedPersistenceWindow.java:59) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.allocateNewWindow(PersistenceWindowPool.java:530) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.refreshBricks(PersistenceWindowPool.java:430) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:122) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:459) at org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:248) at org.neo4j.kernel.impl.nioneo.xa.NeoReadTransaction.getMoreRelationships(NeoReadTransaction.java:103) at org.neo4j.kernel.impl.nioneo.xa.NioNeoDbPersistenceSource$ReadOnlyResourceConnection.getMoreRelationships(NioNeoDbPersistenceSource.java:275) at org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:93) at org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:585) at org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:332) at org.neo4j.kernel.impl.core.NodeImpl.ensureFullRelationships(NodeImpl.java:320) at org.neo4j.kernel.impl.core.NodeImpl.getAllRelationshipsOfType(NodeImpl.java:129) at org.neo4j.kernel.impl.core.NodeImpl.getSingleRelationship(NodeImpl.java:179) at org.neo4j.kernel.impl.core.NodeProxy.getSingleRelationship(NodeProxy.java:98) Caused by: java.io.IOException: Access is denied at sun.nio.ch.FileChannelImpl.truncate0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:728) at org.neo4j.kernel.impl.nioneo.store.MappedPersistenceWindow.init(MappedPersistenceWindow.java:53) ... 78 more Regards Marc ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Tobias Ivarsson tobias.ivars...@neotechnology.com Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] threading
Hi Johan, No, I am strictly reading from the graph. I have single read only instance which I am attempting to use across multiple threads as you suggest. Regards, Marc On Sat, Oct 10, 2009 at 9:23 AM, Johan Svensson jo...@neotechnology.comwrote: Hi, Do you have a NeoService writing to the store concurrently with the embedded read only ones? Regarding the other question you should use one NeoService (instance) per machine/across multiple threads. The read only instance is more suitable when your graph does not change and you need to load balance reads over multiple machines. Regards, -Johan On Thu, Oct 8, 2009 at 11:52 PM, Marc Preddie mpred...@gmail.com wrote: Hi, When using multiple threads if I use normal (non readonly) neo and lucene index everything works perfectly. If I use the readonly versions of neo and luceneIndex; I begin to get errors of the sort --- More than one relationship[BELONGS_TO_CATEGORY, OUTGOING] found for NodeImpl#149816 --- from ... NodeImpl.getSingleRelationship. If I use the readonly versions from a single thread everything works. Has anyone come across this issue? I'm using build 3108. One other question I have is would it better to use a readonly neo/index instance per thread or use a single neo/index across multiple threads? Regards, Marc ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo] neo4j with terracotta?
Hi Peter, Initially simply looking to replicate for high availability. Regards, Marc On Sun, Aug 16, 2009 at 2:01 PM, Peter Neubauer neubauer.pe...@gmail.comwrote: Mmh, have not heard anyone doing that, what kind of scaling are you imagining? Replicating a nodespace in a high availability scenario, sharding or partitioning? /peter GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org- New Energy for Data - The Graph Database. http://www.ops4j.org- New Energy for OSS Communities - Open Participation Software. http://www.linkedprocess.org - Distributed computing on LinkedData scale On Fri, Aug 14, 2009 at 7:54 PM, Marc Preddiempred...@gmail.com wrote: Hi All, I was just wondering if any users of neo4j has had to opportunity to integrate it with terracotta? Regards, Marc Preddie ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo] neo4j with terracotta?
Hi All, I was just wondering if any users of neo4j has had to opportunity to integrate it with terracotta? Regards, Marc Preddie ___ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
