On Wednesday, 4 February 2015 06:19:50 UTC+7, Chris Vest wrote: > > Hi Roman, > > Nice write up. One thing I noticed is that you are mixing up page cache > information from 2.2, with earlier Neo4j versions. We rewrite the page > cache from scratch in 2.2, and that’s where the 8192 byte page size was > introduced. It’s a sensitive number because it both controls the unit of IO > with the underlying storage subsystem, and it controls the coarseness of > the page locks. > Could you please point what information in the post is not valid for 2.2? I was primary basing on master source code, i. e. 2.2.
> We did consider relying on OS memory mapping, but Java will only let you > map 2 GBs in one go, so you’d have to do *something* to support files > larger than that anyway. Another issue is that freeing the native memory > held by a MappedByteBuffer requires two GCs. With our own page cache, we > are better able to reuse the allocated native memory, so we don’t have to > worry about the freeing. We also gain simpler, and more controlled failure > handling. I hadn’t seen this one coming, but it turned out to be a nice > surprise. > Mapping more than 2 GB is possible via sun.nio.ch.FileChannelImpl.map0(). Or you have requirement to run Neo4j on JVM, which doens't support this API? I was not aware about two GCs, but anyway the purpose of the whole mechanism of creating Unmapper object, adding it to ReferenceQueue of PhantomReferences, which makes resource relese not so easy, is that API suppose that ByteBuffer will be used in the user code, and resources could be released only after references to ByteBuffer are dead. Since you don't expose ByteBuffers to Neo4j users, you can deterministically control mannings directly via FileChannelImpl.map0/unmap0() calls, or ((DirectBuffer) bb).cleaner().clean(). Now I try to unserstand, is there any harm in having a lot of small (basically of a single page each) mappings, so that you can combine own page eviction heuristics and manageability with native page mapping. See http://stackoverflow.com/questions/28273878/does-linux-carry-data-structures-abstractions-for-separate-mmap-calls > > -- > Chris Vest > System Engineer, Neo Technology > [ skype: mr.chrisvest, twitter: chvest ] > > > On 03 Feb 2015, at 17:12, Roman Leventov <[email protected] <javascript:>> > wrote: > > I've done some Neo4j architecture analysis: > http://key-value-stories.blogspot.com/2015/02/neo4j-architecture.html > > Corrections and answers to open questions from Neo4j developers are > welcome. > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
