Am 22.07.2013 22:28, schrieb Jan Kotek: > Hi, > > I have something similar in MapDB. It is in Volume.java, a light growable > abstraction ByteBuffers and FileChannel. Major difference is that Volume is > not thread safe, locking is handled at higher level. It will be interesting > to > compare those. The backend was created as a prototype for the company I'm working for and I have very high contention so I had a deep look at leaving out most positions where contention can arise.
> Any chance this backed would work for memory mapped files? In theory it should work (with little changes) for both implementations - ByteBuffer backed and Unsafe backed (last one with mapped ByteBuffers but read / write access through Unsafe). > Jan Kotek > > > > On Friday 12 July 2013 20:55:11 Christoph Engelbert wrote: >> Hey guys >> >> I finally managed to merge everything together :-) >> >> As stated a few weeks before I made a partitioned buffer system for >> good performance and less contention. >> It had different selection strategies like TLA (Thread Local >> Allocation), a simple RoundRobin or (on Linux and Windows) CLA >> (Processor Core Local Allocation), whereas the last is done using OS >> calls and JNA. >> >> It features ByteBuffers for Heap and Offheap as well as Unsafe. It >> has growing buffers (if slice is full a new one is selected) and can >> handle data bigger than Integer.MAX_VALUE (it uses full long >> position pointers). >> >> It is located in directmemory-buffer submodule since it was it's own >> project and it is fully usable even without having to use >> DirectMemory (I would suggest to give users the chance to use it on >> their own). >> >> As stated before it introduces a new dependency and especially a >> platform depending one. At least it is a optional dependency and CLA >> is deactivated if JNA is not available on the classpath. >> >> I although added 3 properties to configure the default strategy of >> creating the PartitionBufferPools: >> directmemory.buffer.pooling.disabled: true deactivates pooling and >> uses lazy creation and immediate destroying on release >> directmemory.buffer.unsafe.enabled: true activates the usage of >> sun.misc.Unsafe raw memory access (a check if unsafe is available is >> applied too) >> directmemory.buffer.offheap.enabled: true enabled DirectByteBuffer >> usage for for the non-unsafe-pools >> >> I merged it into my local fork of DirectMemory on github [1] but had >> to adjust the API of DirectMemory on some places. I introduced a >> MemoryManagerFactory which handles creation of the different >> MemoryManagers (as the old ones - partly renamed - >> UnsafeMemoryManager and AllocatorMemoryManager) and the new >> PartitionBufferMemoryManager. >> The Pointer-API is now able to use PartitionBuffers as well as the >> old way using byte[]. >> >> I'm not yet finished, working on making all unittests pass again but >> I would appreciate some opinions, discussion on the new API changes. >> >> Cheers >> Chris >> >> [1] https://github.com/noctarius/directmemory/tree/buffer
