Øystein Grøvlen (JIRA): > Allow parallel access to data files. > ------------------------------------ > > Key: DERBY-801 > URL: http://issues.apache.org/jira/browse/DERBY-801 > Project: Derby > Type: Improvement > Components: Performance, Store > Versions: 10.0.2.0, 10.0.2.1, 10.1.1.0, 10.1.1.1, 10.1.1.2, 10.1.2.0, > 10.1.2.1 > Environment: Any > Reporter: Øystein Grøvlen > > > Derby currently serializes accesses to a data file. For example, the > implementation of RAFContainer.readPage is as follows: > > synchronized (this) { // 'this' is a FileContainer, i.e. a file object > fileData.seek(pageOffset); // fileData is a RandomAccessFile > fileData.readFully(pageData, 0, pageSize); > } > > I have experiemented with a patch where I have introduced several file > descriptors (RandomAccessFile objects) per RAFContainer. These are > used for reading. The principle is that when all readers are busy, a > readPage request will create a new reader. (There is a maximum number > of readers.) With this patch, throughput was improved by 50% on > linux. For more discussion on this, see > > http://www.nabble.com/Derby-I-O-issues-during-checkpointing-t473523.html > > The challenge with the suggested approach is to make a mechanism to > limit the number of open file descpriptors. Mike Matrigali has > suggested to use the existing CacheManager infrastructure for this > purpose. For a discussion on that, see: > > http://www.nabble.com/new-uses-for-basic-services-cache---looking-for-advice-t756863.html
I've played around a bit with a different approach - using the FileChannel class from Java 1.4's new IO API. I've written a class RAFContainer4 which extends RAFContainer and overrides the readPage and writePage methods of that class to use read/write(ByteBuffer buf, long postition) in FileChannel to access the container's file, without synchronizing on the FileContainer during the read and write calls. With a bit of hackery in BaseDataFileFactory#newContainerObject() this class is then used instead of the regular RAFContainer on creation of new RAFContainer objects when Derby runs in a 1.4+ JVM. This approach gives the JVM and OS the opportunity to issue multiple file operations concurrently, although we have no guarantees that this will actually happen. This is JVM/OS dependent, but stracing the Sun 1.4.2_09 VM on Linux 2.6 shows that the VM now uses pread64()/pwrite64() system calls instead of seek(), read() and write(). pread and pwrite have similar semantics to the FileChannel#read/write(ByteBuffer buf, long position) methods, and do not alter the file's seek() position, and are supposed to be thread safe. Of course only people running Derby on 1.4+ JVMs will have the opportunity to benefit from this approach. As support for 1.3 is to be deprecated this might not be much of an issue? But anyway, I would like to see if this hack of mine actually works. I see mentions of a "TPC-B like benchmark" in the threads Øystein links to above, and wonder if that is something Sun internal, or if it's a publicly available benchmark implementation that I can get my grubby little paws on and try out this patch with? =) Thanks, -- Anders Morken My opinions may have changed, but not the fact that I am right!
