Hi, That sounds interesting. Could some of the the code be implemented in the file system abstraction? Maybe using two write methods (writeAsync and write) or similar.
Regards, Thomas On Sat, Jul 7, 2012 at 2:32 PM, Steve McLeod <[email protected]> wrote: > Noel, > > This approach sounds promising: > >> (4) Use asynchronous IO to speed up the overwriting of the page headers - >> at the moment we perform the operation sequentially, which is hideously slow >> on modern hardware, because it waits for each IO to complete before >> performing the next one. > > That is, if it is okay with Thomas to have some "if (isJava7) {} else {}" > type code. > > I might experiment a little with this in the coming days. > > > > > On Friday, 6 July 2012 12:25:30 UTC+4, Noel Grandin wrote: >> >> Ah, yes, now I remember. >> >> So there are a variety of ways to tackle this. Of the top of my head I >> have: >> >> (1) Sacrifice some recovery resistance by not overwriting the headers of >> the pages. >> >> (2) Change the on-disk format to put headers together, making it quicker >> to overwrite them. But that would sacrifice performance for normal >> operations because it would increase the number of IO's performed. >> >> (3) Do a hybrid approach where we don't immediately overwrite the page >> headers, but use a lower priority background thread to gradually overwrite >> the page headers. >> >> (4) Use asynchronous IO to speed up the overwriting of the page headers - >> at the moment we perform the operation sequentially, which is hideously slow >> on modern hardware, because it waits for each IO to complete before >> performing the next one. >> >> The option that most interests me is option (4). We could extend the >> FileStore interface to have a method >> public void writeToMultiple(byte[] data, int[] offsets, int len) >> which would use asynchronous IO to push a bunch of writes to the disk. >> >> Unfortunately, asynch disk IO (java.nio.channels.AsynchronousFileChannel) >> is only available for Java7. >> So we have 2 options - emulate it using memory mapped files and the >> FileChannel#force() method, or simply only enable it when the code is >> running under Java7. >> >> >> On 2012-07-05 20:37, Steve McLeod wrote: >> >> I looked back at my work on this some months ago, and actually it was with >> TRUNCATE TABLE that I did my investigation. You can read the discussion >> about this here: >> https://groups.google.com/forum/?fromgroups#!topic/h2-database/jUqGLVL1FeE >> >> I posted there the following: >> >> This line is the one consuming the time: >> >> file.readFully(test, 0, 16); >> >> which is org.h2.store.PageStore.java: line 451 in the current SVN trunk. >> >> >> On Thursday, 5 July 2012 at 8:19 PM, Noel Grandin wrote: >> >> >> On 2012-07-05 12:25, Steve McLeod wrote: >> >> I've also experienced this slowness when dropping a large table. I >> spent a considerable amount of time with the H2 source code trying to >> find a way to speed things up, but alas it turns out not to be an easy >> task with the current data store. >> >> Hmm, you're right, that code path is pretty deep and winding. >> >> starts here >> DropTable#executeDrop() >> which calls >> Database#removeSchemaObject(...) >> which calls >> DbObject#removeChildrenAndResources(Session) >> which means it's actually calling >> RegularTable#removeChildrenAndResources(Session) >> which calls >> Index#remove(Session) >> which means it's actually calling >> PageDataIndex#remove(Session) >> which calls >> PageDataIndex#removeAllRows() >> which calls >> PageData#freeRecursive() >> >> Can you run a profiler across the code and see where in this call stack >> it is spending the bulk of it's time? >> >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "H2 Database" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/h2-database/-/aE_96rLMMDMJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/h2-database?hl=en. -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/h2-database?hl=en.
