Raymond Raymond wrote:
Dear Oystein,
In your mail, "Derby I/O issues during checkpointing", you wrote:
OO: Some tests runs we have done show very long transaction response times
OO: during checkpointing. This has been seen on several platforms. The
OO: load is TPC-B like transactions and the write cache is turned off so
OO: the system is I/O bound. There seems to be two major issues:
OO:
OO: 1. Derby does checkpointing by writing all dirty pages by
OO: RandomAccessFile.write() and then do file sync when the entire
OO: cache has been scanned. When the page cache is large, the file
OO: system buffer will overflow during checkpointing, and occasionally
OO: the writes will take very long. I have observed single write
OO: operations that took almost 12 seconds. What is even worse is
that
OO: during this period also read performance on other files can be
very
OO: bad. For example, reading an index page from disk can take close
OO: to 10 seconds when the base table is checkpointed. Hence,
OO: transactions are severely slowed down.
OO:
OO: I have managed to improve response times by flushing every file
for
OO: every 100th write. Is this something we should consider including
OO: in the code? Do you have better suggestions?
Has this been implemented in the latest derby version?
No, not yet. I wanted to look at some other options first.
I am trying to
spread
out disk I/O of checkpoint over the checkpoint interval. If the
RandomAccessFile
still does sync when the entire cache has been scanned, my approach will
not
make much difference from current implementation.
I do not understand why syncing only once should affect your approach
more than syncing many times.
If it has not been
implemented
in the latest derby version, would you please attach your solution? I would
like to use it in my working copy.
I do not have access to it now. I can send it tomorrow. Basically, I
incremented a counter on every write in RAFContainer. When the counter
reached 100, I did a sync and reset the counter.
I am also curious to know, in current implementation, when will the sync
of a RandomAccessFile be executed?
I think Mike already answered this.
--
Øystein
