Olav Sandstaa wrote:
I have run some simple tests where I have changed mode from "rws" to "rwd" for the Derby log file. When running a small numbers of concurrent client threads the throughput is almost doubled and the response time is almost halved. I am enclosing two graphs that show this when running a given number of concurrent "tpc-b" clients. The graphs show the throughput when running with "rws" and "rwd" mode when the disk's write cache has been enabled and disabled.
I am really surprised to notice the difference between rws / rwd throughput in your graphs, Especially when the write cache is enabled. When I ran small tests long time ago, did not notice any substantial difference between rws/rwd, on windows for sure.
Could u please post more details on the hardware/OS/JVM ..etc. and What is the work done in each transaction ?
This change should also have a positive impact on the Derby startup time (DERBY-1664) and derbyall. With this change the time for running derbyall goes down by about 10-15 minutes (approximately 10%) :-)
Another interesting thing to explore here might be to, is it really necessary to flush the log for the create tables during database creation. Some time ago, when I dumped the log for the create database, only thing that I noticed was create table log records; Most of the other action are done in Un-logged mode during creation. Just for the record, these log flush is not specific to write sync mode, they use to happen when the log was synced using file sync.
I was almost tempted to disable logging for those creates, but then thought why add any extra code for the regular create table path for
database creation special case, which is suppose to happen only once :-)
Is there anyone that is aware of any issues by not updating the file meta-data for every write? Is there any recovery scenarios where this can make recovery fail? Derby seems to preallocates the log file before starting using the file, so I think this should not influence the ability to fine the last data written to the file after a power failure.
Log is not preallocated dynamically ; it happens only on a log file switch. Log file switch is triggered during log flush calls (commit/abort). If a transaction is long one, it is possible to write the log records to non-preallocation portion of a log file.
In non-preallocated log file scenario, If a meta data(Never managed to figure out what that exactly means in various operating systems) is not updated, last data written to the log file might be lost.
Thanks -suresht
