I believe providing mode which can cause non-bootable database to gain performace is a bad idea.
Most of the users like me does not read manuals in detail and will find hard way that the database is hosed :-)
Say for example I want to use Derby to store system monitoring information with relaxed durabilty mode.
It would be ok to lose some data , but it will not be acceptable if the database can not be booted.
I think durabilty option#2 should not be available, the transaction log should be synced when a data page is forced to disk to
avoid non-recoverable datbases for sure. These kind of log syncs can be reduced by just making the page cache bigger,
and also there should be checkpoint atleast once in a while may be for a 100MB(just a randon number)
or on a boot otherwise the disk space used by the transaction log will never be released.
Thanks -suresh
Mike Matrigali wrote:
I would also like to see the in memory implementation contributed, but I think that is a different discussion.
From responses to Dan's original post on building a system with the
sync options disabled it seemed like there was enough response that
those options should be made available. I admit I am worried because
this system can no longer guarantee recoverability. It would be
interesting to know how people would use such a configuration. Obviously for a database that need not last longer than a connection
this option would work, and in memory would probably work better.
For databases that last longer than a connection, what risk of
database corruption is an acceptable trade off to better performance?
This is a new idea for me, as all databases I have worked up for now
have not provided a less than durable option.
I believe the current durability options being discussed are: 1) sync of the log file at each commit 2) sync of the log file before data page is forced to disk 3) sync of page allocation when file is grown 4) sync of data writes during checkpoint
The simple change is just to allow each sync to be disabled, maybe allowing control over all 4.
An interesting discussion is if there are changes that could be made to make it less likely to corrupt the whole database on a JVM or machine crash when the above syncs have been disabled.
I would have to think carefully and maybe some tests need to be written, but I believe the only reasone for #3 is to insure that during redo
crash recovery we don't run out of space. We actually already can run out of space for growing the log during undo crash recovery, but in that
case we just halt the boot and print that we need more space - maybe
something similar could be done for page allocation. Note that the
change is still allocating the pages it just is not syncing before inserting onto the pages.
#1 is probably the biggest performance win, in the case of very short
update transactions. Unfortunately the jdbc standard which derby implements defaults to autocommit=true - so often derby intial performance results for new users looks bad when compared to other databases which default to not syncing at commit time.
Part of this change should be to document the possible recovery failures which can result by not syncing.
/mikem
Suresh Thalamati wrote:
Sunitha Kambhampati (JIRA) wrote:
How about adding in-memory support for Derby instead of trying to confuse the users with relaxed durabilty option.Add Relaxed Durability option ------------------------------
Key: DERBY-218 URL: http://issues.apache.org/jira/browse/DERBY-218 Project: Derby Type: Improvement Components: Store Versions: 10.1.0.0 Environment: all Reporter: Sunitha Kambhampati Fix For: 10.1.0.0
Dan Debrunner posted a fix to allow for relaxed durability changes in http://article.gmane.org/gmane.comp.apache.db.derby.user/681/match=relaxed+durability
1) Need to add this option in Derby maybe as some property
2) Also from discussions on the list, Mike suggested that - that the logging system be changed to somehow
record that the database has operated in this manner, so that if
the database goes corrupt we don't waste effort trying to figure out
what when wrong. Probably need some way to mark the log records, the
log control file and write a message to the user error log file.
In this mode , will the database be recoverable to some consistent state always with possible loss of some transaction data in
case of a crash ?
-suresh
