I did some benchmark tests on derby with JDBCBenchmark program.
I set the checkpoint interval as 40MB, and did tests with different
numbers of clients, 1, 10, 20.. I found when the number of clients
is more than 1, sometime derby did not do checkpint on every 40MB
log.
E.g. I tested five
I am going to subscribe to this mailing list.
Thanks.
Raymond
_
Your Space. Your Friends. Your Stories. Share your world with Windows Live
Spaces. http://spaces.live.com/?mkt=en-ca
I did some benchmark tests on derby with JDBCBenchmark program.
I set the checkpoint interval as 40MB, and did tests with different
numbers of clients, 1, 10, 20.. I found when the number of clients
is more than 1, sometime derby did not do checkpint on every 40MB
log.
E.g. I tested five
I used TPC-B workload in my tests. Do you have any idea, how can
I tune up the percentage of dirty page?
Thanks.
Raymond
Raymond Raymond wrote:
I am doing some tests on Derby checkpoint. You know, with the
same number of cache pages, if the percentage of dirty pages is
higher, deby
Hi, everyone, is there any means to set the percentage of dirty pages
in cache pages? I think derby keeps a certain percentage of dirty pages
in memory, but I don't know how to change that.
Thanks.
Raymond
_
Free Alerts : Be smart
I am doing some tests on Derby checkpoint. You know, with the
same number of cache pages, if the percentage of dirty pages is
higher, deby will dump more pages during checkpoint. I want to
see what impact it will give to Derby's performance when I tune
the percentage of dirty pages higher. As I
Hi, everyone, is there any means to set the percentage of dirty pages
in cache pages? I think derby keeps a certain percentage of dirty pages
in memory, but I don't know how to change that.
Thanks.
Raymond
_
Free Alerts : Be smart
2. I saw someone else commented on some code (in derby) said it's
not good to return from inside of a synchronized block. What
potential problem it will cause if return from inside of a
synchronized block?
I'm not sure what problems that could be. When returning from a
synchronized block,
Today, when I read the source code of
org.apache.derby.impl.store.raw.data.CachedPage.java,
I found some synchronized code like:
public boolean isDirty(){
synchronized (this) {
return isDirty || preDirty;
}
}
public boolean isActuallyDirty() {
synchronized (this) {
Today, when I read the source code of
org.apache.derby.impl.store.raw.data.CachedPage.java,
I found some synchronized code like:
public boolean isDirty(){
synchronized (this) {
return isDirty || preDirty;
}
}
public boolean isActuallyDirty() {
synchronized (this)
In the current implementation of checkpoint, derby will get
redoLWM and undoLWM before cleaning cache and get the
TransactionTable after cleaning cache. Someone can tell me
what the TransactionTable is for? Why we don't get the
TransactionTable at the same time we get the redoLWM
and undoLWM?
I am trying to implement incremental checkpoint. In incrememtal checkpoint,
we write out data when system is not very busy, and update the log control
file periodically. When we update the log control file, we need to know the
redoLWM, undoLWM and transactiontable. In current implementation, the
In current derby checkpoint implementation, a disk I/O burst
will be caused during checkpoint. To solve this problem, I am
trying to spread out the disk I/O of checkpoint over the checkpoint
interval. I found there are several things that can trigger a
checkpoint such as too much log generated
Dear Oystein,
In one of you former mails, you said:
From: [EMAIL PROTECTED] (Øystein Grøvlen)
Some tests runs we have done show very long transaction response times
during checkpointing. This has been seen on several platforms. The
load is TPC-B like transactions and the write cache is
From: Øystein Grøvlen [EMAIL PROTECTED]
Raymond Raymond wrote:
Dear Oystein,
In one of you former mails, you said:
From: [EMAIL PROTECTED] (Øystein Grøvlen)
During checkpoint when I/O is slow this creates long queques of
readers. In my run with 20 clients, I observed read
PageKey and ContainerKey are used to identify
a certain cach page or a container. Will they be
changed during the life time of a cache page or a
container?
Thanks.
Raymond
_
Scan and help eliminate destructive viruses from your
I found these two kinds of cache manager in the source
code. I am not sure what difference between containerCache
and pageCache?
Thanks.
Raymond
_
Take charge with a pop-up guard built on patented Microsoft® SmartScreen
updates on rows? Am I right?
Thanks.
Raymond
From: Bryan Pendleton [EMAIL PROTECTED]
Raymond Raymond wrote:
I found these two kinds of cache manager in the source
code. I am not sure what difference between containerCache
and pageCache?
A container is an aggregate object like a table
PageKey and ContainerKey are used to identify
a certain cach page or a container. Will they be
changed during the life time of a page or a
container?
Thanks.
Raymond
_
Powerful Parental Controls Let your child discover the best
I read the source code of
org.apache.derby.impl.services.cache.Clock.java!cleanCache(),
which writes out all of dirty pages. I got a question on the code.
The outline of the cleanCache() method is like:
protected void cleanCache(Matchable partialKey) throws StandardException {
int position;
Dear Oystein,
In your mail, Derby I/O issues during checkpointing, you wrote:
OO: Some tests runs we have done show very long transaction response times
OO: during checkpointing. This has been seen on several platforms. The
OO: load is TPC-B like transactions and the write cache is turned
has been
using in normal logging currently used for scheduling a checkpoint
as a pointer to how to track log size.
Raymond Raymond wrote:
I am trying to do some statistics to estimate recovery speed.
I can get how many log records the recovery process scaned
during the recovery and how long
I am trying to do some statistics to estimate recovery speed.
I can get how many log records the recovery process scaned
during the recovery and how long the recovery takes.The recovery
speed I got so far is something like
X log records/second
But, to use the recovery speed to schedule the
I have a question needs to be confirmed. Anyone can help me?
In checkpoint, derby checks from the last cached item in the
holder array to the first one(ref:
org.apache.derby.impl.services.cache.Clock.java)
to find dirty pages and write them out.
A cache page, which is clean before checkpoint
dependent on the size of the data that is affected.
For instance an insert log record will have the part of the row
on that page in that log record.
Raymond Raymond wrote:
Hi, anyone knows how long is a log records in derby?
I need to caculate something like how many log records
are in 1M log. I
Hi, anyone knows how long is a log records in derby?
I need to caculate something like how many log records
are in 1M log. I refered to
http://db.apache.org/derby/papers/logformats.html
but it does not tell me the answer. Anyone happens
to know that?
Thanks.
Raymond
Suresh,thank you very much.
Raymond
From: Suresh Thalamati [EMAIL PROTECTED]
Reply-To: derby-dev@db.apache.org
To: derby-dev@db.apache.org
Subject: Re: Two questions on derby checkpointing
Date: Thu, 16 Feb 2006 01:11:48 -0800
Raymond Raymond wrote:
I have some questions about derby
I have some questions about derby checkpointing. Someone can
help me to make it clear? ^_^.
1. I want to summarize what events will trigger a checkpoint?
On the derby website :
http://db.apache.org/derby/papers/logformats.html
I found such comments:
few seconds
less or more.
Aside from tuning the checkpoint interval, I think we should find ways to
minimize the effect of checkpoint on the system throughput.
I guess that is a different topic.
Thanks
-suresht
Raymond Raymond wrote:
Mike,
I am sorry, I did not make it very clear what I
Thanks for so many people gave their suggestions on this issue.
I really learned a lot of things from it. Since I am doing a student project
on derby, I am thinking of implementing two versions of checkpointing:
1. Let me call it automaticaly scheduled checkpointing. In this version,
I will not
As what we discussed in the incremental checkpointing issue before,
I need to gather the runtime disk I/O information. So, I need to monitor
the data reads and log writes. So far, I found the basic classes to do data
reads are :
org.apache.derby.impl.store.raw.data.RAFContainer.java
I have some questions about derby cache page, anyone can help me?
1. Derby writes out dirty pages through two ways:
- A background thread writes out dirty pages based on some
algorithm such as LRU.
- Checkpointing process writes out all the dirty pages.
I want to know after a dirty
a checkpoint vs. recovery time
requirements of the applications?
Raymond Raymond wrote:
From: Mike Matrigali [EMAIL PROTECTED]
I think this is the right path, though would need more details:
o does boot mean first time boot for each db?
o how to determine this machine
o and the total time
, percentage of
dirty pages, how long it takes to do a checkpoint vs. recovery time
requirements of the applications?
Raymond Raymond wrote:
From: Mike Matrigali [EMAIL PROTECTED]
I think this is the right path, though would need more details:
o does boot mean first time boot for each db?
o how
From: Mike Matrigali [EMAIL PROTECTED]
I think this is the right path, though would need more details:
o does boot mean first time boot for each db?
o how to determine this machine
o and the total time to run such a test.
There are some very quick and useful tests that would be fine to
add to
Hi, is there anyone happens to know how I can read the content of
derby log control and log files? I know they are binary files. I need to
know the content of those files. Anyone can help me?
Thanks.
Raymond
_
Take charge with a
file??
Date: Wed, 01 Feb 2006 21:05:48 +0100
Raymond Raymond [EMAIL PROTECTED] writes:
Hi, is there anyone happens to know how I can read the content of
derby log control and log files? I know they are binary files. I need to
know the content of those files. Anyone can help me?
The format
Mike,
Last time we discussed about how to map the recovery time into Xmb of log.
I have been thinking on it recently and have a proposal.
How about when the very first time derby boots (not every time) on a
certain
machine, we let the user to chose whether he (or she) want to do some
Mike, last time you gave me some comments about how to map the
recovery time into Xmb of log. I still have some question about it.
RR2. During initilization of Derby, we run some measurement that
RR determines the performance of the system and maps the
RR recovery time into some
Mike, thank you for you comments. They really help me a lot. I would
like to make more discussion on the issue.
RR2. During initilization of Derby, we run some measurement that
RR determines the performance of the system and maps the
RR recovery time into some X megabytes of log.
At the end of last year, I discussed something about how to automatically
decide the checkpoint interval with Oystein and Mike. Now, I am trying to
implement it. As what we discussed before, I write an outline of what I am
going to do.
1. Let the users to set a certain acceptable recovery time
RR == Raymond Raymond [EMAIL PROTECTED] writes:
RR Oystein wrote:
I would like to suggest the following:
- 1. The user may be able to configure a certain recovery time
that Derby should try to satisfy. (An appropriate default
must be determined).
- 2. During
From: [EMAIL PROTECTED] (Øystein Grøvlen)
Reply-To: derby-dev@db.apache.org
To: derby-dev@db.apache.org
Subject: Re: Some idea about automatic checkpointing issue
Date: Mon, 31 Oct 2005 15:36:39 +0100
RR == Raymond Raymond [EMAIL PROTECTED] writes:
RR Oystein wrote:
I would like
to limit it's use of idle
cycles or it enabled to try and schedule work during idle time.
Raymond Raymond wrote:
I have been thinking of the automatic checkpointing issue
recently.I also find someone added another issue about Use
of idle time for background checkpoint into the to-do list.
I
enabled to try and schedule work during idle time.
Raymond Raymond wrote:
I have been thinking of the automatic checkpointing issue
recently.I also find someone added another issue about "Use
of idle time for background checkpoint" into the to-do list.
I think we can consider these
and schedule work during idle time.
Raymond Raymond wrote:
I have been thinking of the automatic checkpointing issue
recently.I also find someone added another issue about Use
of idle time for background checkpoint into the to-do list.
I think we can consider these two issue together. I have
to limit it's use of idle
cycles or it enabled to try and schedule work during idle time.
Raymond Raymond wrote:
I have been thinking of the automatic checkpointing issue
recently.I also find someone added another issue about Use
of idle time for background checkpoint into the to-do list.
I think we
I have been thinking of the automatic checkpointing issue
recently.I also find someone added another issue about Use
of idle time for background checkpoint into the to-do list.
I think we can consider these two issue together. I have
some idea about it.
Instead of doing checkpoint periodically
From: Mike Matrigali [EMAIL PROTECTED]
Reply-To: Derby Development derby-dev@db.apache.org
To: Derby Development derby-dev@db.apache.org
Subject: Re: Can anyone give me some suggestions?
Date: Tue, 23 Aug 2005 10:03:35 -0700
Yes, the key is that in normal operation the call to these routines
is
From: Mike Matrigali [EMAIL PROTECTED]
Reply-To: Derby Development derby-dev@db.apache.org
Could you define fuzzy checkpoint or site what update interference
you are trying to avoid.
Fuzzy checkpointing means checkpoints are asynchronous and occur ant the
same time as new database updates
Oystein.Grovlen,Thanks your for your suggestions. It is exactly what I am
thinking
about.I am considering two aspects of the checkpointing issue.
1. How to make the engine tune the interval of checkpointing by itsself. I
think
It depends on the database buffer size, log buffer size and how many
Hi, everyone, I am a graduate student and trying to do some research with
Derby.
I am interested in the Autonomic checkpointing timing and log file size
issue on the
to-do list. I would like to know is there anyone else who is interested in
that? or
anyone who can give me some suggestions or
Who can tell me how I can find some documentations about
the cache and buffer management of Derby?
Thanks!
Raymond
_
Scan and help eliminate destructive viruses from your inbound and outbound
e-mail and attachments.
53 matches
Mail list logo
