I dont have much help to offer, only some history. I do not think there was
ever any performance work done
in this area, so hopefully there is some low hanging fruit. I have a fuzzy
memory that restart recover undo was
really inefficient as there is not a good way to walk backward in the log per
transaction.
First would be good to understand performance per phase. redo, vs undo. i
would expect redo to run pretty fast
but everything is very i/o bound if there is a lot of log and lot of database
pages being looked at. The whole
thing is single threaded and probably does not use very much memory. so a lot
of your machine resources
are not being used. making it parallel is likely hard as there are likely
assumptions on it being single threaded
and thus it does not need locks/latches for concurrency (or at least it can set
flags saying it is ok with double latching).
to understand the size of the recovery you were looking at, what was the total
size of seg0 of your db and what was
the log directory size. And for log size do you know if you keep more than is
required for restart recovery. It would
be if you have the setting that keeps all old logs so that you can recover from
a backup at specific time and then apply
current set of logs.
Also if you get set up to run experiments an easy test would be to run test
first with I/O configured as we need for
correctness - ie with system sync write enabled. And then run it with system
write sync disabled. If this is a big
difference you know the I/O part of problem is sync write vs some other issue.
We have optimized doing as few sync
writes as necessary in "normal" path - but maybe there are recovery paths that
do not.
Derby was originally designed to fit very small dbs with very low xact load.
so there are very likely some assumptions
that don't fit your problem set.
On 4/25/2016 6:41 AM, Bergquist, Brett wrote:
Thanks for the pointes Kristian. I definitely will be in this code shortly.
The system appeared to not be CPU bound but rather I/O bound. I need to get a
test setup into a condition like at the customer first and then start doing
some debugging and profiling to see where things are.
-----Original Message-----
From: Kristian Waagan [mailto:kristwaa.apa...@gmail.com]
Sent: Saturday, April 23, 2016 4:22 PM
To: derby-dev@db.apache.org
Subject: Re: Is there any debug logging available when a database is being
recovered
Den 20.04.2016 16.46, skrev Bergquist, Brett:
It took longer (and never finished) to process the recovery logs file that it did to
restore the database from the previous night from backup (that took 6 hours to
"untar") At about 14 hours, the recovery log files were not finished
processing so we had to give up and restore from backup. That seems quite an inordinate
amount of time to process the recovery log files. Note that this was not an
underpowered machine either. It was an Oracle M5000 with 32 processors, 128G of memory,
and Fibre Channel attached storage.
I would like to start looking at this, so if anyone has any ideas where to
begin, shout them out.
Hi Brett,
>From what I can see, there is some logging available in debug builds using the
LogTrace-flag. I have no recollection of using it, so I cannot say how useful the
information you get from enabling it is.
To try it, get the debug build and specify "derby.debug.true=LogTrace".
Based on a quick scan of the code, without verifying anything, it looks like
you should see when Derby switches to the next log file. There is also a log
entry for the end of the redo phase.
Obtaining the stack trace at various point might also give some clues if you're
willing to dive into the code :)
I expect this part of the code is singlethreaded, so I hope the machine can
give you decent performance using a single logical CPU. Do you suspect CPU or
IO to be a bottleneck?
Regards,
--
Kristian
-----Original Message-----
From: Bryan Pendleton [mailto:bpendleton.de...@gmail.com]
Sent: Wednesday, April 20, 2016 5:53 AM
To: derby-dev@db.apache.org
Subject: Re: Is there any debug logging available when a database is
being recovered
So what would be useful would be something like:
Performing database recovery
Starting analysis pass
215 transactions detected to be processed Starting redo pass ....
anything that could give some feedback Starting undo pass
....anything that could give some feedback
I'm afraid all I can add is: yes, I completely agree, that would be extremely
useful.
bryan
Canoga Perkins
20600 Prairie Street
Chatsworth, CA 91311
(818) 718-6300
This e-mail and any attached document(s) is confidential and is intended only
for the review of the party to whom it is addressed. If you have received this
transmission in error, please notify the sender immediately and discard the
original message and any attachment(s).
Canoga Perkins
20600 Prairie Street
Chatsworth, CA 91311
(818) 718-6300
This e-mail and any attached document(s) is confidential and is intended only
for the review of the party to whom it is addressed. If you have received this
transmission in error, please notify the sender immediately and discard the
original message and any attachment(s).
--
email: Mike Matrigali - mikema...@gmail.com
linkedin: https://www.linkedin.com/in/MikeMatrigali
