thanks, that makes sense - I just wasn't thinking about the
startup costs and that growing the cache created invalid pages in the
same way as "shrinking" does in the case of drop table.
I tend to ignore the startup and wait for the
steady state and concentrate on performance there, great
you found this issue.
The clock algorithm is an area that may be ripe for improvements
(or probably better a complete new cache factory),
especially when dealing with very large caches. Also the cache
may be the first place to look to use new more concurrent
data structures provided by java. I would expect the current
design to scale reasonably well on 1, 2 and maybe 4 processors -
but it may see problems after that. I would expect the most gain
to be first the buffer manager, next the lock manager, and then
the various other caches (statement cache, open file cache).
Knut Anders Hatlen wrote:
Mike Matrigali <[EMAIL PROTECTED]> writes:
Also I was wondering if you had any idea where the invalid pages
were coming from?
The invalid pages are coming from Clock.growCache() where they are
created but not initialized (hence invalid). The synchronization
policy for Clock allows (to some extent) interleaving, so when there
are more than one client there is a possibility that clients might see
invalid pages not belonging to them. These pages are however marked
with keepForCreate and cannot be taken by any other client than the
one that created it.
The original problem with invalid pages was
from drop tables, where applications didn't like it that we used
all the pages in the cache when they were growing a single table
to 1/2 the cache size and then dropping it. So the invalid case
was sort of special and wasn't expected to happen very often.
The usual case should be the cache growing to full and then LRU'ing
old pages to make for way for new ones.
Maybe there are some temp tables involved in your testing?
No temp tables are involved. The load is really simple, with
transactions consisting of one single-row select (using primary key),
three single-row updates (using primary key) and one insert.
I thought you were doing a TPCC type of benchmark, so I am not
sure where the invalid pages are coming from.
Your work on big caches is very interesting, historically not much
work has been done in this area.
Mike Matrigali wrote:
just got a chance to look at the patch, it would have been nice if
some of the great comments that are in the bug description had made
it into the code changes.
You are right, I should have put more comments into the code. I'll see
what I can do. Thanks for looking at the patch.
