Hi All,
On 5/31/07 12:40 AM, Heikki Linnakangas [EMAIL PROTECTED] wrote:
BTW, we've been talking about the L2 cache effect but we don't really
know for sure if the effect has anything to do with the L2 cache. But
whatever it is, it's real.
The mailing list archives contain the ample evidence
Luke Lonergan [EMAIL PROTECTED] writes:
The mailing list archives contain the ample evidence of:
- it's definitely an L2 cache effect
- on fast I/O hardware tests show large benefits of keeping the ring in L2
Please provide some specific pointers, because I don't remember that.
Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
I just ran a quick test with 4 concurrent scans on a dual-core system,
and it looks like we do leak buffers from the rings because they're
pinned at the time they would be recycled.
Yeah, I noticed the same in some tests here. I
Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
A heapscan would pin the buffer only once and hence bump its count at
most once, so I don't see a big problem here. Also, I'd argue that
buffers that had a positive usage_count shouldn't get sucked into the
ring to begin with.
On Tue, 2007-05-29 at 17:43 -0700, Jeff Davis wrote:
Hmm. But we probably don't want the same buffer in two different
backends' rings, either. You *sure* the sync-scan patch has no
interaction with this one?
I will run some tests again tonight, I think the interaction needs more
Heikki Linnakangas [EMAIL PROTECTED] writes:
Tom Lane wrote:
One other question: I see the patch sets the threshold for switching
from normal to ring-buffer heapscans at table size = NBuffers. Why
so high? I'd have expected maybe at most NBuffers/4 or NBuffers/10.
If you don't want a
Jeff Davis wrote:
On Tue, 2007-05-29 at 17:43 -0700, Jeff Davis wrote:
Hmm. But we probably don't want the same buffer in two different
backends' rings, either. You *sure* the sync-scan patch has no
interaction with this one?
I will run some tests again tonight, I think the interaction
Heikki Linnakangas [EMAIL PROTECTED] writes:
I just ran a quick test with 4 concurrent scans on a dual-core system,
and it looks like we do leak buffers from the rings because they're
pinned at the time they would be recycled.
Yeah, I noticed the same in some tests here. I think there's not
On Wed, 2007-05-30 at 15:56 -0400, Tom Lane wrote:
In
the sync-scan case the idea seems pretty bogus anyway, because the
actual working set will be N backends' rings not just one.
I don't follow. Ideally, in the sync-scan case, the sets of buffers in
the ring of different scans on the same
Jeff Davis [EMAIL PROTECTED] writes:
On Wed, 2007-05-30 at 15:56 -0400, Tom Lane wrote:
In the sync-scan case the idea seems pretty bogus anyway, because the
actual working set will be N backends' rings not just one.
I don't follow. Ideally, in the sync-scan case, the sets of buffers in
the
Tom Lane wrote:
Gregory Stark [EMAIL PROTECTED] writes:
Is there a reason UnpinBuffer has to be the one to increment the usage count
anyways? Why can't ReadBuffer handle incrementing the count and just trust
that it won't be decremented until the buffer is unpinned anyways?
That's a good
Alvaro Herrera wrote:
Tom Lane wrote:
Gregory Stark [EMAIL PROTECTED] writes:
Is there a reason UnpinBuffer has to be the one to increment the usage count
anyways? Why can't ReadBuffer handle incrementing the count and just trust
that it won't be decremented until the buffer is unpinned
Heikki Linnakangas [EMAIL PROTECTED] writes:
Here's a new version, all known issues are now fixed. I'm now happy with
this patch.
I'm looking this over and finding it fairly ugly from a
system-structural point of view. In particular, this has pushed the
single-global-variable
Heikki Linnakangas [EMAIL PROTECTED] writes:
Tom Lane wrote:
A point I have not figured out how to deal with is that in the patch as
given, UnpinBuffer needs to know the strategy; and getting it that info
would make the changes far more invasive. But the patch's behavior here
is pretty risky
Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
It's assumed that the same strategy is used when unpinning, which is
true for the current usage (and apparently needs to be documented).
I don't believe that for a moment. Even in the trivial heapscan case,
the last pin is
Heikki Linnakangas [EMAIL PROTECTED] writes:
One idea is to keep track which pins are taken using the bulk strategy.
It's a bit tricky when a buffer is pinned multiple times since we don't
know which ReleaseBuffer corresponds which ReadBuffer, but perhaps we
could get away with just a flag
Tom Lane [EMAIL PROTECTED] writes:
A point I have not figured out how to deal with is that in the patch as
given, UnpinBuffer needs to know the strategy; and getting it that info
would make the changes far more invasive. But the patch's behavior here
is pretty risky anyway, since the
Gregory Stark [EMAIL PROTECTED] writes:
Is there a reason UnpinBuffer has to be the one to increment the usage count
anyways? Why can't ReadBuffer handle incrementing the count and just trust
that it won't be decremented until the buffer is unpinned anyways?
That's a good question. I think
Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
Here's a new version, all known issues are now fixed. I'm now happy with
this patch.
Next, I'll start looking at the latest version of Jeff's synchronized
scans patch.
I'm a bit confused --- weren't you intending to review these in
Here's a new version, all known issues are now fixed. I'm now happy with
this patch.
Next, I'll start looking at the latest version of Jeff's synchronized
scans patch.
Bruce Momjian wrote:
Great. Based on this, do you have a patch that is ready to apply.
Heikki Linnakangas [EMAIL PROTECTED] writes:
Here's a new version, all known issues are now fixed. I'm now happy with
this patch.
Next, I'll start looking at the latest version of Jeff's synchronized
scans patch.
I'm a bit confused --- weren't you intending to review these in parallel
Great. Based on this, do you have a patch that is ready to apply.
---
Heikki Linnakangas wrote:
Heikki Linnakangas wrote:
In any case, I'd like to see more test results before we make a
decision. I'm running tests
Not yet, there's still one issue that needs fixing.
Bruce Momjian wrote:
Great. Based on this, do you have a patch that is ready to apply.
---
Heikki Linnakangas wrote:
Heikki Linnakangas wrote:
In any case, I'd like to
I forgot to attach the program used to generate test data. Here it is.
Heikki Linnakangas wrote:
Attached is a new version of Simon's scan-resistant buffer manager
patch. It's not ready for committing yet because of a small issue I
found this morning (* see bottom), but here's a status update.
Heikki Linnakangas wrote:
In any case, I'd like to see more test results before we make a
decision. I'm running tests with DBT-2 and a seq scan running in the
background to see if the cache-spoiling effect shows up. I'm also trying
to get hold of some bigger hardware to run on. Running these
CK Tan wrote:
If it is convenient for you, could you run my patch against the same
hardware and data to get some numbers on select for comparison? Although
we don't address updates, copy, or inserts, we are definitely getting
at least 20% improvement in scans here without poisoning the
Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
In any case, I do want this for VACUUMs to fix the WAL flush for every
dirty page problem. Maybe we should indeed drop the other aspects of
the patch and move on, I'm getting tired of this as well.
Can we devise a small patch that
Attached is a new version of Simon's scan-resistant buffer manager
patch. It's not ready for committing yet because of a small issue I
found this morning (* see bottom), but here's a status update.
To recap, the basic idea is to use a small ring of buffers for large
scans like VACUUM, COPY
Heikki Linnakangas [EMAIL PROTECTED] writes:
I've completed a set of performance tests on a test server. The server
has 4 GB of RAM, of which 1 GB is used for shared_buffers.
Perhaps I'm misreading it, but these tests seem to show no improvement
worth spending any effort on --- some of the
Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
I've completed a set of performance tests on a test server. The server
has 4 GB of RAM, of which 1 GB is used for shared_buffers.
Perhaps I'm misreading it, but these tests seem to show no improvement
worth spending any effort on
Heikki Linnakangas [EMAIL PROTECTED] writes:
In any case, I do want this for VACUUMs to fix the WAL flush for every
dirty page problem. Maybe we should indeed drop the other aspects of
the patch and move on, I'm getting tired of this as well.
Can we devise a small patch that fixes that issue
Hi Heikki,
On 5/17/07 10:28 AM, Heikki Linnakangas [EMAIL PROTECTED] wrote:
is also visible on larger scans that don't fit in cache with bigger I/O
hardware, and this patch would increase the max. I/O throughput that we
can handle on such hardware. I don't have such hardware available, I
32 matches
Mail list logo
