Robert Haas wrote:
On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian br...@momjian.us wrote:
Uh, in this C comment:
+ ? ? ? ?* or not we want to take the time to write it. ?We allow up to 5%
of
+ ? ? ? ?* otherwise-not-dirty pages to be written due to hint bit changes,
5% of what? ?5%
On Mon, Feb 7, 2011 at 10:48 AM, Bruce Momjian br...@momjian.us wrote:
Robert Haas wrote:
On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian br...@momjian.us wrote:
Uh, in this C comment:
+ ? ? ? ?* or not we want to take the time to write it. ?We allow up to 5%
of
+ ? ? ? ?*
2011/2/7 Robert Haas robertmh...@gmail.com:
On Mon, Feb 7, 2011 at 10:48 AM, Bruce Momjian br...@momjian.us wrote:
Robert Haas wrote:
On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian br...@momjian.us wrote:
Uh, in this C comment:
+ ? ? ? ?* or not we want to take the time to write it. ?We
2011/2/7 Cédric Villemain cedric.villemain.deb...@gmail.com:
2011/2/7 Robert Haas robertmh...@gmail.com:
On Mon, Feb 7, 2011 at 10:48 AM, Bruce Momjian br...@momjian.us wrote:
Robert Haas wrote:
On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian br...@momjian.us wrote:
Uh, in this C comment:
2011/1/19 Robert Haas robertmh...@gmail.com:
On Wed, Jan 19, 2011 at 11:52 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
... So what we
want to do is write a percentage of them, in a way that guarantees
that they'll all eventually get written if people
On Sat, Feb 5, 2011 at 10:37 AM, Cédric Villemain
cedric.villemain.deb...@gmail.com wrote:
Please update the commitfest with the accurate patch, there is only
the old immature v1 of the patch in it.
I was about reviewing it...
https://commitfest.postgresql.org/action/patch_view?id=500
Woops,
2011/2/5 Robert Haas robertmh...@gmail.com:
On Sat, Feb 5, 2011 at 10:37 AM, Cédric Villemain
cedric.villemain.deb...@gmail.com wrote:
Please update the commitfest with the accurate patch, there is only
the old immature v1 of the patch in it.
I was about reviewing it...
On Sat, Feb 5, 2011 at 3:07 PM, Cédric Villemain
cedric.villemain.deb...@gmail.com wrote:
So I think this probably needs more testing
before we decide whether or not it's a good idea.
I *may* have an opportunity to test that in a real world application
where this hint bit was an issue.
That
2011/2/5 Robert Haas robertmh...@gmail.com:
On Sat, Feb 5, 2011 at 3:07 PM, Cédric Villemain
cedric.villemain.deb...@gmail.com wrote:
So I think this probably needs more testing
before we decide whether or not it's a good idea.
I *may* have an opportunity to test that in a real world
Robert Haas wrote:
On Sat, Feb 5, 2011 at 10:37 AM, C?dric Villemain
cedric.villemain.deb...@gmail.com wrote:
Please update the commitfest with the accurate patch, there is only
the old immature v1 of the patch in it.
I was about reviewing it...
On Sat, Feb 5, 2011 at 4:19 PM, Cédric Villemain
cedric.villemain.deb...@gmail.com wrote:
just reading the patch...
I understand the idea of the 5% flush.
*maybe* it make sense to use effective_io_concurrency GUC here to
improve the ratio, but it might be perceived as a bad usage ..
currently
On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian br...@momjian.us wrote:
Uh, in this C comment:
+ * or not we want to take the time to write it. We allow up to 5% of
+ * otherwise-not-dirty pages to be written due to hint bit changes,
5% of what? 5% of all buffers? 5% of all
2011/2/5 Bruce Momjian br...@momjian.us:
Robert Haas wrote:
On Sat, Feb 5, 2011 at 10:37 AM, C?dric Villemain
cedric.villemain.deb...@gmail.com wrote:
Please update the commitfest with the accurate patch, there is only
the old immature v1 of the patch in it.
I was about reviewing it...
On Sat, Feb 5, 2011 at 5:04 PM, Cédric Villemain
cedric.villemain.deb...@gmail.com wrote:
Robert, I am unsure with the hint_bit_write_allowance counter. It
looks a bit fragile because
nothing prevent hint_bit_write_allowance counter to increase a lot,
so that is not 100 but X*100 next hint
On 01/18/2011 06:44 PM, Robert Haas wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncuremmonc...@gmail.com wrote:
a few weeks back I hacked an experimental patch that removed the hint
bit action completely. the results were very premature and/or
incorrect, but my initial findings suggested
On 01/19/2011 09:03 AM, Andrea Suisani wrote:
On 01/18/2011 06:44 PM, Robert Haas wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncuremmonc...@gmail.com wrote:
a few weeks back I hacked an experimental patch that removed the hint
bit action completely. the results were very premature and/or
On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure mmonc...@gmail.com wrote:
a few weeks back I hacked an experimental patch that removed the hint
bit action completely. the results were very premature and/or
incorrect,
On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncure mmonc...@gmail.com wrote:
On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure mmonc...@gmail.com wrote:
a few weeks back I hacked an experimental patch that removed the hint
On 19.01.2011 15:56, Merlin Moncure wrote:
On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncuremmonc...@gmail.com wrote:
On Tue, Jan 18, 2011 at 12:44 PM, Robert Haasrobertmh...@gmail.com wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncuremmonc...@gmail.com wrote:
a few weeks back I hacked
On Wed, Jan 19, 2011 at 8:56 AM, Merlin Moncure mmonc...@gmail.com wrote:
Ah, I tested your patch vs stock postgres vs my patch, basically your
results are unhappily correct (mine was just a hair faster than yours
which you'd expect). The differential was even wider on my laptop
class
On Wed, Jan 19, 2011 at 10:44 AM, Robert Haas robertmh...@gmail.com wrote:
Here's a new version of the patch based on some experimentation with
ideas I posted yesterday. At least on my Mac laptop, this is pretty
effective at blunting the response time spike for the first table
scan, and it
On Wed, Jan 19, 2011 at 9:13 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 19.01.2011 15:56, Merlin Moncure wrote:
On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncuremmonc...@gmail.com
wrote:
On Tue, Jan 18, 2011 at 12:44 PM, Robert Haasrobertmh...@gmail.com
wrote:
On
On Wed, Jan 19, 2011 at 11:18 AM, Merlin Moncure mmonc...@gmail.com wrote:
On Wed, Jan 19, 2011 at 10:44 AM, Robert Haas robertmh...@gmail.com wrote:
Here's a new version of the patch based on some experimentation with
ideas I posted yesterday. At least on my Mac laptop, this is pretty
Robert Haas robertmh...@gmail.com writes:
... So what we
want to do is write a percentage of them, in a way that guarantees
that they'll all eventually get written if people continue to access
the same data.
The word guarantee seems quite inappropriate here, since as far as I
can see this
On Wed, Jan 19, 2011 at 11:52 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
... So what we
want to do is write a percentage of them, in a way that guarantees
that they'll all eventually get written if people continue to access
the same data.
The word
On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote:
Robert Haas wrote:
a quick-and-dirty attempt to limit the amount of I/O caused by hint
bits. I'm still very interested in knowing what people think about
that.
I found the elimination of the response-time spike promising. I
don't think
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby j...@nasby.net wrote:
On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote:
Robert Haas wrote:
a quick-and-dirty attempt to limit the amount of I/O caused by hint
bits. I'm still very interested in knowing what people think about
that.
I found the
On Jan 18, 2011, at 8:24 AM, Merlin Moncure wrote:
a few weeks back I hacked an experimental patch that removed the hint
bit action completely. the results were very premature and/or
incorrect, but my initial findings suggested that hint bits might not
be worth the cost from performance
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby j...@nasby.net wrote:
On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote:
Robert Haas wrote:
a quick-and-dirty attempt to limit the amount of I/O caused by hint
bits. I'm still very interested in knowing what people think about
that.
I found the
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure mmonc...@gmail.com wrote:
a few weeks back I hacked an experimental patch that removed the hint
bit action completely. the results were very premature and/or
incorrect, but my initial findings suggested that hint bits might not
be worth the cost
Robert Haas robertmh...@gmail.com writes:
I think you may be confused about what the patch does - currently,
pages with hint bit changes are considered dirty, period.
Therefore, they are written whenever any other dirty page would be
written: by the background writer cleaning scan, at
On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
I think you may be confused about what the patch does - currently,
pages with hint bit changes are considered dirty, period.
Therefore, they are written whenever any other dirty page
On 18.01.2011 21:16, Robert Haas wrote:
On Tue, Jan 18, 2011 at 1:32 PM, Tom Lanet...@sss.pgh.pa.us wrote:
While I was trying to performance-test the texteq patch, it occurred to
me that this proposed hint-bit change has got a serious drawback. To
wit, that it will totally destroy
On Tue, Jan 18, 2011 at 3:00 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 18.01.2011 21:16, Robert Haas wrote:
On Tue, Jan 18, 2011 at 1:32 PM, Tom Lanet...@sss.pgh.pa.us wrote:
While I was trying to performance-test the texteq patch, it occurred to
me that this
On Jan 14, 2011, at 7:24 PM, Josh Berkus wrote:
On 1/14/11 11:51 AM, Tom Lane wrote:
The people whose tables are mostly insert-only complain about it, but
that's not the majority of our userbase IMO. We just happen to have a
couple of particularly vocal ones, like Berkus.
It might or might
On Sat, Jan 15, 2011 at 6:28 PM, Josh Berkus j...@agliodbs.com wrote:
If the problem is that all the freezing happens at once, then ISTM the
solution is to add a random factor. Say, when a tuple just passes the
lower threshold it has a 1% chance of being frozen. The chance grows
until it is
Robert Haas wrote:
a quick-and-dirty attempt to limit the amount of I/O caused by hint
bits. I'm still very interested in knowing what people think about
that.
I found the elimination of the response-time spike promising. I
don't think I've seen enough data yet to feel comfortable
On Sun, Jan 16, 2011 at 5:37 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Robert Haas wrote:
a quick-and-dirty attempt to limit the amount of I/O caused by hint
bits. I'm still very interested in knowing what people think about
that.
I found the elimination of the response-time
Robert Haas wrote:
I think you may be confused about what the patch does - currently,
pages with hint bit changes are considered dirty, period.
Therefore, they are written whenever any other dirty page would be
written: by the background writer cleaning scan, at checkpoints,
and when a
On Sun, Jan 16, 2011 at 8:41 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Robert Haas wrote:
\ I think you may be confused about what the patch does - currently,
pages with hint bit changes are considered dirty, period.
Therefore, they are written whenever any other dirty page would
On Fri, Jan 14, 2011 at 05:24:31PM -0800, Josh Berkus wrote:
On 1/14/11 11:51 AM, Tom Lane wrote:
The people whose tables are mostly insert-only complain about it, but
that's not the majority of our userbase IMO. We just happen to have a
couple of particularly vocal ones, like Berkus.
If the problem is that all the freezing happens at once, then ISTM the
solution is to add a random factor. Say, when a tuple just passes the
lower threshold it has a 1% chance of being frozen. The chance grows
until it is 100% as it reaches the upper threshold.
Doesn't have to be random; it
Robert Haas robertmh...@gmail.com writes:
On Thu, Jan 13, 2011 at 10:43 PM, Tom Lane t...@sss.pgh.pa.us wrote:
This appears to remove the BM_JUST_DIRTIED logic. Please explain why
that's not completely broken. Even if it isn't completely broken,
it would seem better to do something like that
On Fri, Jan 14, 2011 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Thu, Jan 13, 2011 at 10:43 PM, Tom Lane t...@sss.pgh.pa.us wrote:
This appears to remove the BM_JUST_DIRTIED logic. Please explain why
that's not completely broken. Even if it
Robert Haas robertmh...@gmail.com wrote:
this seems to be speeding up the first run by 6x at the expense of
slowing down many subsequent runs by 10-15%.
If the overall throughput when measured far enough out to have hit a
steady state again is anywhere in the neighborhood of the unpatched
Robert Haas robertmh...@gmail.com writes:
On Fri, Jan 14, 2011 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Moreover this whole business of not treating hint-bit setting as
a page-dirtying operation is completely experimental/unproven IMO, so it
would be better to keep the patch footprint
On Fri, Jan 14, 2011 at 1:02 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Robert Haas robertmh...@gmail.com wrote:
this seems to be speeding up the first run by 6x at the expense of
slowing down many subsequent runs by 10-15%.
If the overall throughput when measured far enough out to
On Fri, Jan 14, 2011 at 1:06 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Fri, Jan 14, 2011 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Moreover this whole business of not treating hint-bit setting as
a page-dirtying operation is completely
Robert Haas robertmh...@gmail.com writes:
On Fri, Jan 14, 2011 at 1:06 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Well, it reinforces my opinion that it's experimental ;-). But first
run of what, exactly?
See the test case in my OP. The runs in question are select sum(1) from
s.
And are you
Robert Haas robertmh...@gmail.com wrote:
I'm hoping some will pick it up and play with it some more (hint,
hint).
That was a bit of a pun, eh?
Anyway, there are so many ideas in this area, it's hard to keep them
all straight. Personally, if I was going to start with something,
it would
On Fri, Jan 14, 2011 at 1:34 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Robert Haas robertmh...@gmail.com wrote:
I'm hoping some will pick it up and play with it some more (hint,
hint).
That was a bit of a pun, eh?
Unintentional...
Anyway, there are so many ideas in this area,
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Anyway, there are so many ideas in this area, it's hard to keep them
all straight. Personally, if I was going to start with something,
it would probably be to better establish what the impact is on
various workloads of *eliminating* hint
On Fri, Jan 14, 2011 at 1:42 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Anyway, there are so many ideas in this area, it's hard to keep them
all straight. Personally, if I was going to start with something,
it would probably be to better
Robert Haas robertmh...@gmail.com wrote:
Background freezing plays havoc with Hot Standby
I must have missed or forgotten the issue of background vacuums
and hot standby. Can you summarize why that's worse than hitting
thresholds where autovacuum is freezing things?
this test is
Robert Haas robertmh...@gmail.com wrote:
Freezing sooner could be viewed as an alternative to hint bits.
Exactly. And as your test showed, things run faster frozen than
unfrozen with hint bits set.
Trouble is, it breaks Hot Standby, badly.
You're really starting to worry me here. Both
Robert Haas robertmh...@gmail.com writes:
On Fri, Jan 14, 2011 at 1:42 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Um, yeah, I think you're having a problem keeping all the ideas straight
;-). The argument about forensics has to do with how soon we're willing
to freeze tuples, ie replace the XID
On Fri, Jan 14, 2011 at 2:01 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Trouble is, it breaks Hot Standby, badly.
You're really starting to worry me here. Both for performance and
to reduce the WAN bandwidth demands of our backup strategy we are
very aggressive with our freezing.
On Fri, Jan 14, 2011 at 1:52 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Robert Haas robertmh...@gmail.com wrote:
Background freezing plays havoc with Hot Standby
I must have missed or forgotten the issue of background vacuums
and hot standby. Can you summarize why that's worse
On Fri, Jan 14, 2011 at 2:09 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Fri, Jan 14, 2011 at 1:42 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Um, yeah, I think you're having a problem keeping all the ideas straight
;-). The argument about forensics has to
Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
Those things are related, though. Freezing sooner could be
viewed as an alternative to hint bits.
Freezing sooner isn't likely to reduce I/O compared to hint bits.
What that does is create I/O that you *have*
Robert Haas robertmh...@gmail.com wrote:
The critical issue is whether the tuples get frozen while they're
still invisible to some transactions on the standby server.
That's when you get query cancellations.
Oh, OK; I get that. That seems easy enough to at least mitigate to
a large degree
Robert Haas robertmh...@gmail.com writes:
On Fri, Jan 14, 2011 at 2:09 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Freezing sooner isn't likely to reduce I/O compared to hint bits. What
that does is create I/O that you *have* to execute ... both in the pages
themselves, and in WAL.
It depends on
On Fri, Jan 14, 2011 at 2:51 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Fri, Jan 14, 2011 at 2:09 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Freezing sooner isn't likely to reduce I/O compared to hint bits. What
that does is create I/O that you *have* to
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Robert Haas robertmh...@gmail.com wrote:
The critical issue is whether the tuples get frozen while they're
still invisible to some transactions on the standby server.
That's when you get query cancellations.
Oh, OK; I get that. That seems
Tom Lane t...@sss.pgh.pa.us wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Robert Haas robertmh...@gmail.com wrote:
The critical issue is whether the tuples get frozen while
they're still invisible to some transactions on the standby
server. That's when you get query
On 1/14/11 11:51 AM, Tom Lane wrote:
The people whose tables are mostly insert-only complain about it, but
that's not the majority of our userbase IMO. We just happen to have a
couple of particularly vocal ones, like Berkus.
It might or might not be the majority, but it's an extremely common
Robert Haas robertmh...@gmail.com writes:
I whipped up the attached patch tonight.
This appears to remove the BM_JUST_DIRTIED logic. Please explain why
that's not completely broken. Even if it isn't completely broken,
it would seem better to do something like that as a separate patch.
On Thu, Jan 13, 2011 at 10:43 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
I whipped up the attached patch tonight.
This appears to remove the BM_JUST_DIRTIED logic. Please explain why
that's not completely broken. Even if it isn't completely broken,
it
68 matches
Mail list logo
