2011/2/7 Cédric Villemain :
> 2011/2/7 Robert Haas :
>> On Mon, Feb 7, 2011 at 10:48 AM, Bruce Momjian wrote:
>>> Robert Haas wrote:
On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian wrote:
> Uh, in this C comment:
>
> + ? ? ? ?* or not we want to take the time to write it. ?We al
2011/2/7 Robert Haas :
> On Mon, Feb 7, 2011 at 10:48 AM, Bruce Momjian wrote:
>> Robert Haas wrote:
>>> On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian wrote:
>>> > Uh, in this C comment:
>>> >
>>> > + ? ? ? ?* or not we want to take the time to write it. ?We allow up to
>>> > 5% of
>>> > + ? ? ?
On Mon, Feb 7, 2011 at 10:48 AM, Bruce Momjian wrote:
> Robert Haas wrote:
>> On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian wrote:
>> > Uh, in this C comment:
>> >
>> > + ? ? ? ?* or not we want to take the time to write it. ?We allow up to 5%
>> > of
>> > + ? ? ? ?* otherwise-not-dirty pages to
Robert Haas wrote:
> On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian wrote:
> > Uh, in this C comment:
> >
> > + ? ? ? ?* or not we want to take the time to write it. ?We allow up to 5%
> > of
> > + ? ? ? ?* otherwise-not-dirty pages to be written due to hint bit changes,
> >
> > 5% of what? ?5% of
On Sat, Feb 5, 2011 at 5:04 PM, Cédric Villemain
wrote:
> Robert, I am unsure with the hint_bit_write_allowance counter. It
> looks a bit fragile because
> nothing prevent hint_bit_write_allowance counter to increase a lot,
> so that is not 100 but X*100 next hint bit will be written. Isn't it ?
2011/2/5 Bruce Momjian :
> Robert Haas wrote:
>> On Sat, Feb 5, 2011 at 10:37 AM, C?dric Villemain
>> wrote:
>> > Please update the commitfest with the accurate patch, there is only
>> > the old immature v1 of the patch in it.
>> > I was about reviewing it...
>> >
>> > https://commitfest.postgresq
On Sat, Feb 5, 2011 at 4:31 PM, Bruce Momjian wrote:
> Uh, in this C comment:
>
> + * or not we want to take the time to write it. We allow up to 5% of
> + * otherwise-not-dirty pages to be written due to hint bit changes,
>
> 5% of what? 5% of all buffers? 5% of all hint-bit-dirt
On Sat, Feb 5, 2011 at 4:19 PM, Cédric Villemain
wrote:
> just reading the patch...
> I understand the idea of the 5% flush.
> *maybe* it make sense to use effective_io_concurrency GUC here to
> improve the ratio, but it might be perceived as a bad usage ..
> currently effective_io_concurrency is
Robert Haas wrote:
> On Sat, Feb 5, 2011 at 10:37 AM, C?dric Villemain
> wrote:
> > Please update the commitfest with the accurate patch, there is only
> > the old immature v1 of the patch in it.
> > I was about reviewing it...
> >
> > https://commitfest.postgresql.org/action/patch_view?id=500
>
2011/2/5 Robert Haas :
> On Sat, Feb 5, 2011 at 3:07 PM, Cédric Villemain
> wrote:
>>> So I think this probably needs more testing
>>> before we decide whether or not it's a good idea.
>>
>> I *may* have an opportunity to test that in a real world application
>> where this hint bit was an issue.
>
On Sat, Feb 5, 2011 at 3:07 PM, Cédric Villemain
wrote:
>> So I think this probably needs more testing
>> before we decide whether or not it's a good idea.
>
> I *may* have an opportunity to test that in a real world application
> where this hint bit was an issue.
That would be great. But note t
2011/2/5 Robert Haas :
> On Sat, Feb 5, 2011 at 10:37 AM, Cédric Villemain
> wrote:
>> Please update the commitfest with the accurate patch, there is only
>> the old immature v1 of the patch in it.
>> I was about reviewing it...
>>
>> https://commitfest.postgresql.org/action/patch_view?id=500
>
>
On Sat, Feb 5, 2011 at 10:37 AM, Cédric Villemain
wrote:
> Please update the commitfest with the accurate patch, there is only
> the old immature v1 of the patch in it.
> I was about reviewing it...
>
> https://commitfest.postgresql.org/action/patch_view?id=500
Woops, sorry about that. Here's an
2011/1/19 Robert Haas :
> On Wed, Jan 19, 2011 at 11:52 AM, Tom Lane wrote:
>> Robert Haas writes:
>>> ... So what we
>>> want to do is write a percentage of them, in a way that guarantees
>>> that they'll all eventually get written if people continue to access
>>> the same data.
>>
>> The word "
On Wed, Jan 19, 2011 at 11:52 AM, Tom Lane wrote:
> Robert Haas writes:
>> ... So what we
>> want to do is write a percentage of them, in a way that guarantees
>> that they'll all eventually get written if people continue to access
>> the same data.
>
> The word "guarantee" seems quite inappropri
Robert Haas writes:
> ... So what we
> want to do is write a percentage of them, in a way that guarantees
> that they'll all eventually get written if people continue to access
> the same data.
The word "guarantee" seems quite inappropriate here, since as far as I
can see this approach provides n
On Wed, Jan 19, 2011 at 11:18 AM, Merlin Moncure wrote:
> On Wed, Jan 19, 2011 at 10:44 AM, Robert Haas wrote:
>> Here's a new version of the patch based on some experimentation with
>> ideas I posted yesterday. At least on my Mac laptop, this is pretty
>> effective at blunting the response time
On Wed, Jan 19, 2011 at 9:13 AM, Heikki Linnakangas
wrote:
> On 19.01.2011 15:56, Merlin Moncure wrote:
>>
>> On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncure
>> wrote:
>>>
>>> On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas
>>> wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure
On Wed, Jan 19, 2011 at 10:44 AM, Robert Haas wrote:
> Here's a new version of the patch based on some experimentation with
> ideas I posted yesterday. At least on my Mac laptop, this is pretty
> effective at blunting the response time spike for the first table
> scan, and it converges to steady-
On Wed, Jan 19, 2011 at 8:56 AM, Merlin Moncure wrote:
> Ah, I tested your patch vs stock postgres vs my patch, basically your
> results are unhappily correct (mine was just a hair faster than yours
> which you'd expect). The differential was even wider on my laptop
> class hardware, maybe 26%.
On 19.01.2011 15:56, Merlin Moncure wrote:
On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncure wrote:
On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote:
a few weeks back I hacked an experimental patch that removed the hint
bit action comp
On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncure wrote:
> On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas wrote:
>> On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote:
>>> a few weeks back I hacked an experimental patch that removed the hint
>>> bit action completely. the results were very prem
On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas wrote:
> On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote:
>> a few weeks back I hacked an experimental patch that removed the hint
>> bit action completely. the results were very premature and/or
>> incorrect, but my initial findings suggested
On 01/19/2011 09:03 AM, Andrea Suisani wrote:
On 01/18/2011 06:44 PM, Robert Haas wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote:
a few weeks back I hacked an experimental patch that removed the hint
bit action completely. the results were very premature and/or
incorrect, but my i
On 01/18/2011 06:44 PM, Robert Haas wrote:
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote:
a few weeks back I hacked an experimental patch that removed the hint
bit action completely. the results were very premature and/or
incorrect, but my initial findings suggested that hint bits migh
On Tue, Jan 18, 2011 at 3:00 PM, Heikki Linnakangas
wrote:
> On 18.01.2011 21:16, Robert Haas wrote:
>>
>> On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane wrote:
>>>
>>> While I was trying to performance-test the texteq patch, it occurred to
>>> me that this proposed hint-bit change has got a serious d
On 18.01.2011 21:16, Robert Haas wrote:
On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane wrote:
While I was trying to performance-test the texteq patch, it occurred to
me that this proposed hint-bit change has got a serious drawback. To
wit, that it will totally destroy reproducibility of any perform
On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane wrote:
> Robert Haas writes:
I think you may be confused about what the patch does - currently,
pages with hint bit changes are considered dirty, period.
Therefore, they are written whenever any other dirty page would be
written: by th
Robert Haas writes:
>>> I think you may be confused about what the patch does - currently,
>>> pages with hint bit changes are considered dirty, period.
>>> Therefore, they are written whenever any other dirty page would be
>>> written: by the background writer cleaning scan, at checkpoints,
>>> a
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote:
> a few weeks back I hacked an experimental patch that removed the hint
> bit action completely. the results were very premature and/or
> incorrect, but my initial findings suggested that hint bits might not
> be worth the cost from performan
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby wrote:
> On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote:
>> Robert Haas wrote:
>>
>>> a quick-and-dirty attempt to limit the amount of I/O caused by hint
>>> bits. I'm still very interested in knowing what people think about
>>> that.
>>
>> I found th
On Jan 18, 2011, at 8:24 AM, Merlin Moncure wrote:
> a few weeks back I hacked an experimental patch that removed the hint
> bit action completely. the results were very premature and/or
> incorrect, but my initial findings suggested that hint bits might not
> be worth the cost from performance st
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby wrote:
> On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote:
>> Robert Haas wrote:
>>
>>> a quick-and-dirty attempt to limit the amount of I/O caused by hint
>>> bits. I'm still very interested in knowing what people think about
>>> that.
>>
>> I found th
On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote:
> Robert Haas wrote:
>
>> a quick-and-dirty attempt to limit the amount of I/O caused by hint
>> bits. I'm still very interested in knowing what people think about
>> that.
>
> I found the elimination of the response-time spike promising. I
> d
On Jan 14, 2011, at 7:24 PM, Josh Berkus wrote:
> On 1/14/11 11:51 AM, Tom Lane wrote:
>> The people whose tables are mostly insert-only complain about it, but
>> that's not the majority of our userbase IMO. We just happen to have a
>> couple of particularly vocal ones, like Berkus.
>
> It might
On Sun, Jan 16, 2011 at 8:41 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
\>> I think you may be confused about what the patch does - currently,
>> pages with hint bit changes are considered dirty, period.
>> Therefore, they are written whenever any other dirty page would be
>> written: by the
Robert Haas wrote:
> I think you may be confused about what the patch does - currently,
> pages with hint bit changes are considered dirty, period.
> Therefore, they are written whenever any other dirty page would be
> written: by the background writer cleaning scan, at checkpoints,
> and when a
On Sun, Jan 16, 2011 at 5:37 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>> a quick-and-dirty attempt to limit the amount of I/O caused by hint
>> bits. I'm still very interested in knowing what people think about
>> that.
>
> I found the elimination of the response-time spike promising. I
>
Robert Haas wrote:
> a quick-and-dirty attempt to limit the amount of I/O caused by hint
> bits. I'm still very interested in knowing what people think about
> that.
I found the elimination of the response-time spike promising. I
don't think I've seen enough data yet to feel comfortable endor
On Sat, Jan 15, 2011 at 6:28 PM, Josh Berkus wrote:
>> If the problem is that all the freezing happens at once, then ISTM the
>> solution is to add a random factor. Say, when a tuple just passes the
>> lower threshold it has a 1% chance of being frozen. The chance grows
>> until it is 100% as it r
> If the problem is that all the freezing happens at once, then ISTM the
> solution is to add a random factor. Say, when a tuple just passes the
> lower threshold it has a 1% chance of being frozen. The chance grows
> until it is 100% as it reaches the upper threshold.
Doesn't have to be random;
On Fri, Jan 14, 2011 at 05:24:31PM -0800, Josh Berkus wrote:
> On 1/14/11 11:51 AM, Tom Lane wrote:
> > The people whose tables are mostly insert-only complain about it, but
> > that's not the majority of our userbase IMO. We just happen to have a
> > couple of particularly vocal ones, like Berkus
On 1/14/11 11:51 AM, Tom Lane wrote:
> The people whose tables are mostly insert-only complain about it, but
> that's not the majority of our userbase IMO. We just happen to have a
> couple of particularly vocal ones, like Berkus.
It might or might not be the majority, but it's an extremely commo
Tom Lane wrote:
> "Kevin Grittner" writes:
>> Robert Haas wrote:
>>> The critical issue is whether the tuples get frozen while
>>> they're still invisible to some transactions on the standby
>>> server. That's when you get query cancellations.
>
>> Oh, OK; I get that. That seems easy enough
"Kevin Grittner" writes:
> Robert Haas wrote:
>> The critical issue is whether the tuples get frozen while they're
>> still invisible to some transactions on the standby server.
>> That's when you get query cancellations.
> Oh, OK; I get that. That seems easy enough to at least mitigate to
>
On Fri, Jan 14, 2011 at 2:51 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Fri, Jan 14, 2011 at 2:09 PM, Tom Lane wrote:
>>> Freezing sooner isn't likely to reduce I/O compared to hint bits. What
>>> that does is create I/O that you *have* to execute ... both in the pages
>>> themselves, and
Robert Haas writes:
> On Fri, Jan 14, 2011 at 2:09 PM, Tom Lane wrote:
>> Freezing sooner isn't likely to reduce I/O compared to hint bits. What
>> that does is create I/O that you *have* to execute ... both in the pages
>> themselves, and in WAL.
> It depends on which way you tilt your head -
Robert Haas wrote:
> The critical issue is whether the tuples get frozen while they're
> still invisible to some transactions on the standby server.
> That's when you get query cancellations.
Oh, OK; I get that. That seems easy enough to at least mitigate to
a large degree by some threshold
Tom Lane wrote:
> Robert Haas writes:
>> Those things are related, though. Freezing sooner could be
>> viewed as an alternative to hint bits.
>
> Freezing sooner isn't likely to reduce I/O compared to hint bits.
> What that does is create I/O that you *have* to execute ... both
> in the page
On Fri, Jan 14, 2011 at 2:09 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Fri, Jan 14, 2011 at 1:42 PM, Tom Lane wrote:
>>> Um, yeah, I think you're having a problem keeping all the ideas straight
>>> ;-). The argument about forensics has to do with how soon we're willing
>>> to freeze tuple
On Fri, Jan 14, 2011 at 1:52 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>
>> Background freezing plays havoc with Hot Standby
>
> I must have missed or forgotten the issue of background vacuums
> and hot standby. Can you summarize why that's worse than hitting
> thresholds where autovacuum i
On Fri, Jan 14, 2011 at 2:01 PM, Kevin Grittner
wrote:
>> Trouble is, it breaks Hot Standby, badly.
>
> You're really starting to worry me here. Both for performance and
> to reduce the WAN bandwidth demands of our backup strategy we are
> very aggressive with our freezing. Do off-hours VACUUM (
Robert Haas writes:
> On Fri, Jan 14, 2011 at 1:42 PM, Tom Lane wrote:
>> Um, yeah, I think you're having a problem keeping all the ideas straight
>> ;-). The argument about forensics has to do with how soon we're willing
>> to freeze tuples, ie replace the XID with a constant. Not about hint
>
Robert Haas wrote:
> Freezing sooner could be viewed as an alternative to hint bits.
Exactly. And as your test showed, things run faster frozen than
unfrozen with hint bits set.
> Trouble is, it breaks Hot Standby, badly.
You're really starting to worry me here. Both for performance and
Robert Haas wrote:
> Background freezing plays havoc with Hot Standby
I must have missed or forgotten the issue of background vacuums
and hot standby. Can you summarize why that's worse than hitting
thresholds where autovacuum is freezing things?
> this test is sufficient to show that elimi
On Fri, Jan 14, 2011 at 1:42 PM, Tom Lane wrote:
> "Kevin Grittner" writes:
>> Anyway, there are so many ideas in this area, it's hard to keep them
>> all straight. Personally, if I was going to start with something,
>> it would probably be to better establish what the impact is on
>> various wo
"Kevin Grittner" writes:
> Anyway, there are so many ideas in this area, it's hard to keep them
> all straight. Personally, if I was going to start with something,
> it would probably be to better establish what the impact is on
> various workloads of *eliminating* hint bits.
> I know some peop
On Fri, Jan 14, 2011 at 1:34 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>
>> I'm hoping some will pick it up and play with it some more (hint,
>> hint).
>
> That was a bit of a pun, eh?
Unintentional...
> Anyway, there are so many ideas in this area, it's hard to keep them
> all straight.
Robert Haas wrote:
> I'm hoping some will pick it up and play with it some more (hint,
> hint).
That was a bit of a pun, eh?
Anyway, there are so many ideas in this area, it's hard to keep them
all straight. Personally, if I was going to start with something,
it would probably be to better
Robert Haas writes:
> On Fri, Jan 14, 2011 at 1:06 PM, Tom Lane wrote:
>> Well, it reinforces my opinion that it's experimental ;-). But "first
>> run" of what, exactly?
> See the test case in my OP. The "runs" in question are "select sum(1) from
> s".
>> And are you sure you're taking a who
On Fri, Jan 14, 2011 at 1:06 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Fri, Jan 14, 2011 at 12:47 PM, Tom Lane wrote:
>>> Moreover this whole business of not treating hint-bit setting as
>>> a page-dirtying operation is completely experimental/unproven IMO, so it
>>> would be better to kee
On Fri, Jan 14, 2011 at 1:02 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>> this seems to be speeding up the first run by 6x at the expense of
>> slowing down many subsequent runs by 10-15%.
>
> If the overall throughput when measured far enough out to have hit a
> steady state again is anywhe
Robert Haas writes:
> On Fri, Jan 14, 2011 at 12:47 PM, Tom Lane wrote:
>> Moreover this whole business of not treating hint-bit setting as
>> a page-dirtying operation is completely experimental/unproven IMO, so it
>> would be better to keep the patch footprint as small as possible.
> I have so
Robert Haas wrote:
> this seems to be speeding up the first run by 6x at the expense of
> slowing down many subsequent runs by 10-15%.
If the overall throughput when measured far enough out to have hit a
steady state again is anywhere in the neighborhood of the unpatched
throughput, the leveli
On Fri, Jan 14, 2011 at 12:47 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Thu, Jan 13, 2011 at 10:43 PM, Tom Lane wrote:
>>> This appears to remove the BM_JUST_DIRTIED logic. Please explain why
>>> that's not completely broken. Even if it isn't completely broken,
>>> it would seem better t
Robert Haas writes:
> On Thu, Jan 13, 2011 at 10:43 PM, Tom Lane wrote:
>> This appears to remove the BM_JUST_DIRTIED logic. Please explain why
>> that's not completely broken. Even if it isn't completely broken,
>> it would seem better to do something like that as a separate patch.
> Well, th
On Thu, Jan 13, 2011 at 10:43 PM, Tom Lane wrote:
> Robert Haas writes:
>> I whipped up the attached patch tonight.
>
> This appears to remove the BM_JUST_DIRTIED logic. Please explain why
> that's not completely broken. Even if it isn't completely broken,
> it would seem better to do something
Robert Haas writes:
> I whipped up the attached patch tonight.
This appears to remove the BM_JUST_DIRTIED logic. Please explain why
that's not completely broken. Even if it isn't completely broken,
it would seem better to do something like that as a separate patch.
rega
I whipped up the attached patch tonight. It's pretty quick and dirty,
so it's possible I've missed something, but the intent is to suppress
writing of hint bits by buffers allocating backends, and by
checkpoints, and write them only from the background writer cleaning
scan. It therefore should (a
69 matches
Mail list logo
