Sorry for a long pause on this thread. A new arrival at home kept me
occupied all the time.
This thread saw a lot of ideas and suggestions from different people. I
don't think we had an agreement one way or the other on any of them, but
let me summarize the discussion for archival and taking
On 20 December 2012 00:43, Robert Haas robertmh...@gmail.com wrote:
On Wed, Dec 19, 2012 at 12:39 PM, Simon Riggs si...@2ndquadrant.com wrote:
The benefit of saying that only UPDATEs clean the block is that this
penalises only the workload making the mess, rather than everybody
cleaning up
On 20 December 2012 00:10, Pavan Deolasee pavan.deola...@gmail.com wrote:
I just thought that we can fairly easily limit the damage if we are
really worried about SELECTs being penalised. What if we set a
configurable limit on *extra* things that a query may do which is
otherwise not very
On Wed, Dec 19, 2012 at 11:12 PM, Pavan Deolasee
pavan.deola...@gmail.com wrote:
Yeah, VM buffer contention can become prominent if we break the
invariant that page level bit status implies the vm bit status, at
least when its clear.OTOH IMHO we need some mechanism to address the
issue of
On Thu, Dec 20, 2012 at 4:58 AM, Simon Riggs si...@2ndquadrant.com wrote:
ISTM that if someone spots a block that could use cleanup, they mark
the block as BM_PIN_COUNT_WAITER, but don't set pid. Then when they
unpin the block they send a signal/queue work for a special cleaning
process to
On Thu, Dec 20, 2012 at 9:23 PM, Robert Haas robertmh...@gmail.com wrote:
On Wed, Dec 19, 2012 at 11:12 PM, Pavan Deolasee
I'm very reluctant to suggest that we can solve
this my setting aside another page-level bit to track visibility of
tuples for heapscans. Or even have a bit in the tuple
Pavan Deolasee pavan.deola...@gmail.com writes:
Ok. Will try to read archives to see what was actually suggested and
why it was put on back burner. BTW at the risk of being shot down
again, I wonder if can we push down the freeze operation to HOT prune
also.
Seems unlikely to be a win. We
On Thu, Dec 20, 2012 at 11:49 AM, Pavan Deolasee
pavan.deola...@gmail.com wrote:
I wonder if we should add a flag to heap_page_prune and try to do some
additional work if its being called from lazy vacuum such as setting
the VM bit and the tuple freeze. IIRC I had put something like that in
On Thu, Dec 20, 2012 at 10:55 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Pavan Deolasee pavan.deola...@gmail.com writes:
Ok. Will try to read archives to see what was actually suggested and
why it was put on back burner. BTW at the risk of being shot down
again, I wonder if can we push down the
On Thu, Dec 20, 2012 at 10:59 PM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Dec 20, 2012 at 11:49 AM, Pavan Deolasee
pavan.deola...@gmail.com wrote:
I wonder if we should add a flag to heap_page_prune and try to do some
additional work if its being called from lazy vacuum such as
Pavan Deolasee pavan.deola...@gmail.com writes:
On Thu, Dec 20, 2012 at 10:55 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Seems unlikely to be a win. We only care about freezing tuples in the
context of being able to advance a relation-wide relfrozenxid horizon.
Freezing pages retail accomplishes
On Thu, Dec 20, 2012 at 1:30 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Pavan Deolasee pavan.deola...@gmail.com writes:
On Thu, Dec 20, 2012 at 10:55 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Seems unlikely to be a win. We only care about freezing tuples in the
context of being able to advance a
On Wednesday, December 19, 2012, Robert Haas wrote:
On Wed, Dec 19, 2012 at 12:26 PM, Pavan Deolasee
pavan.deola...@gmail.com javascript:; wrote:
I would like to run some pgbench tests where we get the system in a
steady state such as all/most updates are HOT updates (not entirely
On Thursday, December 20, 2012, Robert Haas wrote:
IMHO, it's probably fairly hopeless to make a pure pgbench workload
show a benefit from index-only scans. A large table under a very
heavy, completely random write workload is just about the worst
possible case for index-only scans.
On Sat, Dec 15, 2012 at 4:48 PM, Tom Lane t...@sss.pgh.pa.us wrote:
On the other hand, the HOT prune operation itself is worthless when
we're running a SELECT. The only reason we do it that way is that we
have to prune before the query starts to use the page, else pruning
might invalidate
On Wed, Dec 19, 2012 at 8:32 PM, Robert Haas robertmh...@gmail.com wrote:
On Sat, Dec 15, 2012 at 4:48 PM, Tom Lane t...@sss.pgh.pa.us wrote:
On the other hand, the HOT prune operation itself is worthless when
we're running a SELECT. The only reason we do it that way is that we
have to prune
Robert Haas robertmh...@gmail.com writes:
On Sat, Dec 15, 2012 at 4:48 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Maybe it's time to look at what it'd take for the low-level scan
operations to know whether they're scanning the target relation of
an UPDATE query, so that we could skip pruning
On Wed, Dec 19, 2012 at 9:51 PM, Tom Lane t...@sss.pgh.pa.us wrote:
If we start generating a lot of useless WAL activity and I/O as
a result of thrashing the all-visible bit, it won't be so tolerable
anymore.
What if we wrap that into the WAL generated by HOT prune itself ?
Would that address
On Wed, Dec 19, 2012 at 11:49 AM, Pavan Deolasee
pavan.deola...@gmail.com wrote:
Also, if extra WAL-logging is really worrisome, may be we should again
seriously reconsider my idea of *not* clearing the bit at HOT update.
My apologies for repeating myself. But that seems very important in a
On Wed, Dec 19, 2012 at 10:40 PM, Robert Haas robertmh...@gmail.com wrote:
On Wed, Dec 19, 2012 at 11:49 AM, Pavan Deolasee
pavan.deola...@gmail.com wrote:
Also, if extra WAL-logging is really worrisome, may be we should again
seriously reconsider my idea of *not* clearing the bit at HOT
On 19 December 2012 16:21, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Sat, Dec 15, 2012 at 4:48 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Maybe it's time to look at what it'd take for the low-level scan
operations to know whether they're scanning the target
Pavan Deolasee pavan.deola...@gmail.com writes:
On Wed, Dec 19, 2012 at 9:51 PM, Tom Lane t...@sss.pgh.pa.us wrote:
If we start generating a lot of useless WAL activity and I/O as
a result of thrashing the all-visible bit, it won't be so tolerable
anymore.
What if we wrap that into the WAL
On 19 December 2012 17:26, Pavan Deolasee pavan.deola...@gmail.com wrote:
We definitely
made great progress by having HOT
Yes, definitely. Great work. That is not for debate.
But I think generally HOT made great difference to the
system as a whole, may be at a cost of slowdown for some
On Thu, Dec 20, 2012 at 12:22 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Pavan Deolasee pavan.deola...@gmail.com writes:
What if we wrap that into the WAL generated by HOT prune itself ?
What WAL? The case we're worried about here is that there's nothing
else for HOT prune to do.
Does such a
On Thu, Dec 20, 2012 at 5:05 AM, Simon Riggs si...@2ndquadrant.com wrote:
And those last things are being debated hotly. We definitely need to
ask whether the way things are now can be tweaked to be better. The
major mechanics need not be reviewed, but the tradeoffs and balances?
Definitely.
On Wed, Dec 19, 2012 at 12:26 PM, Pavan Deolasee
pavan.deola...@gmail.com wrote:
This can be handled by breaking 1-to-1 mapping on VM bit and
PD_ALL_VISIBLE bit. So seq scans will only look at PD_ALL_VISIBLE. It
was proposed by Andres up thread, but shot down by Tom and Simon. But
I still feel
On Wed, Dec 19, 2012 at 12:39 PM, Simon Riggs si...@2ndquadrant.com wrote:
The benefit of saying that only UPDATEs clean the block is that this
penalises only the workload making the mess, rather than everybody
cleaning up repeatedly over one messy guy.
Right, but there are plenty of
On Thu, Dec 20, 2012 at 6:12 AM, Robert Haas robertmh...@gmail.com wrote:
On Wed, Dec 19, 2012 at 12:26 PM, Pavan Deolasee
pavan.deola...@gmail.com wrote:
This can be handled by breaking 1-to-1 mapping on VM bit and
PD_ALL_VISIBLE bit. So seq scans will only look at PD_ALL_VISIBLE. It
was
On Thursday, December 20, 2012 6:14 AM Robert Haas wrote:
On Wed, Dec 19, 2012 at 12:39 PM, Simon Riggs si...@2ndquadrant.com
wrote:
The benefit of saying that only UPDATEs clean the block is that this
penalises only the workload making the mess, rather than everybody
cleaning up
On Sunday, December 16, 2012 11:14 PM Tom Lane wrote:
Pavan Deolasee pavan.deola...@gmail.com writes:
On Sun, Dec 16, 2012 at 3:10 PM, Simon Riggs si...@2ndquadrant.com
wrote:
As explained above, I disagree that it looks like a good idea, and
you've shown no evidence it would be or is
On Sun, Dec 16, 2012 at 11:14 PM, Tom Lane t...@sss.pgh.pa.us wrote:
And I agree with him that your proposed
redefinition of the bit's meaning to avoid that is pretty horrid;
it's ugly, complicates the invariant quite a lot, and breaks some
existing usages of the bit.
(slammed.. feels the
On 16 December 2012 07:53, Pavan Deolasee pavan.deola...@gmail.com wrote:
On Sun, Dec 16, 2012 at 3:18 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Simon Riggs si...@2ndquadrant.com writes:
Doing that only makes sense when we're running a SELECT. Setting the
all visible bit immediately prior to an
On 2012-12-16 13:23:56 +0530, Pavan Deolasee wrote:
Another idea could have been to NOT clear the visibility bit when a
HOT update happens. Such tuple can get pruned by HOT prune, so we
don't need vacuum per se, and the index-only scans are not affected
because the update was a HOT update, so
On 2012-12-15 16:48:08 -0500, Tom Lane wrote:
Simon Riggs si...@2ndquadrant.com writes:
Doing that only makes sense when we're running a SELECT. Setting the
all visible bit immediately prior to an UPDATE that clears it again is
pointless effort, generating extra work for no reason.
On the
On Sun, Dec 16, 2012 at 3:10 PM, Simon Riggs si...@2ndquadrant.com wrote:
As explained above, I disagree that it looks like a good idea, and
you've shown no evidence it would be or is true.
Lets separate out these two issues. What you are suggesting as a
follow up to Tom's idea, I've no
On 16 December 2012 14:41, Andres Freund and...@2ndquadrant.com wrote:
On 2012-12-16 13:23:56 +0530, Pavan Deolasee wrote:
Another idea could have been to NOT clear the visibility bit when a
HOT update happens. Such tuple can get pruned by HOT prune, so we
don't need vacuum per se, and the
On 2012-12-16 16:25:03 +, Simon Riggs wrote:
On 16 December 2012 14:41, Andres Freund and...@2ndquadrant.com wrote:
On 2012-12-16 13:23:56 +0530, Pavan Deolasee wrote:
Another idea could have been to NOT clear the visibility bit when a
HOT update happens. Such tuple can get pruned by
Pavan Deolasee pavan.deola...@gmail.com writes:
On Sun, Dec 16, 2012 at 3:10 PM, Simon Riggs si...@2ndquadrant.com wrote:
As explained above, I disagree that it looks like a good idea, and
you've shown no evidence it would be or is true.
Lets separate out these two issues. What you are
We discussed this idea in the past [1] and Robert recently again
mentioned this in another thread [2]. Please see a rebased/revised
patch attached with the mail. This is mostly similar to what I's
submitted in [1] except some additions to also compute visibility
cut-off XID. I also removed a
On 15 December 2012 12:42, Pavan Deolasee pavan.deola...@gmail.com wrote:
We discussed this idea in the past [1] and Robert recently again
mentioned this in another thread [2]. Please see a rebased/revised
patch attached with the mail. This is mostly similar to what I's
submitted in [1] except
Simon Riggs si...@2ndquadrant.com writes:
Doing that only makes sense when we're running a SELECT. Setting the
all visible bit immediately prior to an UPDATE that clears it again is
pointless effort, generating extra work for no reason.
On the other hand, the HOT prune operation itself is
On Sun, Dec 16, 2012 at 3:18 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Simon Riggs si...@2ndquadrant.com writes:
Doing that only makes sense when we're running a SELECT. Setting the
all visible bit immediately prior to an UPDATE that clears it again is
pointless effort, generating extra work for
42 matches
Mail list logo
