Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-16 Thread Peter Eisentraut
On lör, 2009-08-15 at 16:55 -0700, Jeff Davis wrote: Similarly, you could call vacuum_freeze_min_age the maximum age a tuple can be before a vacuum will freeze it. Heh, you could also call max_connections the minimum number of connections before the server will refuse new connection attempts.

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-15 Thread Peter Eisentraut
On fre, 2009-08-14 at 13:57 -0700, Jeff Davis wrote: Looking at the definitions of vacuum_freeze_min_age and autovacuum_freeze_max_age there seems to be almost no distinction between min and max in those two names. For min, the action happens at or above the min values. For max, the action

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-15 Thread Jeff Davis
On Sun, 2009-08-16 at 02:02 +0300, Peter Eisentraut wrote: For min, the action happens at or above the min values. For max, the action happens at or below the max value. From the docs, 23.1.4: autovacuum is invoked on any table that might contain XIDs older than the age specified by the

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-14 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: Yes. There are two ways to do the threshold: 1. Constant fraction of vacuum_freeze_min_age 2. Extra GUC I lean toward #1, because it avoids an extra GUC*, and it avoids the awkwardness when the lower setting is higher than the higher setting. I tend

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-14 Thread Jeff Davis
On Fri, 2009-08-14 at 14:37 -0400, Tom Lane wrote: I tend to agree with Josh that you do need to offer two knobs. But expressing the second knob as a fraction (with range 0 to 1) might be better than an independent min parameter. As you say, that'd be useful to prevent people from setting

[HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Jeff Davis
[ moving to -hackers ] If this topic has been discussed previously, please point me to the earlier threads. Why aren't we more opportunistic about freezing tuples? For instance, if we already have a dirty buffer in cache, we should be more aggressive about freezing those tuples than freezing

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Alvaro Herrera
Jeff Davis wrote: Why aren't we more opportunistic about freezing tuples? For instance, if we already have a dirty buffer in cache, we should be more aggressive about freezing those tuples than freezing tuples on disk. The most widely cited reason is that you lose forensics data. Although

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Jeff Davis
On Thu, 2009-08-13 at 17:58 -0400, Alvaro Herrera wrote: The most widely cited reason is that you lose forensics data. Although they are increasingly rare, there are still situations in which the heap tuple machinery messes up and the xmin/xmax/etc fields of the tuple are the best/only way to

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Robert Haas
On Thu, Aug 13, 2009 at 5:33 PM, Jeff Davispg...@j-davis.com wrote: Or, perhaps when the bgwriter is flushing dirty buffers, it can look for opportunities to set hint bits or freeze tuples. One of the tricky things here is that the time you are mostly likely to want to do this is when you are

Re: [PERFORM] Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Josh Berkus
Why aren't we more opportunistic about freezing tuples? For instance, if we already have a dirty buffer in cache, we should be more aggressive about freezing those tuples than freezing tuples on disk. The most widely cited reason is that you lose forensics data. Although they are

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: Let's say that we had a range like 50-100M, where if it's older than 100M, we freeze it, and if it's older than 50M we freeze it only if it's on a dirty page. We would still have forensic evidence, but we could make a range such that we avoid writing

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Jeff Davis
On Thu, 2009-08-13 at 18:46 -0400, Tom Lane wrote: Yeah, making the limit slushy would doubtless save some writes, with not a lot of downside. OK, then should we make this a TODO? I'll make an attempt at this. And people who don't care about forensic evidence can set it to 0-100M.

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Thu, 2009-08-13 at 18:46 -0400, Tom Lane wrote: Everybody *thinks* they don't care about forensic evidence. Until they need it. We already allow setting vacuum_freeze_min_age to zero, so I don't see a solution here other than documentation. Yeah, we

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Josh Berkus
Jeff, Tom, Let's say that we had a range like 50-100M, where if it's older than 100M, we freeze it, and if it's older than 50M we freeze it only if it's on a dirty page. We would still have forensic evidence, but we could make a range such that we avoid writing multiple times. Yeah, making

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Josh Berkus
What are you envisioning exactly? If vacuum finds any reason to dirty a page (or it's already dirty), then freeze everything on the page that's got age some lower threshold? I was envisioning, if the page is already dirty and in memory *for any reason*, the freeze rows at below some

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Jeff Davis
On Thu, 2009-08-13 at 19:05 -0400, Tom Lane wrote: What are you envisioning exactly? If vacuum finds any reason to dirty a page (or it's already dirty), then freeze everything on the page that's got age some lower threshold? Yes. There are two ways to do the threshold: 1. Constant fraction

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Jeff Davis
On Thu, 2009-08-13 at 18:25 -0400, Robert Haas wrote: On Thu, Aug 13, 2009 at 5:33 PM, Jeff Davispg...@j-davis.com wrote: Or, perhaps when the bgwriter is flushing dirty buffers, it can look for opportunities to set hint bits or freeze tuples. One of the tricky things here is that the time

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: What are you envisioning exactly? If vacuum finds any reason to dirty a page (or it's already dirty), then freeze everything on the page that's got age some lower threshold? I was envisioning, if the page is already dirty and in memory *for any

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Greg Stark
On Fri, Aug 14, 2009 at 12:07 AM, Josh Berkusj...@agliodbs.com wrote: freeze this table agressively if it's in memory, but wait a long time to vaccuum if it's on disk Waitasec, in memory? There are two projects here: 1) Make vacuum when it's freezing tuples freeze every tuple lesser age if

Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Greg Stark
On Fri, Aug 14, 2009 at 12:21 AM, Tom Lanet...@sss.pgh.pa.us wrote: I was envisioning, if the page is already dirty and in memory *for any reason*, the freeze rows at below some threshold. I believe we've had this discussion before.  I do *NOT* want freezing operations pushed into any random