Re: [HACKERS] Visibility map and freezing

2009-01-22 Thread Heikki Linnakangas
Heikki Linnakangas wrote: ITAGAKI Takahiro wrote: - What relation are there between autovacuum_freeze_max_age, vacuum_freeze_min_age and vacuum_freeze_table_age? If we increase one of them, should we also increase the others? Yeah, that's a fair question. I'll try to work a doc patch to

Re: [HACKERS] Visibility map and freezing

2009-01-22 Thread Heikki Linnakangas
Euler Taveira de Oliveira wrote: Simon Riggs escreveu: On Tue, 2009-01-20 at 21:51 +0200, Heikki Linnakangas wrote: Attached is a simple patch to only start skipping pages after 20 consecutive pages marked as visible in the visibility map. This doesn't do any look-ahead, so it will always

Re: [HACKERS] Visibility map and freezing

2009-01-20 Thread Heikki Linnakangas
ITAGAKI Takahiro wrote: Gregory Stark st...@enterprisedb.com wrote: I don't think we can perfectly capture the meaning of these GUCs in the name. I think our goal should be to avoid confusion between them. I was thinking it would be clearer if the options which control *when* autovacuum fires

Re: [HACKERS] Visibility map and freezing

2009-01-20 Thread Simon Riggs
On Tue, 2009-01-20 at 11:03 +0200, Heikki Linnakangas wrote: - Are there some conditions where whole-table-scanning vacuum is more effective than vacuums using visibility map? If so, we should switch to full-scan *automatically*, without relying on user configurations. Hmm, the

Re: [HACKERS] Visibility map and freezing

2009-01-20 Thread Heikki Linnakangas
Simon Riggs wrote: On Tue, 2009-01-20 at 11:03 +0200, Heikki Linnakangas wrote: - Are there some conditions where whole-table-scanning vacuum is more effective than vacuums using visibility map? If so, we should switch to full-scan *automatically*, without relying on user

Re: [HACKERS] Visibility map and freezing

2009-01-20 Thread Simon Riggs
On Tue, 2009-01-20 at 21:51 +0200, Heikki Linnakangas wrote: Attached is a simple patch to only start skipping pages after 20 consecutive pages marked as visible in the visibility map. This doesn't do any look-ahead, so it will always scan the first 20 pages of a table before it starts to

Re: [HACKERS] Visibility map and freezing

2009-01-20 Thread Euler Taveira de Oliveira
Simon Riggs escreveu: On Tue, 2009-01-20 at 21:51 +0200, Heikki Linnakangas wrote: Attached is a simple patch to only start skipping pages after 20 consecutive pages marked as visible in the visibility map. This doesn't do any look-ahead, so it will always scan the first 20 pages of a

Re: [HACKERS] Visibility map and freezing

2009-01-18 Thread ITAGAKI Takahiro
Gregory Stark st...@enterprisedb.com wrote: I don't think we can perfectly capture the meaning of these GUCs in the name. I think our goal should be to avoid confusion between them. I was thinking it would be clearer if the options which control *when* autovacuum fires off a worker

Re: [HACKERS] Visibility map and freezing

2009-01-16 Thread Gregory Stark
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Jeff Davis wrote: On Fri, 2009-01-09 at 13:49 +0200, Heikki Linnakangas wrote: I'm now leaning towards: autovacuum_freeze_max_age vacuum_freeze_table_age vacuum_freeze_min_age where autovacuum_freeze_max_age and

Re: [HACKERS] Visibility map and freezing

2009-01-16 Thread Heikki Linnakangas
Alvaro Herrera wrote: Heikki Linnakangas escribió: This patch adds a new column to pg_autovacuum, reflecting the new vacuum_freeze_table_age GUC just like freeze_min_age column reflects vacuum_freeze_min_age. If I commit this tomorrow, will I cause you a lot of trouble with the reloptions

Re: [HACKERS] Visibility map and freezing

2009-01-15 Thread Heikki Linnakangas
Jeff Davis wrote: On Fri, 2009-01-09 at 13:49 +0200, Heikki Linnakangas wrote: If the distinction you're making is that autovacuum_freeze_max_age affects the launching of a vacuum rather than the behavior of a vacuum, maybe we could incorporate the word launch like:

Re: [HACKERS] Visibility map and freezing

2009-01-15 Thread Alvaro Herrera
Heikki Linnakangas escribió: Alvaro, are you getting rid of pg_autovacuum in favor of the reloptions for 8.4? That's the intention, yes. This patch adds a new column to pg_autovacuum, reflecting the new vacuum_freeze_table_age GUC just like freeze_min_age column reflects

Re: [HACKERS] Visibility map and freezing

2009-01-09 Thread Heikki Linnakangas
Jeff Davis wrote: On Wed, 2009-01-07 at 09:34 +0200, Heikki Linnakangas wrote: autovacuum_freeze_max_age - autovacuum_freeze_scan_age vacuum_freeze_max_age - vacuum_freeze_scan_age vacuum_freeze_min_age - vacuum_freeze_tuple_age *_scan_age settings control when the table is fully scanned

Re: [HACKERS] Visibility map and freezing

2009-01-09 Thread Jeff Davis
On Fri, 2009-01-09 at 13:49 +0200, Heikki Linnakangas wrote: Thinking about this some more, I'm not too happy with those names either. vacuum_freeze_scan_age and autovacuum_freeze_scan_age don't mean quite the same thing, like vacuum_cost_delay and autovacuum_vacuum_cost_delay do, for

Re: [HACKERS] Visibility map and freezing

2009-01-07 Thread Jeff Davis
On Wed, 2009-01-07 at 09:34 +0200, Heikki Linnakangas wrote: autovacuum_freeze_max_age - autovacuum_freeze_scan_age vacuum_freeze_max_age - vacuum_freeze_scan_age vacuum_freeze_min_age - vacuum_freeze_tuple_age *_scan_age settings control when the table is fully scanned to freeze tuples

Re: [HACKERS] Visibility map and freezing

2009-01-06 Thread Heikki Linnakangas
Jeff Davis wrote: On Mon, 2008-12-22 at 21:24 +0200, Heikki Linnakangas wrote: Introduce new vacuum_freeze_max_age setting. Manual VACUUM will scan the whole table and advance relfrozenxid, if relfrozenxid is older than vacuum_freeze_max_age. It's confusing to have two GUCs named

Re: [HACKERS] Visibility map and freezing

2008-12-23 Thread Jeff Davis
On Mon, 2008-12-22 at 21:24 +0200, Heikki Linnakangas wrote: Introduce new vacuum_freeze_max_age setting. Manual VACUUM will scan the whole table and advance relfrozenxid, if relfrozenxid is older than vacuum_freeze_max_age. It's confusing to have two GUCs named vacuum_freeze_min_age and

Re: [HACKERS] Visibility map and freezing

2008-12-22 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Peter Eisentraut wrote: Heikki Linnakangas wrote: I think we need a threshold similar to autovacuum_freeze_max_age for manual vacuums as well: vacuum_freeze_max_age. If you run VACUUM, and relfrozenxid is older than vacuum_freeze_max_age, the visibility map is

Re: [HACKERS] Visibility map and freezing

2008-12-22 Thread Fujii Masao
On Tue, Dec 23, 2008 at 4:24 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Heikki Linnakangas wrote: Peter Eisentraut wrote: Heikki Linnakangas wrote: I think we need a threshold similar to autovacuum_freeze_max_age for manual vacuums as well: vacuum_freeze_max_age. If

Re: [HACKERS] Visibility map and freezing

2008-12-18 Thread Peter Eisentraut
Heikki Linnakangas wrote: I think we need a threshold similar to autovacuum_freeze_max_age for manual vacuums as well: vacuum_freeze_max_age. If you run VACUUM, and relfrozenxid is older than vacuum_freeze_max_age, the visibility map is ignored and all pages are scanned. Would one parameter

Re: [HACKERS] Visibility map and freezing

2008-12-18 Thread Heikki Linnakangas
Peter Eisentraut wrote: Heikki Linnakangas wrote: I think we need a threshold similar to autovacuum_freeze_max_age for manual vacuums as well: vacuum_freeze_max_age. If you run VACUUM, and relfrozenxid is older than vacuum_freeze_max_age, the visibility map is ignored and all pages are

[HACKERS] Visibility map and freezing

2008-12-17 Thread Heikki Linnakangas
The way VACUUM works with the visibility map is that if any pages are skipped, relfrozenxid can't be updated. That means that plain VACUUM won't advance relfrozenxid, and doesn't protect from XID wraparound. We discussed this in the context of autovacuum before, and we have that covered now.

Re: [HACKERS] Visibility map and freezing

2008-12-17 Thread Gregory Stark
Zdenek Kotala zdenek.kot...@sun.com writes: What's about add second bit which mark frozen page (all tuples have freeze XID)? It should avoid full scan, but extend size of map. That would only really work if you have a very static table where entire pages get frozen and stay frozen long before

Re: [HACKERS] Visibility map and freezing

2008-12-17 Thread Heikki Linnakangas
Zdenek Kotala wrote: What's about add second bit which mark frozen page (all tuples have freeze XID)? It should avoid full scan, but extend size of map. First of all, we'd still have to make the decision of when to scan pages that need freezing, and when to only scan pages that have dead

Re: [HACKERS] Visibility map and freezing

2008-12-17 Thread Zdenek Kotala
What's about add second bit which mark frozen page (all tuples have freeze XID)? It should avoid full scan, but extend size of map. Zdenek Heikki Linnakangas napsal(a): The way VACUUM works with the visibility map is that if any pages are skipped, relfrozenxid can't