Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-06-07 Thread Greg Stark
On Jun 3, 2011 8:38 PM, "Bruce Momjian" wrote: > > I realize we just read the pages from the kernel to maintain sequential > I/O, but do we actually read the contents of the page if we know it > doesn't need vacuuming? If so, do we need to? I dont follow. What's your question? Tom's final versi

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-06-06 Thread Heikki Linnakangas
On 03.06.2011 22:16, Bruce Momjian wrote: I realize we just read the pages from the kernel to maintain sequential I/O, but do we actually read the contents of the page if we know it doesn't need vacuuming? Yes. If so, do we need to? Not necessarily, but it allows us to freeze old tuples, a

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-06-03 Thread Bruce Momjian
Heikki Linnakangas wrote: > On 27.05.2011 16:52, Pavan Deolasee wrote: > > On closer inspection, I realized that we have > > deliberately put in this hook to ensure that we use visibility maps > > only when we see at least SKIP_PAGES_THRESHOLD worth of all-visible > > sequential pages to take advan

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-29 Thread Pavan Deolasee
On Fri, May 27, 2011 at 8:40 PM, Greg Stark wrote: > > Separately it's a bit strange that we actually have to visit the > pages. We have all the information we need in the VM to determine > whether there's a run of 32 vacuum-clean pages. Why can't we look at > the next 32 pages and if they're all

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Robert Haas
On Fri, May 27, 2011 at 11:10 AM, Greg Stark wrote: > It would be nice if the VM had a bit for "all-frozen" but that > wouldn't help much except in the case of truly cold data. We could > perhaps keep the frozen data per segment or per VM page (which covers > a large section of the table) which wo

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Pavan Deolasee
On Fri, May 27, 2011 at 7:41 PM, Heikki Linnakangas wrote: > On 27.05.2011 16:52, Pavan Deolasee wrote: >> >> On closer inspection, I realized that we have >> deliberately put in this hook to ensure that we use visibility maps >> only when we see at least SKIP_PAGES_THRESHOLD worth of all-visible

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Greg Stark
On Fri, May 27, 2011 at 7:11 AM, Heikki Linnakangas wrote: > Well, as with normal queries, it's usually faster to just seqscan the whole > table if you need to access more than a few percent of the pages, because > sequential I/O is so much faster than random I/O. Well it's not strictly random ac

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Pavan Deolasee
On Fri, May 27, 2011 at 7:36 PM, Tom Lane wrote: > Pavan Deolasee writes: >> My statistical skills are limited, but wouldn't that mean that for a >> fairly well distributed write activity across a large table, if there >> are even 3-4% update/deletes, we would most likely hit a >> not-all-visible

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Cédric Villemain
2011/5/27 Cédric Villemain : > 2011/5/27 Pavan Deolasee : >> I wonder if we have tested the reasoning behind having >> SKIP_PAGES_THRESHOLD and the magic number of 32 assigned to it >> currently. While looking at the code after a long time and doing some >> tests, I realized that a manual VACUUM wo

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Heikki Linnakangas
On 27.05.2011 16:52, Pavan Deolasee wrote: On closer inspection, I realized that we have deliberately put in this hook to ensure that we use visibility maps only when we see at least SKIP_PAGES_THRESHOLD worth of all-visible sequential pages to take advantage of possible OS seq scan optimizations

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Cédric Villemain
2011/5/27 Pavan Deolasee : > I wonder if we have tested the reasoning behind having > SKIP_PAGES_THRESHOLD and the magic number of 32 assigned to it > currently. While looking at the code after a long time and doing some > tests, I realized that a manual VACUUM would always scan first 31 > pages of

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Tom Lane
Pavan Deolasee writes: > My statistical skills are limited, but wouldn't that mean that for a > fairly well distributed write activity across a large table, if there > are even 3-4% update/deletes, we would most likely hit a > not-all-visible page for every 32 pages scanned ? Huh? With a typical

[HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-05-27 Thread Pavan Deolasee
I wonder if we have tested the reasoning behind having SKIP_PAGES_THRESHOLD and the magic number of 32 assigned to it currently. While looking at the code after a long time and doing some tests, I realized that a manual VACUUM would always scan first 31 pages of a relation which has not received an