Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-09-26 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Heikki Linnakangas wrote: Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Maybe we should keep

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-27 Thread Bruce Momjian
Heikki Linnakangas wrote: Alvaro Herrera wrote: Heikki Linnakangas wrote: I ran two 24h test runs with DBT-2, one with the patch and one without. To get comparable, predictable results, I turned autovacuum off and run a manual vacuum in a loop on the stock-table alone. As

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-27 Thread Heikki Linnakangas
Bruce Momjian wrote: Heikki Linnakangas wrote: Alvaro Herrera wrote: Heikki Linnakangas wrote: I ran two 24h test runs with DBT-2, one with the patch and one without. To get comparable, predictable results, I turned autovacuum off and run a manual vacuum in a loop on the stock-table alone.

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-27 Thread Gregory Stark
Heikki Linnakangas [EMAIL PROTECTED] writes: But what surprises me is that response times went up a with the patch. I don't know why. Maybe because of increased contention of ProcArrayLock? (I assume you are using that, althought I haven't seen the patch) I am, but I doubt that's it. The

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-27 Thread Heikki Linnakangas
Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Yes, at least for now. I can't believe the patch actually hurts performance, but I'm not going to spend time investigating it. Isn't this exactly what you would expect? It will clean up more tuples so it'll dirty more pages.

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-27 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Bruce Momjian wrote: So are you stopping work on the patch? I assume so. Yes, at least for now. I can't believe the patch actually hurts performance, but I'm not going to spend time investigating it. Are we withdrawing the patch from

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-27 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Bruce Momjian wrote: So are you stopping work on the patch? I assume so. Yes, at least for now. I can't believe the patch actually hurts performance, but I'm not going to spend time investigating it. Are we withdrawing the

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-27 Thread Bruce Momjian
Heikki Linnakangas wrote: Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Bruce Momjian wrote: So are you stopping work on the patch? I assume so. Yes, at least for now. I can't believe the patch actually hurts performance, but I'm not going to spend time investigating

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-27 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Maybe we should keep this issue open until we resolve the vacuum WAL flush issue? I can then rerun the same tests to see if this patch is a win after that. Sounds like a plan, if you are willing to do that. Sure, just rerunning

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-13 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Alvaro Herrera wrote: Was this revisited? Arranging the tests has taken me longer than I thought, but I now finally have the hardware and DBT-2 set up. I just finished a pair of 2h tests with autovacuum off, and continuous vacuum of the stock table. I'm trying to

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-13 Thread Alvaro Herrera
Heikki Linnakangas wrote: I ran two 24h test runs with DBT-2, one with the patch and one without. To get comparable, predictable results, I turned autovacuum off and run a manual vacuum in a loop on the stock-table alone. As expected, the steady-state of the stock table is smaller with

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-03-13 Thread Heikki Linnakangas
Alvaro Herrera wrote: Heikki Linnakangas wrote: I ran two 24h test runs with DBT-2, one with the patch and one without. To get comparable, predictable results, I turned autovacuum off and run a manual vacuum in a loop on the stock-table alone. As expected, the steady-state of the stock

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-02-26 Thread NikhilS
Hi, I was wondering if we can apply the same logic of recalculating OldestXmin within IndexBuildHeapScan which occurs as part of create index operation? Having to index lesser number of tuples should be a good save in the CREATE INDEX CONCURRENTLY case, if the above is possible? Regards,

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-02-23 Thread Heikki Linnakangas
Alvaro Herrera wrote: I'm wondering if there has been any effort to make this work in conjunction with ITAGAKI Takahiro's patch to correct the dead tuple count estimate. I just looked at that patch. If we applied both patches, the dead_tuples estimate would be off by the number of dead tuples

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-02-19 Thread Alvaro Herrera
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: BTW I've got serious reservations about whether this bit is safe: + /* The table could've grown since vacuum started, and there + * might already be dead

Re: [PATCHES] Recalculating OldestXmin in a long-running vacuum

2007-02-19 Thread Heikki Linnakangas
Alvaro Herrera wrote: Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: BTW I've got serious reservations about whether this bit is safe: + /* The table could've grown since vacuum started, and there +* might already

Re: [PATCHES] [pgsql-patches] Recalculating OldestXmin in a long-running vacuum

2007-02-04 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: BTW I've got serious reservations about whether this bit is safe: + /* The table could've grown since vacuum started, and there +* might already be dead tuples on the new pages. Catch them

Re: [PATCHES] [pgsql-patches] Recalculating OldestXmin in a long-running vacuum

2007-02-03 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: I have two runs of DBT-2, one with the patch and one without. Patched: autovac public.stock scans:1 pages:1285990(-0) tuples:25303056(-2671265) CPU 95.22s/38.02u sec elapsed 10351.17 sec Unpatched: autovac public.stock

Re: [pgsql-patches] Recalculating OldestXmin in a long-running vacuum

2007-01-17 Thread Gregory Stark
Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: In any case I'd like to see some evidence of significant real-world benefit before adding such a conceptual wart ... I've asked our testers to do a TPC-C run with and without the patch. I'm not expecting it to make a huge

[pgsql-patches] Recalculating OldestXmin in a long-running vacuum

2007-01-16 Thread Heikki Linnakangas
Hi, It seems to me that we could easily reclaim a bit more dead tuples in a vacuum by recalculating the OldestXmin every now and then. In a large table with a constant stream of updates/deletes and concurrent vacuums, this could make a big difference. With the attached patch, OldestXmin is

Re: [pgsql-patches] Recalculating OldestXmin in a long-running vacuum

2007-01-16 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: It seems to me that we could easily reclaim a bit more dead tuples in a vacuum by recalculating the OldestXmin every now and then. Doesn't this break relfrozenxid maintenance? In any case I'd like to see some evidence of significant real-world

Re: [pgsql-patches] Recalculating OldestXmin in a long-running vacuum

2007-01-16 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: It seems to me that we could easily reclaim a bit more dead tuples in a vacuum by recalculating the OldestXmin every now and then. Doesn't this break relfrozenxid maintenance? Not AFAICS. relfrozenxid is nowadays updated with