Re: [HACKERS] Interesting glitch in autovacuum

2008-09-11 Thread Tom Lane
Pavan Deolasee [EMAIL PROTECTED] writes: On Wed, Sep 10, 2008 at 9:22 PM, Tom Lane [EMAIL PROTECTED] wrote: On reflection I'm not even sure that this is strictly an autovacuum bug. It can be cast more generically as RecentGlobalXmin getting used without ever having been set, and it sure looks

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-11 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Actually, 8.2 initializes it to InvalidTransactionId too, so it doesn't seem like there's a problem there either. Since the problem stems from using it as initializer for the Min() calculation, there's no actual bug on 8.2 either. The bug only

[HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Tom Lane
I observed a curious bug in autovac just now. Since plain vacuum avoids calling GetTransactionSnapshot, an autovac worker that happens not to analyze any tables will never call GetTransactionSnapshot at all. This means it will arrive at vac_update_datfrozenxid with RecentGlobalXmin never having

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Alvaro Herrera
Tom Lane wrote: I observed a curious bug in autovac just now. Since plain vacuum avoids calling GetTransactionSnapshot, an autovac worker that happens not to analyze any tables will never call GetTransactionSnapshot at all. This means it will arrive at vac_update_datfrozenxid with

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I observed a curious bug in autovac just now. ... Maybe we should boot RecentGlobalXmin with InvalidOid, and ensure where it's going to be used that it's not that. Good idea --- an Assert at the use sites should be sufficient.

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: Lastly, now that we have the PROC_IN_VACUUM test in GetSnapshotData, is it actually necessary for lazy vacuum to avoid setting a snapshot? It seems like it might be a good idea for it to do so in order to keep its

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I'm worried about keeping RecentGlobalXmin up to date during the vacuums, not only at the end, because it will be used for HOT page pruning during the vacuums. Oh, I see. I didn't know we were doing HOT pruning during vacuum; does it

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Heikki Linnakangas
Alvaro Herrera wrote: I didn't know we were doing HOT pruning during vacuum; does it make sense? Removing HOT-updated, dead tuples is a bit complicated. It needs to be done one HOT-chain at a time, and the root line pointer needs to be updated to the next live tuple in the chain.

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Simon Riggs
On Wed, 2008-09-10 at 11:52 -0400, Tom Lane wrote: I'm thinking that maybe an appropriate fix is to insert a GetTransactionSnapshot call at the beginning of InitPostgres' transaction, thus ensuring that every backend has some vaguely sane value for RecentGlobalXmin before it tries to do any

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Wed, 2008-09-10 at 11:52 -0400, Tom Lane wrote: I'm thinking that maybe an appropriate fix is to insert a GetTransactionSnapshot call at the beginning of InitPostgres' transaction, thus ensuring that every backend has some vaguely sane value for

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Alvaro Herrera
Tom Lane wrote: Sorry, I got a bit confused there. The vacuum's intentional pruning will use its own OldestXmin variable, which is known current at the start of the vacuum (and I think there were even proposals to advance it more frequently than that). However, a vacuum could require some

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Simon Riggs
On Wed, 2008-09-10 at 16:11 -0400, Tom Lane wrote: If there were a serious performance argument against that, then yeah, but I don't see one. Maybe! Just finishing other patch then I'll be starting Hot Standby discussions, so we'll see. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: Anyway I think we are on the same page about the rest of the issues. Did you want to work on fixing them, or shall I? Is this more or less what you had in mind? Yeah, this looks like exactly what I had in mind for HEAD. I'm not too

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: Anyway I think we are on the same page about the rest of the issues. Did you want to work on fixing them, or shall I? Is this more or less what you had in mind? Yeah, this looks like exactly what I had in mind

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Tom Lane
I wrote: Yeah, this looks like exactly what I had in mind for HEAD. I'm not too sure about the back branches though. I think we could apply all of it to 8.3, but further back is going to require a separate investigation for each branch. Will you take that on? BTW, I did a quick look at all

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Alvaro Herrera
Tom Lane wrote: BTW, I did a quick look at all the uses of RecentGlobalXmin in the back branches, and I think we might be all right before 8.2. The older branches do in fact init RecentGlobalXmin to InvalidTransactionId, and the only thing they use it for is is this tuple dead to everyone

Re: [HACKERS] Interesting glitch in autovacuum

2008-09-10 Thread Pavan Deolasee
On Wed, Sep 10, 2008 at 9:22 PM, Tom Lane [EMAIL PROTECTED] wrote: On reflection I'm not even sure that this is strictly an autovacuum bug. It can be cast more generically as RecentGlobalXmin getting used without ever having been set, and it sure looks to me like the HOT patch may have