Re: improving wraparound behavior

2019-05-06 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-05-03 23:08:44 -0400, Stephen Frost wrote: > > As mentioned though, it's likely to be a > > quite rare thing to run into, so you'd have to be extra unlucky to even > > hit this case and perhaps the extra code complication just isn't

Re: improving wraparound behavior

2019-05-06 Thread Andres Freund
Hi, On 2019-05-03 23:08:44 -0400, Stephen Frost wrote: > This makes me think we should both order it and combine it into one > message... but that'd then be pretty difficult to deal with, > potentially, from a translation standpoint and just from a "wow, that's > a huge log message", which is kind

Re: improving wraparound behavior

2019-05-03 Thread Euler Taveira
Em sex, 3 de mai de 2019 às 17:27, Robert Haas escreveu: > > HINT: Commit or roll back old prepared transactions, drop stale > replication slots, or kill long-running sessions. > Ensure that autovacuum is progressing, or run a manual database-wide VACUUM. > First of all, +1 for this patch. But af

Re: improving wraparound behavior

2019-05-03 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-05-03 22:41:11 -0400, Stephen Frost wrote: > > I suppose it is a pretty big change in the base autovacuum launcher to > > be something that's run per database instead and then deal with the > > coordination between the two... but I

Re: improving wraparound behavior

2019-05-03 Thread Andres Freund
Hi, On 2019-05-03 22:41:11 -0400, Stephen Frost wrote: > I suppose it is a pretty big change in the base autovacuum launcher to > be something that's run per database instead and then deal with the > coordination between the two... but I can't help but feel like it > wouldn't be that much *work*.

Re: improving wraparound behavior

2019-05-03 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-05-03 22:03:18 -0400, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > > > I am not sure exactly how to fix this, > > > because the calculation we use to determine the XID that can be used > > > to vacuum a spe

Re: improving wraparound behavior

2019-05-03 Thread Andres Freund
Hi, On 2019-05-03 22:03:18 -0400, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > I am not sure exactly how to fix this, > > because the calculation we use to determine the XID that can be used > > to vacuum a specific table is pretty complex; how can the postmaster > > kno

Re: improving wraparound behavior

2019-05-03 Thread Andres Freund
Hi, On 2019-05-03 22:11:08 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > I don't think we necessarily need a new WAL record for what I'm > > describing above (as XLOG_SMGR_TRUNCATE already carries information > > about which forks are truncated, we could just have i

Re: improving wraparound behavior

2019-05-03 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > I don't think we necessarily need a new WAL record for what I'm > describing above (as XLOG_SMGR_TRUNCATE already carries information > about which forks are truncated, we could just have it acquire the > exclusive lock), and I don't think w

Re: improving wraparound behavior

2019-05-03 Thread Andres Freund
Hi, On 2019-05-03 21:36:24 -0400, Robert Haas wrote: > On Fri, May 3, 2019 at 8:45 PM Andres Freund wrote: > > Part of my opposition to just disabling it when close to a wraparound, > > is that it still allows to get close to wraparound because of truncation > > issues. > > Sure ... it would def

Re: improving wraparound behavior

2019-05-03 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > I am not sure exactly how to fix this, > because the calculation we use to determine the XID that can be used > to vacuum a specific table is pretty complex; how can the postmaster > know whether it's going to be able to make any progress i

Re: improving wraparound behavior

2019-05-03 Thread Robert Haas
On Fri, May 3, 2019 at 8:45 PM Andres Freund wrote: > Part of my opposition to just disabling it when close to a wraparound, > is that it still allows to get close to wraparound because of truncation > issues. Sure ... it would definitely be better if vacuum didn't consume XIDs when it truncates.

Re: improving wraparound behavior

2019-05-03 Thread Andres Freund
Hi, On 2019-05-03 18:42:35 -0400, Robert Haas wrote: > On Fri, May 3, 2019 at 4:47 PM Andres Freund wrote: > > I'd actually say the proper fix would be to instead move the truncation > > to *after* finishing updating relfrozenxid etc. If we truncate, the > > additional cost of another in-place p

Re: improving wraparound behavior

2019-05-03 Thread Tom Lane
Robert Haas writes: > I spent a significant chunk of today burning through roughly 2^31 XIDs > just to see what would happen. ... > 2. Once you get to the point where you start to emit errors when > attempting to assign an XID, you can still run plain old VACUUM > because it doesn't consume an XI

Re: improving wraparound behavior

2019-05-03 Thread Robert Haas
On Fri, May 3, 2019 at 4:47 PM Andres Freund wrote: > I'd actually say the proper fix would be to instead move the truncation > to *after* finishing updating relfrozenxid etc. If we truncate, the > additional cost of another in-place pg_class update, to update relpages, > is basically insignifica

Re: improving wraparound behavior

2019-05-03 Thread Andres Freund
Hi, On 2019-05-03 16:26:46 -0400, Robert Haas wrote: > 2. Once you get to the point where you start to emit errors when > attempting to assign an XID, you can still run plain old VACUUM > because it doesn't consume an XID ... except that if it tries to > truncate the relation, then it will take Ac

improving wraparound behavior

2019-05-03 Thread Robert Haas
I spent a significant chunk of today burning through roughly 2^31 XIDs just to see what would happen. My test setup consisted of autovacuum=off plus a trivial prepared transaction plus a lot of this: +BeginInternalSubTransaction("txid_burn"); +(void) GetCurrentTransactionId(); +