Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Wed, Oct 19, 2016 at 9:09 PM, Bruce Momjianwrote: > On Wed, Oct 19, 2016 at 08:17:46PM -0400, Robert Haas wrote: >> On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjian wrote: >> > Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms >> > of xids, that a standby query can run before cancel, like >> > old_snapshot_threshold, no? >> >> No, not really. It affects the behavior of the master, not the standby. > > But it controls when/if cancels happen on the standby. True. I see your point. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Wed, Oct 19, 2016 at 08:17:46PM -0400, Robert Haas wrote: > On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjianwrote: > > Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms > > of xids, that a standby query can run before cancel, like > > old_snapshot_threshold, no? > > No, not really. It affects the behavior of the master, not the standby. But it controls when/if cancels happen on the standby. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjianwrote: > Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms > of xids, that a standby query can run before cancel, like > old_snapshot_threshold, no? No, not really. It affects the behavior of the master, not the standby. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On 10/19/2016 09:59 AM, Bruce Momjian wrote: > On Wed, Oct 19, 2016 at 09:00:06AM -0400, Robert Haas wrote: >> On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjianwrote: >>> On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote: Actually, I think vacuum_defer_cleanup_age is, and always has been, an ugly hack. But for some people it may be the ugly hack that is letting them continue to use PostgreSQL. >>> >>> I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby >>> servers --- it cancels transactions rather than delaying cleanup. >> >> I think it's the opposite, isn't it? vacuum_defer_cleanup_age >> prevents cancellations. > > Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms > of xids, that a standby query can run before cancel, like > old_snapshot_threshold, no? After that, we can cancel standby queries. > I see hot_standby_feedback as our current behavior on the master where > we never cancel standby queries. > > To me, hot_standby_feedback extends no-cleanup-no-cancel from the > standby to the master, while vacuum_defer_cleanup_age behaves like > old_snapshot_threshold in that it causes cancel for long-running > queries. See Andres' response on this thread. He's already covered why the setting is still useful, but why we might want to remove it anyway. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Wed, Oct 19, 2016 at 09:00:06AM -0400, Robert Haas wrote: > On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjianwrote: > > On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote: > >> On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus wrote: > >> > Based on that argument, we would never be able to remove any > >> > configuration parameter ever. > >> > >> Well... no. Based on that argument, we should only remove > >> configuration parameters if we're fairly certain that they are not > >> useful any more, which will be rare, but is not never. I agree that > >> *if* vacuum_defer_cleanup_age is no longer useful, it should be > >> removed. I'm just not convinced that it's truly obsolete, and you > >> haven't really offered much of an argument for that proposition. It > >> does something sufficiently different from hot_standby_feedback that > >> I'm not sure it's accurate to say that one can substitute for the > >> other, and indeed, I see Andres has already suggested some scenarios > >> where it could still be useful. > >> > >> Actually, I think vacuum_defer_cleanup_age is, and always has been, an > >> ugly hack. But for some people it may be the ugly hack that is > >> letting them continue to use PostgreSQL. > > > > I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby > > servers --- it cancels transactions rather than delaying cleanup. > > I think it's the opposite, isn't it? vacuum_defer_cleanup_age > prevents cancellations. Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms of xids, that a standby query can run before cancel, like old_snapshot_threshold, no? After that, we can cancel standby queries. I see hot_standby_feedback as our current behavior on the master where we never cancel standby queries. To me, hot_standby_feedback extends no-cleanup-no-cancel from the standby to the master, while vacuum_defer_cleanup_age behaves like old_snapshot_threshold in that it causes cancel for long-running queries. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjianwrote: > On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote: >> On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus wrote: >> > Based on that argument, we would never be able to remove any >> > configuration parameter ever. >> >> Well... no. Based on that argument, we should only remove >> configuration parameters if we're fairly certain that they are not >> useful any more, which will be rare, but is not never. I agree that >> *if* vacuum_defer_cleanup_age is no longer useful, it should be >> removed. I'm just not convinced that it's truly obsolete, and you >> haven't really offered much of an argument for that proposition. It >> does something sufficiently different from hot_standby_feedback that >> I'm not sure it's accurate to say that one can substitute for the >> other, and indeed, I see Andres has already suggested some scenarios >> where it could still be useful. >> >> Actually, I think vacuum_defer_cleanup_age is, and always has been, an >> ugly hack. But for some people it may be the ugly hack that is >> letting them continue to use PostgreSQL. > > I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby > servers --- it cancels transactions rather than delaying cleanup. I think it's the opposite, isn't it? vacuum_defer_cleanup_age prevents cancellations. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote: > On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkuswrote: > > Based on that argument, we would never be able to remove any > > configuration parameter ever. > > Well... no. Based on that argument, we should only remove > configuration parameters if we're fairly certain that they are not > useful any more, which will be rare, but is not never. I agree that > *if* vacuum_defer_cleanup_age is no longer useful, it should be > removed. I'm just not convinced that it's truly obsolete, and you > haven't really offered much of an argument for that proposition. It > does something sufficiently different from hot_standby_feedback that > I'm not sure it's accurate to say that one can substitute for the > other, and indeed, I see Andres has already suggested some scenarios > where it could still be useful. > > Actually, I think vacuum_defer_cleanup_age is, and always has been, an > ugly hack. But for some people it may be the ugly hack that is > letting them continue to use PostgreSQL. I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby servers --- it cancels transactions rather than delaying cleanup. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkuswrote: > Based on that argument, we would never be able to remove any > configuration parameter ever. Well... no. Based on that argument, we should only remove configuration parameters if we're fairly certain that they are not useful any more, which will be rare, but is not never. I agree that *if* vacuum_defer_cleanup_age is no longer useful, it should be removed. I'm just not convinced that it's truly obsolete, and you haven't really offered much of an argument for that proposition. It does something sufficiently different from hot_standby_feedback that I'm not sure it's accurate to say that one can substitute for the other, and indeed, I see Andres has already suggested some scenarios where it could still be useful. Actually, I think vacuum_defer_cleanup_age is, and always has been, an ugly hack. But for some people it may be the ugly hack that is letting them continue to use PostgreSQL. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On 10/18/2016 01:37 PM, Andres Freund wrote: > Hi, > > On 2016-10-09 21:51:07 -0700, Josh Berkus wrote: >> Given that hot_standby_feedback is pretty bulletproof now, and a lot of >> the work in reducing replay conflicts, I think the utility of >> vacuum_defer_cleanup_age is at an end. I really meant so submit a patch >> to remove it to 9.6, but it got away from me. > > HS feedback doesn't e.g. work well with delayed and/or archived replay, > whereas defer_cleanup does. Oh, point! See, that's why I polled, I knew there was something I was forgetting about. > On the other hand, removing it would make some of the reasoning around > GetOldestXmin() a bit easier. Enough to make it worth breaking the above? -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
Hi, On 2016-10-09 21:51:07 -0700, Josh Berkus wrote: > Given that hot_standby_feedback is pretty bulletproof now, and a lot of > the work in reducing replay conflicts, I think the utility of > vacuum_defer_cleanup_age is at an end. I really meant so submit a patch > to remove it to 9.6, but it got away from me. HS feedback doesn't e.g. work well with delayed and/or archived replay, whereas defer_cleanup does. On the other hand, removing it would make some of the reasoning around GetOldestXmin() a bit easier. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On 10/18/2016 01:28 PM, Robert Haas wrote: > On Tue, Oct 18, 2016 at 4:18 PM, Josh Berkuswrote: >> On 10/12/2016 05:00 PM, Robert Haas wrote: >>> On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus wrote: Given that hot_standby_feedback is pretty bulletproof now, and a lot of the work in reducing replay conflicts, I think the utility of vacuum_defer_cleanup_age is at an end. I really meant so submit a patch to remove it to 9.6, but it got away from me. Any objections to removing the option in 10? >>> >>> I'm not sure I see the point. >> >> Redusing the number of configuration variables is an a-priori good. In >> aggregate, the more knobs we have, the harder it is to learn how to >> admin Postgres. Therefore any time a config variable becomes obsolete, >> we should remove it. > > Meh. I agree that more configuration knobs makes it harder to learn > to configure the system, but we've got enough of them that removing > exactly one isn't going to make a material difference. Against that, > if you are wrong about it being obsolete and there are actually people > relying on it heavily, those people will be very sad if we remove it, > and unless they read this mailing list, we probably won't find out > until it's too late. Based on that argument, we would never be able to remove any configuration parameter ever. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Tue, Oct 18, 2016 at 4:18 PM, Josh Berkuswrote: > On 10/12/2016 05:00 PM, Robert Haas wrote: >> On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus wrote: >>> Given that hot_standby_feedback is pretty bulletproof now, and a lot of >>> the work in reducing replay conflicts, I think the utility of >>> vacuum_defer_cleanup_age is at an end. I really meant so submit a patch >>> to remove it to 9.6, but it got away from me. >>> >>> Any objections to removing the option in 10? >> >> I'm not sure I see the point. > > Redusing the number of configuration variables is an a-priori good. In > aggregate, the more knobs we have, the harder it is to learn how to > admin Postgres. Therefore any time a config variable becomes obsolete, > we should remove it. Meh. I agree that more configuration knobs makes it harder to learn to configure the system, but we've got enough of them that removing exactly one isn't going to make a material difference. Against that, if you are wrong about it being obsolete and there are actually people relying on it heavily, those people will be very sad if we remove it, and unless they read this mailing list, we probably won't find out until it's too late. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On 10/12/2016 05:00 PM, Robert Haas wrote: > On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkuswrote: >> Given that hot_standby_feedback is pretty bulletproof now, and a lot of >> the work in reducing replay conflicts, I think the utility of >> vacuum_defer_cleanup_age is at an end. I really meant so submit a patch >> to remove it to 9.6, but it got away from me. >> >> Any objections to removing the option in 10? > > I'm not sure I see the point. Redusing the number of configuration variables is an a-priori good. In aggregate, the more knobs we have, the harder it is to learn how to admin Postgres. Therefore any time a config variable becomes obsolete, we should remove it. -- -- Josh Berkus Red Hat OSAS (any opinions are my own) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Remove vacuum_defer_cleanup_age
On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkuswrote: > Given that hot_standby_feedback is pretty bulletproof now, and a lot of > the work in reducing replay conflicts, I think the utility of > vacuum_defer_cleanup_age is at an end. I really meant so submit a patch > to remove it to 9.6, but it got away from me. > > Any objections to removing the option in 10? I'm not sure I see the point. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers