Re: [HACKERS] Rework the way multixact truncations work

2015-12-12 Thread Robert Haas
On Sat, Dec 12, 2015 at 12:02 PM, Andres Freund wrote: > Noah, Robert, All > > On 2015-12-11 11:20:21 -0500, Robert Haas wrote: >> On Thu, Dec 10, 2015 at 9:32 AM, Robert Haas wrote: >> >> Adding a 'prevOffsetStopLimit' and using it seems like a ~5 line patch. >> > >> > So let's do that, then. >>

Re: [HACKERS] Rework the way multixact truncations work

2015-12-12 Thread Andres Freund
Noah, Robert, All On 2015-12-11 11:20:21 -0500, Robert Haas wrote: > On Thu, Dec 10, 2015 at 9:32 AM, Robert Haas wrote: > >> Adding a 'prevOffsetStopLimit' and using it seems like a ~5 line patch. > > > > So let's do that, then. > > Who is going to take care of this? Here's two patches: 1) Th

Re: [HACKERS] Rework the way multixact truncations work

2015-12-11 Thread Noah Misch
On Thu, Dec 10, 2015 at 08:55:54AM -0500, Robert Haas wrote: > I don't know have anything to add to what others have said in response > to this point, except this: the whole point of using a source code > management system is to tell you what changed and when. What you are > proposing to do makes

Re: [HACKERS] Rework the way multixact truncations work

2015-12-11 Thread Robert Haas
On Thu, Dec 10, 2015 at 9:32 AM, Robert Haas wrote: > On Thu, Dec 10, 2015 at 9:04 AM, Andres Freund wrote: >> On 2015-12-10 08:55:54 -0500, Robert Haas wrote: >>> Maybe. But I think we could use a little more vigorous discussion of >>> that issue, since Andres doesn't seem to be very convinced

Re: [HACKERS] Rework the way multixact truncations work

2015-12-10 Thread Robert Haas
On Thu, Dec 10, 2015 at 9:04 AM, Andres Freund wrote: > On 2015-12-10 08:55:54 -0500, Robert Haas wrote: >> Maybe. But I think we could use a little more vigorous discussion of >> that issue, since Andres doesn't seem to be very convinced by your >> analysis, and I don't currently understand what

Re: [HACKERS] Rework the way multixact truncations work

2015-12-10 Thread Andres Freund
On 2015-12-10 08:55:54 -0500, Robert Haas wrote: > Maybe. But I think we could use a little more vigorous discussion of > that issue, since Andres doesn't seem to be very convinced by your > analysis, and I don't currently understand what you've fixed because I > can't, as mentioned several times,

Re: [HACKERS] Rework the way multixact truncations work

2015-12-10 Thread Robert Haas
On Wed, Dec 9, 2015 at 8:23 PM, Noah Misch wrote: > On Wed, Dec 09, 2015 at 11:08:32AM -0500, Robert Haas wrote: >> On Wed, Dec 9, 2015 at 9:43 AM, Noah Misch wrote: >> > I hope those who have not already read commit 4f627f8 will not waste time >> > reading it. They should instead ignore multixa

Re: [HACKERS] Rework the way multixact truncations work

2015-12-10 Thread Bert
+1 On Thu, Dec 10, 2015 at 9:58 AM, Peter Geoghegan wrote: > On Thu, Dec 10, 2015 at 12:34 AM, Andres Freund > wrote: > >> > Ripping it out and replacing it monolithically will not > >> > change that; it will only make the detailed history harder to > >> > reconstruct, and I *will* want to reco

Re: [HACKERS] Rework the way multixact truncations work

2015-12-10 Thread Peter Geoghegan
On Thu, Dec 10, 2015 at 12:34 AM, Andres Freund wrote: >> > Ripping it out and replacing it monolithically will not >> > change that; it will only make the detailed history harder to >> > reconstruct, and I *will* want to reconstruct it. >> >> What's something that might happen six months from now

Re: [HACKERS] Rework the way multixact truncations work

2015-12-10 Thread Andres Freund
On 2015-12-09 20:23:06 -0500, Noah Misch wrote: > By the way, it occurs to me that I should also make pg_upgrade blacklist the > range of catversions that might have data loss. No sense in putting ourselves > in the position of asking whether data files of a 9.9.3 cluster spent time in > a 9.5beta

Re: [HACKERS] Rework the way multixact truncations work

2015-12-09 Thread Noah Misch
On Wed, Dec 09, 2015 at 11:08:32AM -0500, Robert Haas wrote: > On Wed, Dec 9, 2015 at 9:43 AM, Noah Misch wrote: > > I hope those who have not already read commit 4f627f8 will not waste time > > reading it. They should instead ignore multixact changes from commit > > 4f627f8 > > through its reve

Re: [HACKERS] Rework the way multixact truncations work

2015-12-09 Thread Andres Freund
On 2015-12-09 11:18:39 -0500, Robert Haas wrote: > If I correctly understand the scenario that you are describing, that > does happen - not for the same MXID, but for different ones. At least > the last time I checked, and I'm not sure if we've fixed this, it > could happen because the SLRU page t

Re: [HACKERS] Rework the way multixact truncations work

2015-12-09 Thread Robert Haas
On Wed, Dec 9, 2015 at 10:41 AM, Andres Freund wrote: >> (I am glad you talked the author out of back-patching; otherwise, >> 9.4.5 and 9.3.10 would have introduced a data loss bug.) > > Isn't that a bug in a, as far as we know, impossible scenario? Unless I > miss something there's no known case

Re: [HACKERS] Rework the way multixact truncations work

2015-12-09 Thread Robert Haas
On Wed, Dec 9, 2015 at 9:43 AM, Noah Misch wrote: > Andres writing the patch that became commit 4f627f8 and you reviewing it were > gifts to Alvaro and to the community. Aware of that, I have avoided[1] saying > that I was shocked to see that commit's defects. Despite a committer-author > and _t

Re: [HACKERS] Rework the way multixact truncations work

2015-12-09 Thread Andres Freund
On 2015-12-09 09:43:19 -0500, Noah Misch wrote: > Aware of that, I have avoided[1] saying that I was shocked to see that > commit's defects. Despite a committer-author and _two_ committer > reviewers, the patch was rife with wrong new comments, omitted updates > to comments it caused to become wro

Re: [HACKERS] Rework the way multixact truncations work

2015-12-09 Thread Noah Misch
On Tue, Dec 08, 2015 at 01:05:03PM -0500, Robert Haas wrote: > On Tue, Dec 8, 2015 at 6:43 AM, Andres Freund wrote: > > On 2015-12-04 21:55:29 -0500, Noah Misch wrote: > >> On Thu, Dec 03, 2015 at 07:03:21PM +0100, Andres Freund wrote: > >> > Sorry, but I really just want to see these changes as i

Re: [HACKERS] Rework the way multixact truncations work

2015-12-08 Thread Robert Haas
On Tue, Dec 8, 2015 at 6:43 AM, Andres Freund wrote: > On 2015-12-04 21:55:29 -0500, Noah Misch wrote: >> On Thu, Dec 03, 2015 at 07:03:21PM +0100, Andres Freund wrote: >> > Sorry, but I really just want to see these changes as iterative patches >> > ontop of 9.5/HEAD instead of this process. I wo

Re: [HACKERS] Rework the way multixact truncations work

2015-12-08 Thread Andres Freund
On 2015-12-04 21:55:29 -0500, Noah Misch wrote: > On Thu, Dec 03, 2015 at 07:03:21PM +0100, Andres Freund wrote: > > Sorry, but I really just want to see these changes as iterative patches > > ontop of 9.5/HEAD instead of this process. I won't revert the reversion > > if you push it anyway, but I t

Re: [HACKERS] Rework the way multixact truncations work

2015-12-04 Thread Noah Misch
On Thu, Dec 03, 2015 at 07:03:21PM +0100, Andres Freund wrote: > On 2015-12-03 04:38:45 -0500, Noah Misch wrote: > > On Wed, Dec 02, 2015 at 04:46:26PM +0100, Andres Freund wrote: > > > Especially if reverting and redoing includes conflicts that mainly > > > increases the chance of accidental bugs.

Re: [HACKERS] Rework the way multixact truncations work

2015-12-03 Thread Andres Freund
On 2015-12-03 04:38:45 -0500, Noah Misch wrote: > On Wed, Dec 02, 2015 at 04:46:26PM +0100, Andres Freund wrote: > > Especially if reverting and redoing includes conflicts that mainly > > increases the chance of accidental bugs. > > True. (That doesn't apply to these patches.) Uh, it does. You h

Re: [HACKERS] Rework the way multixact truncations work

2015-12-03 Thread Noah Misch
On Wed, Dec 02, 2015 at 04:46:26PM +0100, Andres Freund wrote: > On 2015-12-02 09:57:19 -0500, Noah Misch wrote: > > Hackers have been too reticent to revert and redo defect-laden > > commits. If doing that is weird today, let it be normal. > > Why? See my paragraph ending with the two sentences

Re: [HACKERS] Rework the way multixact truncations work

2015-12-02 Thread Andres Freund
On 2015-12-02 09:57:19 -0500, Noah Misch wrote: > On Tue, Dec 01, 2015 at 05:07:15PM -0500, Robert Haas wrote: > > I think it's weird to back a commit out only to put a bunch of very similar > > stuff back in. > > I agree with that. If the original patches and their replacements shared 95% > of di

Re: [HACKERS] Rework the way multixact truncations work

2015-12-02 Thread Noah Misch
On Tue, Dec 01, 2015 at 05:07:15PM -0500, Robert Haas wrote: > I think it's weird to back a commit out only to put a bunch of very similar > stuff back in. I agree with that. If the original patches and their replacements shared 95% of diff lines in common, we wouldn't be having this conversation

Re: [HACKERS] Rework the way multixact truncations work

2015-12-01 Thread Peter Geoghegan
On Tue, Dec 1, 2015 at 2:07 PM, Robert Haas wrote: > Hmm. I read Peter's message as agreeing with Andres rather than with > you. And I have to say I agree with Andres as well. I think it's > weird to back a commit out only to put a bunch of very similar stuff > back in. Your interpretation was

Re: [HACKERS] Rework the way multixact truncations work

2015-12-01 Thread Robert Haas
On Fri, Nov 27, 2015 at 5:16 PM, Noah Misch wrote: > On Mon, Nov 23, 2015 at 11:44:45AM -0800, Peter Geoghegan wrote: >> On Sun, Nov 8, 2015 at 11:52 AM, Noah Misch wrote: >> >> I'm not following along right now - in order to make cleanups the plan is >> >> to revert a couple commits and then re

Re: [HACKERS] Rework the way multixact truncations work

2015-11-27 Thread Noah Misch
On Mon, Nov 23, 2015 at 11:44:45AM -0800, Peter Geoghegan wrote: > On Sun, Nov 8, 2015 at 11:52 AM, Noah Misch wrote: > >> I'm not following along right now - in order to make cleanups the plan is > >> to revert a couple commits and then redo them prettyfied? > > > > Yes, essentially. Given the

Re: [HACKERS] Rework the way multixact truncations work

2015-11-23 Thread Peter Geoghegan
On Sun, Nov 8, 2015 at 11:52 AM, Noah Misch wrote: >> I'm not following along right now - in order to make cleanups the plan is to >> revert a couple commits and then redo them prettyfied? > > Yes, essentially. Given the volume of updates, this seemed neater than > framing those updates as in-tr

Re: [HACKERS] Rework the way multixact truncations work

2015-11-16 Thread Noah Misch
On Sun, Nov 08, 2015 at 11:59:33AM -0800, Andres Freund wrote: > On November 8, 2015 11:52:05 AM PST, Noah Misch wrote: > >On Sun, Nov 08, 2015 at 11:11:42AM -0800, Andres Freund wrote: > >> On November 8, 2015 12:54:07 AM PST, Noah Misch wrote: > >> > >> >I have pushed a stack of branches to >

Re: [HACKERS] Rework the way multixact truncations work

2015-11-08 Thread Andres Freund
On November 8, 2015 11:52:05 AM PST, Noah Misch wrote: >On Sun, Nov 08, 2015 at 11:11:42AM -0800, Andres Freund wrote: >> On November 8, 2015 12:54:07 AM PST, Noah Misch >wrote: >> >> >I have pushed a stack of branches to >> >https://github.com/nmisch/postgresql.git: >> > >> >mxt0-revert - rever

Re: [HACKERS] Rework the way multixact truncations work

2015-11-08 Thread Noah Misch
On Sun, Nov 08, 2015 at 11:11:42AM -0800, Andres Freund wrote: > On November 8, 2015 12:54:07 AM PST, Noah Misch wrote: > > >I have pushed a stack of branches to > >https://github.com/nmisch/postgresql.git: > > > >mxt0-revert - reverts commits 4f627f8 and aa29c1c > >mxt1-disk-independent - see be

Re: [HACKERS] Rework the way multixact truncations work

2015-11-08 Thread Andres Freund
On November 8, 2015 12:54:07 AM PST, Noah Misch wrote: >I have pushed a stack of branches to >https://github.com/nmisch/postgresql.git: > >mxt0-revert - reverts commits 4f627f8 and aa29c1c >mxt1-disk-independent - see below >mxt2-cosmetic - update already-wrong comments and formatting >mxt3-main

Re: [HACKERS] Rework the way multixact truncations work

2015-11-08 Thread Noah Misch
On Thu, Oct 29, 2015 at 08:46:52AM +0100, Andres Freund wrote: > On October 29, 2015 7:59:03 AM GMT+01:00, Noah Misch > wrote: > >That helps; thanks. Your design seems good. I've located only insipid > >defects. > > Great! > > > I propose to save some time by writing a patch series > >elimina

Re: [HACKERS] Rework the way multixact truncations work

2015-10-29 Thread Andres Freund
Hi, On October 29, 2015 7:59:03 AM GMT+01:00, Noah Misch wrote: >On Tue, Oct 27, 2015 at 03:44:10PM +0100, Andres Freund wrote: >> On 2015-10-24 22:07:00 -0400, Noah Misch wrote: >> > On Tue, Sep 22, 2015 at 07:57:27PM +0200, Andres Freund wrote: >> > > On 2015-09-22 13:38:58 -0400, Robert Haas w

Re: [HACKERS] Rework the way multixact truncations work

2015-10-29 Thread Noah Misch
On Tue, Oct 27, 2015 at 03:44:10PM +0100, Andres Freund wrote: > On 2015-10-24 22:07:00 -0400, Noah Misch wrote: > > On Tue, Sep 22, 2015 at 07:57:27PM +0200, Andres Freund wrote: > > > On 2015-09-22 13:38:58 -0400, Robert Haas wrote: > > > > - If SlruDeleteSegment fails in unlink(), shouldn't we a

Re: [HACKERS] Rework the way multixact truncations work

2015-10-27 Thread Josh Berkus
On 10/27/2015 07:44 AM, Andres Freund wrote: >> Unlinking old pg_clog files is strictly an optimization. If you were to >> > comment out every unlink() call in slru.c, the only ill effect on CLOG is >> > the >> > waste of disk space. Is the same true of MultiXact? > Well, multixacts are a lot la

Re: [HACKERS] Rework the way multixact truncations work

2015-10-27 Thread Andres Freund
Hi, On 2015-10-24 22:07:00 -0400, Noah Misch wrote: > I'm several days into a review of this change (commits 4f627f8 and > aa29c1c). Cool! > There's one part of the design I want to understand before commenting on > specific code. What did you anticipate to be the consequences of failing to > r

Re: [HACKERS] Rework the way multixact truncations work

2015-10-24 Thread Noah Misch
I'm several days into a review of this change (commits 4f627f8 and aa29c1c). There's one part of the design I want to understand before commenting on specific code. What did you anticipate to be the consequences of failing to remove SLRU segment files that MultiXactState->oldestMultiXactId implies

Re: [HACKERS] Rework the way multixact truncations work

2015-09-29 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Sep 28, 2015 at 10:48 PM, Jim Nasby wrote: > > Maybe I'm confused, but I thought the whole purpose of this was to get rid > > of the risk associated with that calculation in favor of explicit truncation > > boundaries in the WAL log. > > Yes. But if the master hasn't

Re: [HACKERS] Rework the way multixact truncations work

2015-09-29 Thread Joel Jacobson
On Tue, Sep 22, 2015 at 3:20 PM, Andres Freund wrote: > What I've tested is the following: > * continous burning of multis, both triggered via members and offsets > * a standby keeping up when the primary is old > * a standby keeping up when the primary is new > * basebackups made while a new prim

Re: [HACKERS] Rework the way multixact truncations work

2015-09-29 Thread Robert Haas
On Mon, Sep 28, 2015 at 10:48 PM, Jim Nasby wrote: > Maybe I'm confused, but I thought the whole purpose of this was to get rid > of the risk associated with that calculation in favor of explicit truncation > boundaries in the WAL log. Yes. But if the master hasn't been updated yet, then we stil

Re: [HACKERS] Rework the way multixact truncations work

2015-09-29 Thread Andres Freund
On 2015-09-28 21:48:00 -0500, Jim Nasby wrote: > On 9/28/15 8:49 PM, Robert Haas wrote: > >If at some point we back-patch this further, then it potentially > >becomes a live issue, but I would like to respectfully inquire what > >exactly you think making it a PANIC would accomplish? There are a lo

Re: [HACKERS] Rework the way multixact truncations work

2015-09-28 Thread Jim Nasby
On 9/28/15 8:49 PM, Robert Haas wrote: If at some point we back-patch this further, then it potentially becomes a live issue, but I would like to respectfully inquire what exactly you think making it a PANIC would accomplish? There are a lot of scary things about this patch, but the logic for de

Re: [HACKERS] Rework the way multixact truncations work

2015-09-28 Thread Robert Haas
On Mon, Sep 28, 2015 at 5:47 PM, Jim Nasby wrote: > There was discussion about making this a PANIC instead of a LOG, which I > think is a good idea... but then there'd need to be some way to not PANIC if > you were doing an upgrade. I think you're worrying about a non-problem. This code has not

Re: [HACKERS] Rework the way multixact truncations work

2015-09-28 Thread Jim Nasby
On 9/27/15 2:25 PM, Andres Freund wrote: On 2015-09-27 14:21:08 -0500, Jim Nasby wrote: IMHO doing just a log of something this serious; it should at least be a WARNING. In postgres LOG, somewhat confusingly, is more severe than WARNING. Ahh, right. Which in this case stinks, because WARNING

Re: [HACKERS] Rework the way multixact truncations work

2015-09-27 Thread Andres Freund
On 2015-09-27 14:21:08 -0500, Jim Nasby wrote: > IMHO doing just a log of something this serious; it should at least be a > WARNING. In postgres LOG, somewhat confusingly, is more severe than WARNING. > I think the concern about upgrading a replica before the master is valid; is > there some way

Re: [HACKERS] Rework the way multixact truncations work

2015-09-27 Thread Jim Nasby
On 9/23/15 1:48 PM, Andres Freund wrote: Honestly, I wonder whether this message >ereport(LOG, >(errmsg("performing legacy multixact truncation"), > errdetail("Legacy truncations are sometimes performed

Re: [HACKERS] Rework the way multixact truncations work

2015-09-26 Thread Tom Lane
Andres Freund writes: > Should this get a release note entry given that we're not (at least > immediately) backpatching this? I'll probably put something in when I update the release notes for beta1 (next week sometime); no real need to deal with it individually. regards,

Re: [HACKERS] Rework the way multixact truncations work

2015-09-26 Thread Andres Freund
Hi, I pushed this to 9.5 and master, committing the xlog page magic bump separately. To avoid using a magic value from master in 9.5 I bumped the numbers by two in both branches. Should this get a release note entry given that we're not (at least immediately) backpatching this? Greetings, Andre

Re: [HACKERS] Rework the way multixact truncations work

2015-09-23 Thread Andres Freund
On 2015-09-23 15:57:02 -0300, Alvaro Herrera wrote: > I think we need to make a decision here. Is this a terribly serious > bug/misdesign that needs addressing? Imo yes. Not sure about terribly, but definitely serious. It's several data loss bugs in one package. > If so, we need to backpatch. I

Re: [HACKERS] Rework the way multixact truncations work

2015-09-23 Thread Alvaro Herrera
Andres Freund wrote: > On 2015-09-23 15:03:05 -0300, Alvaro Herrera wrote: > > Honestly, I wonder whether this message > > ereport(LOG, > > (errmsg("performing legacy multixact > > truncation"), > > errde

Re: [HACKERS] Rework the way multixact truncations work

2015-09-23 Thread Andres Freund
On 2015-09-23 15:03:05 -0300, Alvaro Herrera wrote: > The comment on top of TrimMultiXact states that "no locks are needed > here", but then goes on to grab a few locks. Hm. Yea. Although that was the case before. > It's a bit odd that SetMultiXactIdLimit has the "finishedStartup" test > so low.

Re: [HACKERS] Rework the way multixact truncations work

2015-09-23 Thread Alvaro Herrera
The comment on top of TrimMultiXact states that "no locks are needed here", but then goes on to grab a few locks. I think we should remove the comment, or rephrase it to state that we still grab them for consistency or whatever; or perhaps even remove the lock acquisitions. (I think the comment is

Re: [HACKERS] Rework the way multixact truncations work

2015-09-23 Thread Andres Freund
On 2015-09-23 10:29:09 -0300, Alvaro Herrera wrote: > > /* > > + * Delete an individual SLRU segment, identified by the segment number. > > + */ > > +void > > +SlruDeleteSegment(SlruCtl ctl, int segno) > > Is it okay to change the ABI of SlruDeleteSegment? I think so. Any previous user of the AP

Re: [HACKERS] Rework the way multixact truncations work

2015-09-23 Thread Alvaro Herrera
> @@ -1210,8 +1211,14 @@ restart:; > (void) SlruScanDirectory(ctl, SlruScanDirCbDeleteCutoff, &cutoffPage); > } > > -void > -SlruDeleteSegment(SlruCtl ctl, char *filename) > +/* > + * Delete an individual SLRU segment, identified by the filename. > + * > + * NB: This does not touch the SLR

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Andres Freund
On 2015-09-22 20:14:11 -0400, Robert Haas wrote: > I'm not disagreeing with any of that. I'm just disagreeing with you > on the likelihood that we're going to make things better vs. making > them worse. But, really, I've said everything I have to say about > this. You have a commit bit. I'm not

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Robert Haas
On Tue, Sep 22, 2015 at 7:45 PM, Andres Freund wrote: > On 2015-09-23 01:24:31 +0200, Andres Freund wrote: >> I think we put at least three layers on bandaid on this issue since >> 9.3.2, and each layer made things more complicated. > > 2a9b01928f193f529b885ac577051c4fd00bd427 - Cope with possible

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Andres Freund
On 2015-09-23 01:24:31 +0200, Andres Freund wrote: > I think we put at least three layers on bandaid on this issue since > 9.3.2, and each layer made things more complicated. 2a9b01928f193f529b885ac577051c4fd00bd427 - Cope with possible failure of the oldest MultiXact to exist. 5bbac7ec1b5754043e

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Andres Freund
On 2015-09-22 14:52:49 -0400, Robert Haas wrote: > 1. It would be possible to write a patch that included ONLY the > changes needed to make that happen, and did nothing else. It would be > largely a subset of this. If we want to change 9.3 and 9.4, I > recommend we do that first, and then come ba

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Robert Haas
On Tue, Sep 22, 2015 at 1:57 PM, Andres Freund wrote: > On 2015-09-22 13:38:58 -0400, Robert Haas wrote: >> Regarding 0003, I'm still very much not convinced that it's a good >> idea to apply this to 9.3 and 9.4. This patch changes the way we do >> truncation in those older releases; instead of h

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Andres Freund
On 2015-09-22 13:38:58 -0400, Robert Haas wrote: > Regarding 0003, I'm still very much not convinced that it's a good > idea to apply this to 9.3 and 9.4. This patch changes the way we do > truncation in those older releases; instead of happening at a > restartpoint, it happens when oldestMultiXid

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Alvaro Herrera
Robert Haas wrote: > Regarding 0003, I'm still very much not convinced that it's a good > idea to apply this to 9.3 and 9.4. This patch changes the way we do > truncation in those older releases; instead of happening at a > restartpoint, it happens when oldestMultiXid advances. I admit that I >

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Robert Haas
On Tue, Sep 22, 2015 at 9:20 AM, Andres Freund wrote: > On 2015-09-21 16:36:03 +0200, Andres Freund wrote: >> Agreed. I'll update the patch. > > Here's updated patches against master. These include the "legacy" > truncation support. There's no meaningful functional differences in this > version ex

Re: [HACKERS] Rework the way multixact truncations work

2015-09-22 Thread Andres Freund
On 2015-07-02 11:52:04 -0400, Robert Haas wrote: > On Mon, Jun 29, 2015 at 3:48 PM, Andres Freund wrote: > > New version attached. > > 0002 looks good, but the commit message should perhaps mention the > comment fix. Or commit that separately. I'm inclined to backpatch the applicable parts to 9

Re: [HACKERS] Rework the way multixact truncations work

2015-09-21 Thread Andres Freund
On 2015-09-21 10:30:59 -0700, Josh Berkus wrote: > On 09/21/2015 07:36 AM, Andres Freund wrote: > > On 2015-09-21 10:31:17 -0400, Robert Haas wrote: > >> So, where are we with this patch? > > > > Uh. I'd basically been waiting on further review and then forgot about > > it. > > Does the current p

Re: [HACKERS] Rework the way multixact truncations work

2015-09-21 Thread Josh Berkus
On 09/21/2015 07:36 AM, Andres Freund wrote: > On 2015-09-21 10:31:17 -0400, Robert Haas wrote: >> So, where are we with this patch? > > Uh. I'd basically been waiting on further review and then forgot about > it. Does the current plan to never expire XIDs in 9.6 affect multixact truncation at al

Re: [HACKERS] Rework the way multixact truncations work

2015-09-21 Thread Andres Freund
On 2015-09-21 10:31:17 -0400, Robert Haas wrote: > On Sun, Jul 5, 2015 at 3:16 PM, Andres Freund wrote: > >>On the other hand, in the common case, by the time we perform a > >>restartpoint, we're consistent: I think the main exception to that is > >>if we do a base backup that spans multiple check

Re: [HACKERS] Rework the way multixact truncations work

2015-09-21 Thread Robert Haas
On Sun, Jul 5, 2015 at 3:16 PM, Andres Freund wrote: >>On the other hand, in the common case, by the time we perform a >>restartpoint, we're consistent: I think the main exception to that is >>if we do a base backup that spans multiple checkpoints. I think that >>in the new location, the chances

Re: [HACKERS] Rework the way multixact truncations work

2015-07-05 Thread Andres Freund
On July 5, 2015 8:50:57 PM GMT+02:00, Robert Haas wrote: >On Sun, Jul 5, 2015 at 2:28 PM, Andres Freund >wrote: >> (quick answer, off now) >> >> On 2015-07-05 14:20:11 -0400, Robert Haas wrote: >>> On Thu, Jul 2, 2015 at 2:28 PM, Andres Freund >wrote: >>> > On 2015-07-02 13:58:45 -0400, Robert H

Re: [HACKERS] Rework the way multixact truncations work

2015-07-05 Thread Robert Haas
On Sun, Jul 5, 2015 at 2:28 PM, Andres Freund wrote: > (quick answer, off now) > > On 2015-07-05 14:20:11 -0400, Robert Haas wrote: >> On Thu, Jul 2, 2015 at 2:28 PM, Andres Freund wrote: >> > On 2015-07-02 13:58:45 -0400, Robert Haas wrote: >> >> I seriously, seriously doubt that it is a good id

Re: [HACKERS] Rework the way multixact truncations work

2015-07-05 Thread Andres Freund
(quick answer, off now) On 2015-07-05 14:20:11 -0400, Robert Haas wrote: > On Thu, Jul 2, 2015 at 2:28 PM, Andres Freund wrote: > > On 2015-07-02 13:58:45 -0400, Robert Haas wrote: > >> I seriously, seriously doubt that it is a good idea to perform the > >> legacy truncation from MultiXactAdvance

Re: [HACKERS] Rework the way multixact truncations work

2015-07-05 Thread Robert Haas
On Thu, Jul 2, 2015 at 2:28 PM, Andres Freund wrote: > On 2015-07-02 13:58:45 -0400, Robert Haas wrote: >> On Thu, Jul 2, 2015 at 11:52 AM, Robert Haas wrote: >> > Will look at 0003 next. >> >> +appendStringInfo(buf, "offsets [%u, %u), members [%u, %u)", >> >> I don't think we typically u

Re: [HACKERS] Rework the way multixact truncations work

2015-07-02 Thread Andres Freund
On 2015-07-02 13:58:45 -0400, Robert Haas wrote: > On Thu, Jul 2, 2015 at 11:52 AM, Robert Haas wrote: > > Will look at 0003 next. > > +appendStringInfo(buf, "offsets [%u, %u), members [%u, %u)", > > I don't think we typically use this style for notating intervals. I don't think we real

Re: [HACKERS] Rework the way multixact truncations work

2015-07-02 Thread Robert Haas
On Thu, Jul 2, 2015 at 11:52 AM, Robert Haas wrote: > Will look at 0003 next. +appendStringInfo(buf, "offsets [%u, %u), members [%u, %u)", I don't think we typically use this style for notating intervals. case XLOG_MULTIXACT_CREATE_ID: id = "CREATE_ID";

Re: [HACKERS] Rework the way multixact truncations work

2015-07-02 Thread Robert Haas
On Mon, Jun 29, 2015 at 3:48 PM, Andres Freund wrote: > New version attached. 0002 looks good, but the commit message should perhaps mention the comment fix. Or commit that separately. Will look at 0003 next. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL C

Re: [HACKERS] Rework the way multixact truncations work

2015-06-29 Thread Andres Freund
On 2015-06-29 23:54:40 +1200, Thomas Munro wrote: > On Mon, Jun 22, 2015 at 7:24 AM, Andres Freund wrote: > > It'd be very welcome to see some wider testing and review on this. > > I started looking at this and doing some testing. Here is some > initial feedback: > > Perhaps vac_truncate_clog n

Re: [HACKERS] Rework the way multixact truncations work

2015-06-29 Thread Thomas Munro
On Mon, Jun 22, 2015 at 7:24 AM, Andres Freund wrote: > It'd be very welcome to see some wider testing and review on this. I started looking at this and doing some testing. Here is some initial feedback: Perhaps vac_truncate_clog needs a new name now that it does more, maybe vac_truncate_transa

Re: [HACKERS] Rework the way multixact truncations work

2015-06-27 Thread Andres Freund
On 2015-06-26 14:48:35 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > Rework the way multixact truncations work. > > I spent some time this morning reviewing this patch and had some > comments that I relayed over IM to Andres. Thanks for that! > 2. We set PGXACT->delayChkpt while

Re: [HACKERS] Rework the way multixact truncations work

2015-06-26 Thread Alvaro Herrera
Andres Freund wrote: > Rework the way multixact truncations work. I spent some time this morning reviewing this patch and had some comments that I relayed over IM to Andres. The vast majority is cosmetic, but there are two larger things: 1. I think this part of PerformMembersTruncation() is