Re: [HACKERS] what to revert

On Sun, May 15, 2016 at 4:06 PM, Tomas Vondra wrote: > Overall, I think this shows that there seems to be no performance penalty > with "disabled" vs. "reverted" - i.e. even with the least favorable (100% > read-only) workload. OK, that's good, as far as it goes. > Whatever the metric is, I thin

Re: [HACKERS] what to revert

On Tue, May 10, 2016 at 10:05:13AM -0500, Kevin Grittner wrote: > On Tue, May 10, 2016 at 9:02 AM, Tom Lane wrote: > > Kevin Grittner writes: > >> There were 75 samples each of "disabled" and "reverted" in the > >> spreadsheet. Averaging them all, I see this: > > > >> reverted: 290,660 TPS > >>

Re: [HACKERS] what to revert

On 2016-05-10 13:36:32 -0400, Robert Haas wrote: > On Tue, May 10, 2016 at 12:31 PM, Tomas Vondra > wrote: > > The following table shows the differences between the disabled and reverted > > cases like this: > > > > sum('reverted' results with N clients) > >

Re: [HACKERS] what to revert

On Tue, May 10, 2016 at 2:41 PM, Kevin Grittner wrote: > http://www.postgresql.org/message-id/flat/1402267501.4.yahoomail...@web122304.mail.ne1.yahoo.com Re-reading that thread I was reminded that I had more NUMA problems when data all landed in one memory node, as can happen with pgbench -i

Re: [HACKERS] what to revert

On Tue, May 10, 2016 at 11:13 AM, Tomas Vondra wrote: > On 05/10/2016 10:29 AM, Kevin Grittner wrote: >> On Mon, May 9, 2016 at 9:01 PM, Tomas Vondra >> wrote: >>> * It's also seems to me the feature greatly amplifies the >>> variability of the results, somehow. It's not uncommon to see >>> res

Re: [HACKERS] what to revert

Hi, On 05/10/2016 07:36 PM, Robert Haas wrote: On Tue, May 10, 2016 at 12:31 PM, Tomas Vondra wrote: The following table shows the differences between the disabled and reverted cases like this: sum('reverted' results with N clients) - 1.0 su

Re: [HACKERS] what to revert

On Tue, May 10, 2016 at 12:31 PM, Tomas Vondra wrote: > The following table shows the differences between the disabled and reverted > cases like this: > > sum('reverted' results with N clients) > - 1.0 > sum('disabled' results with N clients) > >

Re: [HACKERS] what to revert

On 05/10/2016 03:04 PM, Kevin Grittner wrote: On Tue, May 10, 2016 at 3:29 AM, Kevin Grittner mailto:kgri...@gmail.com>> wrote: * The results are a bit noisy, but I think in general this shows that for certain cases there's a clearly measurable difference (up to 5%) between the "disabled" and

Re: [HACKERS] what to revert

Tomas Vondra wrote: > Yes, I'd like to repeat the tests with other workloads - I'm thinking about > regular pgbench and perhaps something that'd qualify as 'mostly read-only' > (not having a clear idea how that should work). You can use "-bselect-only@9 -bsimple-update@1" for a workload that's 10

Re: [HACKERS] what to revert

Hi, On 05/10/2016 10:29 AM, Kevin Grittner wrote: On Mon, May 9, 2016 at 9:01 PM, Tomas Vondra mailto:tomas.von...@2ndquadrant.com>> wrote: Over the past few days I've been running benchmarks on a fairly large NUMA box (4 sockets, 32 cores / 64 with HR, 256GB of RAM) to see the impact of the '

Re: [HACKERS] what to revert

On Tue, May 10, 2016 at 9:02 AM, Tom Lane wrote: > Kevin Grittner writes: >> There were 75 samples each of "disabled" and "reverted" in the >> spreadsheet. Averaging them all, I see this: > >> reverted: 290,660 TPS >> disabled: 292,014 TPS > >> That's a 0.46% overall increase in performance wi

Re: [HACKERS] what to revert

Kevin Grittner writes: > There were 75 samples each of "disabled" and "reverted" in the > spreadsheet. Averaging them all, I see this: > reverted: 290,660 TPS > disabled: 292,014 TPS > That's a 0.46% overall increase in performance with the patch, > disabled, compared to reverting it. I'm su

Re: [HACKERS] what to revert

On Tue, May 10, 2016 at 3:29 AM, Kevin Grittner wrote: >> * The results are a bit noisy, but I think in general this shows >> that for certain cases there's a clearly measurable difference >> (up to 5%) between the "disabled" and "reverted" cases. This is >> particularly visible on the smallest d

Re: [HACKERS] what to revert

On Mon, May 9, 2016 at 9:01 PM, Tomas Vondra wrote: > Over the past few days I've been running benchmarks on a fairly > large NUMA box (4 sockets, 32 cores / 64 with HR, 256GB of RAM) > to see the impact of the 'snapshot too old' - both when disabled > and enabled with various values in the old_s

Re: [HACKERS] what to revert

Hi, On 05/04/2016 12:42 AM, Andres Freund wrote: On 2016-05-03 20:57:13 +0200, Tomas Vondra wrote: On 05/03/2016 07:41 PM, Andres Freund wrote: ... I'm pretty sure that I said that somewhere else at least once: But to be absolutely clear, I'm *not* really concerned with the performance with t

Re: [HACKERS] what to revert

On Thu, May 5, 2016 at 3:39 AM, Ants Aasma wrote: > 5. mai 2016 6:14 AM kirjutas kuupäeval "Andres Freund" : >> >> On 2016-05-05 06:08:39 +0300, Ants Aasma wrote: >>> On 5 May 2016 1:28 a.m., "Andres Freund" wrote: On 2016-05-04 18:22:27 -0400, Robert Haas wrote: > How would the semantic

Re: [HACKERS] what to revert

5. mai 2016 6:14 AM kirjutas kuupäeval "Andres Freund" : > > On 2016-05-05 06:08:39 +0300, Ants Aasma wrote: > > On 5 May 2016 1:28 a.m., "Andres Freund" wrote: > > > On 2016-05-04 18:22:27 -0400, Robert Haas wrote: > > > > How would the semantics change? > > > > > > Right now the time for computi

Re: [HACKERS] what to revert

On Thu, May 5, 2016 at 8:44 AM, Andres Freund wrote: > > On 2016-05-05 06:08:39 +0300, Ants Aasma wrote: > > On 5 May 2016 1:28 a.m., "Andres Freund" wrote: > > > On 2016-05-04 18:22:27 -0400, Robert Haas wrote: > > > > How would the semantics change? > > > > > > Right now the time for computing

Re: [HACKERS] what to revert

On 2016-05-05 06:08:39 +0300, Ants Aasma wrote: > On 5 May 2016 1:28 a.m., "Andres Freund" wrote: > > On 2016-05-04 18:22:27 -0400, Robert Haas wrote: > > > How would the semantics change? > > > > Right now the time for computing the snapshot is relevant, if > > maintenance of xids is moved, it'll

Re: [HACKERS] what to revert

On 5 May 2016 1:28 a.m., "Andres Freund" wrote: > On 2016-05-04 18:22:27 -0400, Robert Haas wrote: > > How would the semantics change? > > Right now the time for computing the snapshot is relevant, if > maintenance of xids is moved, it'll likely be tied to the time xids are > assigned. That seems

Re: [HACKERS] what to revert

On Wed, May 4, 2016 at 6:28 PM, Andres Freund wrote: > On 2016-05-04 18:22:27 -0400, Robert Haas wrote: >> On Wed, May 4, 2016 at 6:06 PM, Andres Freund wrote: >> >> Some of the proposals involve fairly small tweaks to call >> >> MaintainOldSnapshotTimeMapping() from elsewhere or only when >> >>

Re: [HACKERS] what to revert

On 2016-05-04 18:22:27 -0400, Robert Haas wrote: > On Wed, May 4, 2016 at 6:06 PM, Andres Freund wrote: > >> Some of the proposals involve fairly small tweaks to call > >> MaintainOldSnapshotTimeMapping() from elsewhere or only when > >> something changes (like crossing a minute boundary or seeing

Re: [HACKERS] what to revert

On Wed, May 4, 2016 at 6:06 PM, Andres Freund wrote: >> Some of the proposals involve fairly small tweaks to call >> MaintainOldSnapshotTimeMapping() from elsewhere or only when >> something changes (like crossing a minute boundary or seeing that a >> new TransactionId has been assigned). If you

Re: [HACKERS] what to revert

Hi, On 2016-05-04 16:22:41 -0500, Kevin Grittner wrote: > On Wed, May 4, 2016 at 3:42 PM, Andres Freund wrote: > > > My concern isn't performing checks at snapshot build time and at buffer > > access time. That seems fairly reasonable. My concern is that the > > time->xid mapping used to determ

Re: [HACKERS] what to revert

On Wed, May 4, 2016 at 1:42 PM, Andres Freund wrote: >> Surely nobody here is blind to the fact that holding back xmin often >> creates a lot of bloat for no reason - many or all of the retained old >> row versions may never be accessed. > > Definitely. I *like* the feature. +1. >> I do not thin

Re: [HACKERS] what to revert

On Wed, May 4, 2016 at 3:42 PM, Andres Freund wrote: > My concern isn't performing checks at snapshot build time and at buffer > access time. That seems fairly reasonable. My concern is that the > time->xid mapping used to determine the xid horizon is built at snapshot > access time. The relevan

Re: [HACKERS] what to revert

Hi, On 2016-05-04 14:25:14 -0400, Robert Haas wrote: > On Wed, May 4, 2016 at 12:42 PM, Andres Freund wrote: > > On 2016-05-04 13:35:02 -0300, Alvaro Herrera wrote: > >> Honestly, I don't see any strong ground in which to base a revert threat > >> for this feature. > > > > It's datastructures are

Re: [HACKERS] what to revert

On Wed, May 4, 2016 at 1:25 PM, Robert Haas wrote: > If somebody had even hinted that such a problem might exist, Kevin > probably would have fixed it before commit, but nobody did. As soon > as it was raised, Kevin started working on it. That's about all you > can ask, I think. Right; I have

Re: [HACKERS] what to revert

On Wed, May 4, 2016 at 12:42 PM, Andres Freund wrote: > On 2016-05-04 13:35:02 -0300, Alvaro Herrera wrote: >> Honestly, I don't see any strong ground in which to base a revert threat >> for this feature. > > It's datastructures are badly designed. But releasing it there's no > pressure to fix tha

Re: [HACKERS] what to revert

Hi, On 2016-05-04 13:35:02 -0300, Alvaro Herrera wrote: > Honestly, I don't see any strong ground in which to base a revert threat > for this feature. It's datastructures are badly designed. But releasing it there's no pressure to fix that. If this were an intrinsic cost - ok. But it's not. >

Re: [HACKERS] what to revert

Andres Freund wrote: > On 2016-05-04 00:01:20 +0300, Ants Aasma wrote: > > On Tue, May 3, 2016 at 9:57 PM, Tomas Vondra > > wrote: > > > If you tell me how to best test it, I do have a 4-socket server sitting > > > idly > > > in the corner (well, a corner reachable by SSH). I can get us some > >

Re: [HACKERS] what to revert

Craig Ringer wrote: > On 4 May 2016 at 13:03, Euler Taveira wrote: > > > Question is: is the actual code so useless that it can't even be a 1.0 > > release? > > What's committed suffers from a design problem and cannot work correctly, > nor can it be fixed without an API change and significant r

Re: [HACKERS] what to revert

On 4 May 2016 at 13:03, Euler Taveira wrote: > Question is: is the actual code so useless that it can't even be a 1.0 > release? What's committed suffers from a design problem and cannot work correctly, nor can it be fixed without an API change and significant revision. The revised version I p

Re: [HACKERS] what to revert

On 03-05-2016 20:25, Craig Ringer wrote: > On 4 May 2016 at 01:12, Peter Geoghegan > wrote: > > On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera > mailto:alvhe...@2ndquadrant.com>> wrote: > > As its committer, I tend to agree about reverting that feature. Craig

Re: [HACKERS] what to revert

Amit Kapila writes: > Yes, that would be a way forward for 9.6 if we are not able to close > blocking open items before beta1. However, I think it would be bad if we > miss some of the below listed important features like snapshot_too_old or > atomic pin/unpin for 9.6. Can we consider to postpon

Re: [HACKERS] what to revert

On Tue, May 3, 2016 at 9:28 PM, Robert Haas wrote: > > On Tue, May 3, 2016 at 11:36 AM, Tom Lane wrote: > >> There are a lot more than 2 patchsets that are busted at the moment, > >> unfortunately, but I assume you are referring to "snapshot too old" > >> and "Use Foreign Key relationships to inf

Re: [HACKERS] what to revert

On 4 May 2016 at 01:12, Peter Geoghegan wrote: > On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera > wrote: > > As its committer, I tend to agree about reverting that feature. Craig > > was just posting some more patches, and I have the pg_recvlogical > > changes here (--endpos) which after some t

Re: [HACKERS] what to revert

On 2016-05-03 20:57:13 +0200, Tomas Vondra wrote: > On 05/03/2016 07:41 PM, Andres Freund wrote: > >On 2016-05-03 11:46:23 -0500, Kevin Grittner wrote: > >>>was immediately addressed by another round of benchmarks after you > >>>pointed it out. > >> > >>Which showed a 4% maximum hit before moving t

Re: [HACKERS] what to revert

On 2016-05-04 00:01:20 +0300, Ants Aasma wrote: > On Tue, May 3, 2016 at 9:57 PM, Tomas Vondra > wrote: > > If you tell me how to best test it, I do have a 4-socket server sitting idly > > in the corner (well, a corner reachable by SSH). I can get us some numbers, > > but I haven't been following

Re: [HACKERS] what to revert

On Tue, May 3, 2016 at 9:57 PM, Tomas Vondra wrote: > If you tell me how to best test it, I do have a 4-socket server sitting idly > in the corner (well, a corner reachable by SSH). I can get us some numbers, > but I haven't been following the snapshot_too_old so I'll need some guidance > on what

Re: [HACKERS] what to revert

On 05/03/2016 07:41 PM, Andres Freund wrote: On 2016-05-03 11:46:23 -0500, Kevin Grittner wrote: was immediately addressed by another round of benchmarks after you pointed it out. Which showed a 4% maximum hit before moving the test for whether it was "off" inline. (I'm not clear from the p

Re: [HACKERS] what to revert

On 05/03/2016 07:12 PM, Peter Geoghegan wrote: On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera wrote: As its committer, I tend to agree about reverting that feature. Craig was just posting some more patches, and I have the pg_recvlogical changes here (--endpos) which after some testing are not

Re: [HACKERS] what to revert

On 2016-05-03 10:12:51 -0700, Peter Geoghegan wrote: > On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera > wrote: > > As its committer, I tend to agree about reverting that feature. Craig > > was just posting some more patches, and I have the pg_recvlogical > > changes here (--endpos) which after s

Re: [HACKERS] what to revert

On 2016-05-03 11:46:23 -0500, Kevin Grittner wrote: > > was immediately addressed by another round of benchmarks after you > > pointed it out. > > Which showed a 4% maximum hit before moving the test for whether it > was "off" inline. > (I'm not clear from the posted results whether that was befo

Re: [HACKERS] what to revert

* Robert Haas (robertmh...@gmail.com) wrote: > Here's a list of what I think is currently broken in 9.6 that we might > conceivably fix by reverting patches: [...] > - Predefined Roles. Neither you nor I liked Stephen's design. It > slowed down pg_dump. It also broke pg_dump for non-superusers a

Re: [HACKERS] what to revert

On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera wrote: > As its committer, I tend to agree about reverting that feature. Craig > was just posting some more patches, and I have the pg_recvlogical > changes here (--endpos) which after some testing are not quite looking > ready to go -- plus we still

Re: [HACKERS] what to revert

Andres Freund wrote: > I'm *really* doubtful about the logical timeline following patches. I > think they're, as committed, over-complicated and pretty much unusable. As its committer, I tend to agree about reverting that feature. Craig was just posting some more patches, and I have the pg_recvl

Re: [HACKERS] what to revert

On Tue, May 3, 2016 at 11:22 AM, Andres Freund wrote: > The issue with 0 v. -1 (and 0 vs. > 0 makes a big performance > difference, so it's not that surprising to interpret numbers that way) ... if you fail to read the documentation of the feature or the code implementing it before testing. > w

Re: [HACKERS] what to revert

On Tue, May 3, 2016 at 12:22 PM, Andres Freund wrote: >> > but that might be fixed now. >> >> Certainly all evidence suggests that, FUD to the contrary. > > So it's now FUD to report issues with a patch that obviously hasn't > received sufficient benchmarking? Give me break. Yeah, I don't think t

Re: [HACKERS] what to revert

On 2016-05-03 11:12:03 -0500, Kevin Grittner wrote: > On Tue, May 3, 2016 at 10:58 AM, Robert Haas wrote: > > > - Snapshot Too Old. Tom, Andres, and Bruce want this reverted. > > It regresses performance significantly when turned on. > > When turned on, it improves performance in some cases and

Re: [HACKERS] what to revert

On 2016-05-03 11:58:34 -0400, Robert Haas wrote: > - Atomic Pin/Unpin. There are two different open items related to > this, both apparently relating to testing done by Jeff Janes. I am > not sure whether they are really independent reports of different > problems or just two reports of the same

Re: [HACKERS] what to revert

On Tue, May 3, 2016 at 10:58 AM, Robert Haas wrote: > - Snapshot Too Old. Tom, Andres, and Bruce want this reverted. > It regresses performance significantly when turned on. When turned on, it improves performance in some cases and regresses performance in others. Don't forget it is currently