Re: [HACKERS] pg_dump broken for non-super user

2016-05-03 Thread Rushabh Lathia
On Tue, May 3, 2016 at 8:34 PM, Stephen Frost wrote: > * Rushabh Lathia (rushabh.lat...@gmail.com) wrote: > > With commit a9f0e8e5a2e779a888988cb64479a6723f668c84, now pg_dump, use a > > bitmap > > to represent what to include. With this commit if non-super user is > unable > > to perform the dum

Re: [HACKERS] Make PG's "NOT NULL"s and attnotnull ("is_nullable") conform to SQL-2011

2016-05-03 Thread Vitaly Burovoy
On 5/3/16, Tom Lane wrote: > Vitaly Burovoy writes: >> On 4/27/16, Alvaro Herrera wrote: >>> Point 2 is where things differ from what I remember; my (possibly >>> flawed) understanding was that there's no difference between those >>> things. Many (maybe all) of the things from this point on are

Re: [HACKERS] Make PG's "NOT NULL"s and attnotnull ("is_nullable") conform to SQL-2011

2016-05-03 Thread David G. Johnston
On Monday, February 8, 2016, Vitaly Burovoy wrote: > > 12. At the same time in (subcl. 4.13) mentioned there can be "at least > one NNC" (may be via inheritance?). > > This is a bit hard to reason about given that our implementation of inheritance is non-standard. Are we close to the standard se

Re: [HACKERS] Apparent race condition in standby promotion

2016-05-03 Thread Noah Misch
On Mon, Apr 25, 2016 at 02:09:27PM -0400, Tom Lane wrote: > I'm looking at buildfarm member tern's recent failure: > http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2016-04-25%2001%3A08%3A08 > # Running: pg_ctl -D > /home/nm/farm/gcc32/HEAD/pgsql.build/src/bin/pg_rewind/tmp_check/d

Re: [HACKERS] text search changes vs. binary upgrade

2016-05-03 Thread Noah Misch
On Tue, May 03, 2016 at 11:13:54PM -0400, Tom Lane wrote: > Noah Misch writes: > > Commit bb14050 said: > > - change order for tsquery, so, users, who has a btree index over > > tsquery, > > should reindex it > > We undid that in 1ec4c7c05, no? Ah, looks that way. > > Commit 61d66c4

Re: [HACKERS] Make PG's "NOT NULL"s and attnotnull ("is_nullable") conform to SQL-2011

2016-05-03 Thread David G. Johnston
Quick flyby here... On Tuesday, May 3, 2016, Tom Lane wrote: > Vitaly Burovoy > writes: > > On 4/27/16, Alvaro Herrera > > wrote: > >> Point 2 is where things differ from what I remember; my (possibly > >> flawed) understanding was that there's no difference between those > >> things. Many (may

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-05-03 Thread Fabien COELHO
Hello Andres, An enum doesn't have a benefit for a bitmask imo - you can't "legally" use it as a type for functions accepting the bitmask. I do not understand. I suggested to use enum to enumerate the bitmask constants, ISTM that it does not preclude to use it as a bitmask as you do, it is ju

Re: [HACKERS] Make PG's "NOT NULL"s and attnotnull ("is_nullable") conform to SQL-2011

2016-05-03 Thread Tom Lane
Vitaly Burovoy writes: > On 4/27/16, Alvaro Herrera wrote: >> Point 2 is where things differ from what I remember; my (possibly >> flawed) understanding was that there's no difference between those >> things. Many (maybe all) of the things from this point on are probably >> fallout from that one

Re: [HACKERS] what to revert

2016-05-03 Thread Euler Taveira
On 03-05-2016 20:25, Craig Ringer wrote: > On 4 May 2016 at 01:12, Peter Geoghegan > wrote: > > On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera > mailto:alvhe...@2ndquadrant.com>> wrote: > > As its committer, I tend to agree about reverting that feature. Craig

Re: [HACKERS] what to revert

2016-05-03 Thread Tom Lane
Amit Kapila writes: > Yes, that would be a way forward for 9.6 if we are not able to close > blocking open items before beta1. However, I think it would be bad if we > miss some of the below listed important features like snapshot_too_old or > atomic pin/unpin for 9.6. Can we consider to postpon

Re: [HACKERS] Make PG's "NOT NULL"s and attnotnull ("is_nullable") conform to SQL-2011

2016-05-03 Thread Vitaly Burovoy
I'm sorry for the late answer. On 4/27/16, Alvaro Herrera wrote: > Vitaly Burovoy wrote: > > Hi, > >> But before starting working on it I had a look at the SQL-2011 >> standard (ISO/IEC 9075-2)[3] and found that: >> >> 1. A name for a "NOT NULL" constraint can be given by a table >> definition (

Re: [HACKERS] what to revert

2016-05-03 Thread Amit Kapila
On Tue, May 3, 2016 at 9:28 PM, Robert Haas wrote: > > On Tue, May 3, 2016 at 11:36 AM, Tom Lane wrote: > >> There are a lot more than 2 patchsets that are busted at the moment, > >> unfortunately, but I assume you are referring to "snapshot too old" > >> and "Use Foreign Key relationships to inf

Re: [HACKERS] old_snapshot_threshold's interaction with hash index

2016-05-03 Thread Amit Kapila
On Tue, May 3, 2016 at 9:47 PM, Kevin Grittner wrote: > > On Tue, May 3, 2016 at 11:09 AM, Robert Haas wrote: > > On Tue, May 3, 2016 at 11:46 AM, Kevin Grittner wrote: > >>> Uh, I have no idea how this would be fixed if the PageLSN is zero. Do > >>> you? > >> > >> Yes, I see three ways, the mo

Re: [HACKERS] Rename max_parallel_degree?

2016-05-03 Thread David Rowley
On 4 May 2016 at 15:12, Amit Kapila wrote: > On Mon, May 2, 2016 at 11:47 PM, Robert Haas wrote: >> >> On Tue, Apr 26, 2016 at 11:49 AM, Robert Haas >> wrote: >> > On Tue, Apr 26, 2016 at 11:44 AM, Tom Lane wrote: >> >> Robert Haas writes: >> >> To summarize the positions as I understand them:

Re: [HACKERS] text search changes vs. binary upgrade

2016-05-03 Thread Tom Lane
Noah Misch writes: > Commit bb14050 said: > - change order for tsquery, so, users, who has a btree index over tsquery, > should reindex it We undid that in 1ec4c7c05, no? (Even if we didn't, the usefulness of a btree index on tsquery seems negligibly small.) > Commit 61d66c4 may or ma

Re: [HACKERS] Rename max_parallel_degree?

2016-05-03 Thread Amit Kapila
On Mon, May 2, 2016 at 11:47 PM, Robert Haas wrote: > > On Tue, Apr 26, 2016 at 11:49 AM, Robert Haas wrote: > > On Tue, Apr 26, 2016 at 11:44 AM, Tom Lane wrote: > >> Robert Haas writes: > > To summarize the positions as I understand them: > > Magnus seems OK with the way things are. > Peter w

[HACKERS] modifying WaitEventSets (was: Performance degradation in commit ac1d794)

2016-05-03 Thread Robert Haas
On Sun, Mar 20, 2016 at 8:31 AM, Thomas Munro wrote: > One thing that I want to do but can't with this interface is remove an > fd from the set. I can AddWaitEventToSet returning a position, and I > can ModifyWait to provide new event mask by position including zero > mask, I can't actually remov

[HACKERS] text search changes vs. binary upgrade

2016-05-03 Thread Noah Misch
Commit bb14050 said: - change order for tsquery, so, users, who has a btree index over tsquery, should reindex it The last change of this sort also modified pg_upgrade to issue REINDEX guidance. See old_8_3_invalidate_hash_gin_indexes() in the PostgreSQL 9.4 source. PostgreSQL 9.6 pg_

Re: [HACKERS] pg_xlog -> pg_xjournal?

2016-05-03 Thread Joshua D. Drake
On 05/03/2016 06:38 PM, Michael Paquier wrote: On Mon, May 2, 2016 at 10:29 PM, Robert Haas wrote: On Thu, Apr 28, 2016 at 11:46 PM, Craig Ringer wrote: I just helped another person yesterday who deleted their pg_xlog. The biggest reason I've seen pg_xlog get deleted is not because it's cal

Re: [HACKERS] pg_xlog -> pg_xjournal?

2016-05-03 Thread Michael Paquier
On Mon, May 2, 2016 at 10:29 PM, Robert Haas wrote: > On Thu, Apr 28, 2016 at 11:46 PM, Craig Ringer wrote: >> I just helped another person yesterday who deleted their pg_xlog. > > The biggest reason I've seen pg_xlog get deleted is not because it's > called pg_xlog, but because it's located some

Re: [HACKERS] psql :: support for \ev viewname and \sv viewname

2016-05-03 Thread Peter Eisentraut
On 5/3/16 3:10 PM, Dean Rasheed wrote: On 3 May 2016 at 16:52, Peter Eisentraut wrote: I would change appendReloptionsArrayAH() to a function and keep AH as the first argument (similar to other functions that take such a handle). I can understand changing it to a function, but I don't think

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-03 Thread David Rowley
On 4 May 2016 at 09:18, David Rowley wrote: > On 4 May 2016 at 02:10, Tomas Vondra wrote: >> There are probably a few reasonably simple things we could do - e.g. ignore >> foreign keys with just a single column, as the primary goal of the patch is >> improving estimates with multi-column foreign

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-05-03 Thread Andres Freund
Hi, On 2016-05-03 07:24:46 +0200, Fabien COELHO wrote: > >>I'm unsure about switching enum to #define, could be an enum still with > >>explicit values set, something like: > > > >An enum doesn't have a benefit for a bitmask imo - you can't "legally" > >use it as a type for functions accepting the

Re: [HACKERS] what to revert

2016-05-03 Thread Craig Ringer
On 4 May 2016 at 01:12, Peter Geoghegan wrote: > On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera > wrote: > > As its committer, I tend to agree about reverting that feature. Craig > > was just posting some more patches, and I have the pg_recvlogical > > changes here (--endpos) which after some t

Re: [HACKERS] atomic pin/unpin causing errors

2016-05-03 Thread Andres Freund
Hi Jeff, On 2016-04-29 10:38:55 -0700, Jeff Janes wrote: > I've bisected the errors I was seeing, discussed in > http://www.postgresql.org/message-id/CAMkU=1xqehc0ok4d+tkjfq1nvuho37pyrkhjp6q8oxifmx7...@mail.gmail.com > > It look like they first appear in: > > commit 48354581a49c30f5757c203415aa8

Re: [HACKERS] what to revert

2016-05-03 Thread Andres Freund
On 2016-05-03 20:57:13 +0200, Tomas Vondra wrote: > On 05/03/2016 07:41 PM, Andres Freund wrote: > >On 2016-05-03 11:46:23 -0500, Kevin Grittner wrote: > >>>was immediately addressed by another round of benchmarks after you > >>>pointed it out. > >> > >>Which showed a 4% maximum hit before moving t

Re: [HACKERS] what to revert

2016-05-03 Thread Andres Freund
On 2016-05-04 00:01:20 +0300, Ants Aasma wrote: > On Tue, May 3, 2016 at 9:57 PM, Tomas Vondra > wrote: > > If you tell me how to best test it, I do have a 4-socket server sitting idly > > in the corner (well, a corner reachable by SSH). I can get us some numbers, > > but I haven't been following

Re: [HACKERS] Timeline following for logical slots

2016-05-03 Thread Alvaro Herrera
I don't like reverting patches, but this patch is making me more and more uncomfortable. We have two open items, one of which requires writing new test code that doesn't exist yet; and we have the pg_recvlogical changes that were approved post-feature freeze, but that I now have second thoughts ab

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-03 Thread David Rowley
On 4 May 2016 at 02:10, Tomas Vondra wrote: > There are probably a few reasonably simple things we could do - e.g. ignore > foreign keys with just a single column, as the primary goal of the patch is > improving estimates with multi-column foreign keys. I believe that covers a > vast majority of f

Re: [HACKERS] what to revert

2016-05-03 Thread Ants Aasma
On Tue, May 3, 2016 at 9:57 PM, Tomas Vondra wrote: > If you tell me how to best test it, I do have a 4-socket server sitting idly > in the corner (well, a corner reachable by SSH). I can get us some numbers, > but I haven't been following the snapshot_too_old so I'll need some guidance > on what

Re: [HACKERS] Pg_stop_backup process does not run - Backup Intervals

2016-05-03 Thread David G. Johnston
On Mon, May 2, 2016 at 12:03 PM, Rodrigo Cavalcante wrote: > Hi, > > On alternate days my backup is failing, by the pg_stop_backup process () > does not perform or quit. > > Version PostgreSQL: 9.1.6 > ​Reporting unusual behavior while running a years-old point release is unlikely to be producti

Re: [HACKERS] Pg_stop_backup process does not run - Backup Intervals

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 4:21 PM, Rodrigo Cavalcante wrote: > The my script works, but after a few days it stops working because the > process does not end pg_stop_backup. Well, that shouldn't happen, but without any logs or debugging information, it's hard to guess why. > The pg_basebackup alread

Re: [HACKERS] Pg_stop_backup process does not run - Backup Intervals

2016-05-03 Thread Rodrigo Cavalcante
Hello Robert, Thanks for the feedback. The my script works, but after a few days it stops working because the process does not end pg_stop_backup. The pg_basebackup already does this substitution? -- View this message in context: http://postgresql.nabble.com/Pg-stop-backup-process-does-not-r

Re: [HACKERS] pg_upgrade and toasted pg_largeobject

2016-05-03 Thread Tom Lane
Alvaro Herrera writes: > Robert Haas wrote: >> On Tue, May 3, 2016 at 2:09 PM, Alvaro Herrera >> wrote: >>> How about backpatching patch 1 all the way back, and putting the others >>> in 9.6? >> Why would we do that? It seems very odd to back-patch a pure >> refactoring - isn't that taking a r

Re: [HACKERS] pg_upgrade and toasted pg_largeobject

2016-05-03 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, May 3, 2016 at 2:09 PM, Alvaro Herrera > wrote: > > Tom Lane wrote: > >> Any thoughts what to do with this? We could decide that it's a bug fix > >> and backpatch, or decide that it's a new feature and delay till 9.7, > >> or decide that it's a minor bug fix and add

Re: [HACKERS] psql :: support for \ev viewname and \sv viewname

2016-05-03 Thread Dean Rasheed
On 3 May 2016 at 16:52, Peter Eisentraut wrote: > I would change appendReloptionsArrayAH() to a function and keep AH as the > first argument (similar to other functions that take such a handle). I can understand changing it to a function, but I don't think AH should be the first argument. All oth

Re: [HACKERS] what to revert

2016-05-03 Thread Tomas Vondra
On 05/03/2016 07:41 PM, Andres Freund wrote: On 2016-05-03 11:46:23 -0500, Kevin Grittner wrote: was immediately addressed by another round of benchmarks after you pointed it out. Which showed a 4% maximum hit before moving the test for whether it was "off" inline. (I'm not clear from the p

Re: [HACKERS] what to revert

2016-05-03 Thread Tomas Vondra
On 05/03/2016 07:12 PM, Peter Geoghegan wrote: On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera wrote: As its committer, I tend to agree about reverting that feature. Craig was just posting some more patches, and I have the pg_recvlogical changes here (--endpos) which after some testing are not

Re: [HACKERS] pg_upgrade and toasted pg_largeobject

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 2:09 PM, Alvaro Herrera wrote: > Tom Lane wrote: >> Any thoughts what to do with this? We could decide that it's a bug fix >> and backpatch, or decide that it's a new feature and delay till 9.7, >> or decide that it's a minor bug fix and add it to 9.6 only. I kinda lean >>

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andrew Dunstan
On 05/03/2016 01:58 PM, Alvaro Herrera wrote: Andrew Dunstan wrote: And if this is of any use, here are the dump differences from every live version to git tip, as of this morning. Interesting, thanks. I wonder if some of these diffs could be reduced further by using pg_dump -Fd instead of

Re: [HACKERS] Accidentally parallel unsafe functions

2016-05-03 Thread Robert Haas
On Fri, Apr 29, 2016 at 6:06 PM, Andreas Karlsson wrote: > I am currently looking into adding the correct parallel options to all > functions in the extensions and I noticed that some built-in functions seems > to have been marked as unsafe by accident. The main culprit is > system_views.sql which

Re: [HACKERS] pg_upgrade and toasted pg_largeobject

2016-05-03 Thread Alvaro Herrera
Tom Lane wrote: > Any thoughts what to do with this? We could decide that it's a bug fix > and backpatch, or decide that it's a new feature and delay till 9.7, > or decide that it's a minor bug fix and add it to 9.6 only. I kinda lean > towards the last alternative. How about backpatching patch

Re: [HACKERS] Timeline following for logical slots

2016-05-03 Thread Alvaro Herrera
Craig Ringer wrote: > With this patch pg_recvlogical takes a new --endpos LSN argument, and will > exit if either: > > * it receives an XLogData message with dataStart >= endpos; or > * it receives a keepalive with walEnd >= endpos > > The latter allows it to work without needing a dummy transa

Re: [HACKERS] pg_upgrade and toasted pg_largeobject

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 1:38 PM, Tom Lane wrote: > Any thoughts what to do with this? We could decide that it's a bug fix > and backpatch, or decide that it's a new feature and delay till 9.7, > or decide that it's a minor bug fix and add it to 9.6 only. I kinda lean > towards the last alternativ

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Tom Lane
Andres Freund writes: > On 2016-05-03 12:07:51 -0400, Tom Lane wrote: >> I think possibly the easiest fix for this is to have pg_upgrade, >> instead of RESETting a nonexistent option, RESET something that's >> still considered to require AccessExclusiveLock. "user_catalog_table" >> would work, lo

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andres Freund
On 2016-05-03 13:47:14 -0400, Tom Lane wrote: > I've been thinking of proposing that it's time (not now, at this point, > but for 9.7) to rip out libpq's support for V2 protocol as well as > pg_dump's support for pre-7.4 backends. +1 > There might be an argument for moving pg_dump's cutoff furth

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Alvaro Herrera
Andrew Dunstan wrote: > And if this is of any use, here are the dump differences from every live > version to git tip, as of this morning. Interesting, thanks. I wonder if some of these diffs could be reduced further by using pg_dump -Fd instead of a single text dump -- then internal ordering wo

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Stephen Frost
* Andres Freund (and...@anarazel.de) wrote: > On 2016-05-03 12:07:51 -0400, Tom Lane wrote: > > I think possibly the easiest fix for this is to have pg_upgrade, > > instead of RESETting a nonexistent option, RESET something that's > > still considered to require AccessExclusiveLock. "user_catalog_

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Tom Lane
Stephen Frost writes: > One other point is that pg_dump goes quite a bit farther back than just > what we currently support (or at least, it tries to). I think that, > generally, that's a good thing, but it does mean we have a lot of cases > that don't get tested nearly as much... Yeah. I do pe

Re: [HACKERS] what to revert

2016-05-03 Thread Andres Freund
On 2016-05-03 10:12:51 -0700, Peter Geoghegan wrote: > On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera > wrote: > > As its committer, I tend to agree about reverting that feature. Craig > > was just posting some more patches, and I have the pg_recvlogical > > changes here (--endpos) which after s

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andres Freund
Hi, On 2016-05-03 12:07:51 -0400, Tom Lane wrote: > I think possibly the easiest fix for this is to have pg_upgrade, > instead of RESETting a nonexistent option, RESET something that's > still considered to require AccessExclusiveLock. "user_catalog_table" > would work, looks like; though I'd wan

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Alvaro Herrera
Stephen Frost wrote: > One other point is that pg_dump goes quite a bit farther back than just > what we currently support (or at least, it tries to). I think that, > generally, that's a good thing, but it does mean we have a lot of cases > that don't get tested nearly as much... > > I was able

Re: [HACKERS] what to revert

2016-05-03 Thread Andres Freund
On 2016-05-03 11:46:23 -0500, Kevin Grittner wrote: > > was immediately addressed by another round of benchmarks after you > > pointed it out. > > Which showed a 4% maximum hit before moving the test for whether it > was "off" inline. > (I'm not clear from the posted results whether that was befo

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andrew Dunstan
On 05/03/2016 01:33 PM, Andrew Dunstan wrote: On 05/03/2016 01:28 PM, Andrew Dunstan wrote: On 05/03/2016 01:21 PM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Tom Lane wrote: More generally, though, I wonder how we can have some test coverage on such cases

Re: [HACKERS] pg_upgrade and toasted pg_largeobject

2016-05-03 Thread Tom Lane
I wrote: > Alvaro Herrera writes: >> I'm happy with the solution that pg_upgrade has a step in the check >> stage that says "catalog XYZ has a toast table but shouldn't, aborting >> the upgrade". (Well, not _happy_, but at least it's a lot easier to >> diagnose). > I think though that you're def

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andrew Dunstan
On 05/03/2016 01:28 PM, Andrew Dunstan wrote: On 05/03/2016 01:21 PM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Tom Lane wrote: More generally, though, I wonder how we can have some test coverage on such cases going forward. Is the patch below too ugly to co

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > > Tom Lane wrote: > > > > > > > More generally, though, I wonder how we can have some test coverage > > > > on such cases going forward. Is the patch below too ugly

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Andrew Dunstan
On 05/03/2016 01:21 PM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Tom Lane wrote: More generally, though, I wonder how we can have some test coverage on such cases going forward. Is the patch below too ugly to commit permanently, and if so, what other idea can

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Tom Lane wrote: > > > More generally, though, I wonder how we can have some test coverage > > on such cases going forward. Is the patch below too ugly to commit > > permanently, and if so, what other idea can you suggest? > > I suggest a build

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Alvaro Herrera
Stephen Frost wrote: > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > Tom Lane wrote: > > > > > More generally, though, I wonder how we can have some test coverage > > > on such cases going forward. Is the patch below too ugly to commit > > > permanently, and if so, what other idea can yo

Re: [HACKERS] what to revert

2016-05-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Here's a list of what I think is currently broken in 9.6 that we might > conceivably fix by reverting patches: [...] > - Predefined Roles. Neither you nor I liked Stephen's design. It > slowed down pg_dump. It also broke pg_dump for non-superusers a

Re: [HACKERS] what to revert

2016-05-03 Thread Peter Geoghegan
On Tue, May 3, 2016 at 9:58 AM, Alvaro Herrera wrote: > As its committer, I tend to agree about reverting that feature. Craig > was just posting some more patches, and I have the pg_recvlogical > changes here (--endpos) which after some testing are not quite looking > ready to go -- plus we still

Re: [HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Alvaro Herrera
Tom Lane wrote: > More generally, though, I wonder how we can have some test coverage > on such cases going forward. Is the patch below too ugly to commit > permanently, and if so, what other idea can you suggest? I suggest a buildfarm animal running a custom buildfarm module that exercises the

Re: [HACKERS] what to revert

2016-05-03 Thread Alvaro Herrera
Andres Freund wrote: > I'm *really* doubtful about the logical timeline following patches. I > think they're, as committed, over-complicated and pretty much unusable. As its committer, I tend to agree about reverting that feature. Craig was just posting some more patches, and I have the pg_recvl

Re: [HACKERS] old_snapshot_threshold's interaction with hash index

2016-05-03 Thread Kevin Grittner
On Tue, May 3, 2016 at 11:48 AM, Robert Haas wrote: > OK, I see now: the basic idea here is that we can't prune based on the > newer XID unless the page LSN is guaranteed to advance whenever data > is removed. Currently, we attempt to limit bloat in non-unlogged, > non-catalog tables. You're sa

Re: [HACKERS] old_snapshot_threshold's interaction with hash index

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 12:17 PM, Kevin Grittner wrote: > On Tue, May 3, 2016 at 11:09 AM, Robert Haas wrote: >> On Tue, May 3, 2016 at 11:46 AM, Kevin Grittner wrote: Uh, I have no idea how this would be fixed if the PageLSN is zero. Do you? >>> >>> Yes, I see three ways, the most obv

Re: [HACKERS] what to revert

2016-05-03 Thread Kevin Grittner
On Tue, May 3, 2016 at 11:22 AM, Andres Freund wrote: > The issue with 0 v. -1 (and 0 vs. > 0 makes a big performance > difference, so it's not that surprising to interpret numbers that way) ... if you fail to read the documentation of the feature or the code implementing it before testing. > w

Re: [HACKERS] what to revert

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 12:22 PM, Andres Freund wrote: >> > but that might be fixed now. >> >> Certainly all evidence suggests that, FUD to the contrary. > > So it's now FUD to report issues with a patch that obviously hasn't > received sufficient benchmarking? Give me break. Yeah, I don't think t

Re: [HACKERS] what to revert

2016-05-03 Thread Andres Freund
On 2016-05-03 11:12:03 -0500, Kevin Grittner wrote: > On Tue, May 3, 2016 at 10:58 AM, Robert Haas wrote: > > > - Snapshot Too Old. Tom, Andres, and Bruce want this reverted. > > It regresses performance significantly when turned on. > > When turned on, it improves performance in some cases and

Re: [HACKERS] old_snapshot_threshold's interaction with hash index

2016-05-03 Thread Kevin Grittner
On Tue, May 3, 2016 at 11:09 AM, Robert Haas wrote: > On Tue, May 3, 2016 at 11:46 AM, Kevin Grittner wrote: >>> Uh, I have no idea how this would be fixed if the PageLSN is zero. Do >>> you? >> >> Yes, I see three ways, the most obvious of which is what Amit >> suggested -- don't do early vacuu

Re: [HACKERS] Processes and caches in postgresql

2016-05-03 Thread Petr Jelinek
On 03/05/16 16:35, Craig Ringer wrote: On 3 May 2016 at 21:37, Merlin Moncure mailto:mmonc...@gmail.com>> wrote: There is library out there, unfortunately GPL licensed, that attempts to fully implement posix including fork(): http://midipix.org/. One of these days I'd like to have

Re: [HACKERS] what to revert

2016-05-03 Thread Andres Freund
On 2016-05-03 11:58:34 -0400, Robert Haas wrote: > - Atomic Pin/Unpin. There are two different open items related to > this, both apparently relating to testing done by Jeff Janes. I am > not sure whether they are really independent reports of different > problems or just two reports of the same

Re: [HACKERS] what to revert

2016-05-03 Thread Kevin Grittner
On Tue, May 3, 2016 at 10:58 AM, Robert Haas wrote: > - Snapshot Too Old. Tom, Andres, and Bruce want this reverted. > It regresses performance significantly when turned on. When turned on, it improves performance in some cases and regresses performance in others. Don't forget it is currently

Re: [HACKERS] old_snapshot_threshold's interaction with hash index

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 11:46 AM, Kevin Grittner wrote: >> Uh, I have no idea how this would be fixed if the PageLSN is zero. Do >> you? > > Yes, I see three ways, the most obvious of which is what Amit > suggested -- don't do early vacuum on a table which has a hash index. What do you mean by "e

[HACKERS] ALTER TABLE lock downgrades have broken pg_upgrade

2016-05-03 Thread Tom Lane
There is logic in pg_upgrade plus the backend, mostly added by commit 4c6780fd1, to cope with the corner cases that sometimes arise where the old and new versions have different ideas about whether a given table needs a TOAST table. The more common case is where there's a TOAST table in the old DB

[HACKERS] what to revert

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 11:36 AM, Tom Lane wrote: >> There are a lot more than 2 patchsets that are busted at the moment, >> unfortunately, but I assume you are referring to "snapshot too old" >> and "Use Foreign Key relationships to infer multi-column join >> selectivity". > > Yeah, those are the

Re: [HACKERS] psql :: support for \ev viewname and \sv viewname

2016-05-03 Thread Peter Eisentraut
On 5/2/16 8:53 AM, Dean Rasheed wrote: Here are updated patches doing that --- the first moves fmtReloptionsArray() from pg_dump.c to fe_utils/string_utils.c so that it can be shared by pg_dump and psql, and renames it to appendReloptionsArray(). The second patch fixes the actual psql bug. This

Re: [HACKERS] old_snapshot_threshold's interaction with hash index

2016-05-03 Thread Kevin Grittner
On Tue, May 3, 2016 at 10:45 AM, Bruce Momjian wrote: > On Mon, May 2, 2016 at 04:02:35PM -0500, Kevin Grittner wrote: >> On Sun, May 1, 2016 at 1:43 AM, Amit Kapila wrote: >> > On Sun, May 1, 2016 at 12:05 PM, Amit Kapila >> > wrote: >> >> >> >> Currently we do the test for old snapshot (Test

Re: [HACKERS] old_snapshot_threshold's interaction with hash index

2016-05-03 Thread Bruce Momjian
On Mon, May 2, 2016 at 04:02:35PM -0500, Kevin Grittner wrote: > On Sun, May 1, 2016 at 1:43 AM, Amit Kapila wrote: > > On Sun, May 1, 2016 at 12:05 PM, Amit Kapila > > wrote: > >> > >> Currently we do the test for old snapshot (TestForOldSnapshot) for hash > >> indexes while scanning them. Do

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-05-03 Thread Andres Freund
On 2016-05-02 10:32:28 -0400, Robert Haas wrote: > You are certainly welcome to add a new open item to cover those > complaints. Done that. > But I do not want to blur together the discussion of > whether the feature is well-designed with the question of whether it > regresses performance when it

Re: [HACKERS] pgindent fixups

2016-05-03 Thread Tom Lane
Robert Haas writes: > On Tue, May 3, 2016 at 11:23 AM, Tom Lane wrote: >> Well, there are at least two patchsets we're actively discussing >> reverting, so I think this should wait till those decisions are resolved. > OK, but that may well mean we don't get this done before beta1, which > I thin

Re: [HACKERS] pgindent fixups

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 11:23 AM, Tom Lane wrote: > Robert Haas writes: >> OK, I committed this. Barring objections, I'll go ahead and pgindent >> the whole tree tomorrow. If we're going to revert anything big then >> we might want to hold off, but otherwise I think its better to get >> this don

Re: [HACKERS] pgindent fixups

2016-05-03 Thread Tom Lane
Robert Haas writes: > OK, I committed this. Barring objections, I'll go ahead and pgindent > the whole tree tomorrow. If we're going to revert anything big then > we might want to hold off, but otherwise I think its better to get > this done sooner rather than later. Well, there are at least tw

Re: [HACKERS] Pg_stop_backup process does not run - Backup Intervals

2016-05-03 Thread Robert Haas
On Mon, May 2, 2016 at 3:03 PM, Rodrigo Cavalcante wrote: > On alternate days my backup is failing, by the pg_stop_backup process () > does not perform or quit. > > Version PostgreSQL: 9.1.6 > > The following backup script: > > PGDATA=/dados > SAVE_BASE_DIR=/backup/diario > backup="'bkpfull'" > da

Re: [HACKERS] pg_dump broken for non-super user

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 10:57 AM, Stephen Frost wrote: > * Craig Ringer (cr...@2ndquadrant.com) wrote: >> On 3 May 2016 at 19:04, Rushabh Lathia wrote: >> >> > With commit a9f0e8e5a2e779a888988cb64479a6723f668c84, now pg_dump, use a >> > bitmap >> > to represent what to include. With this commit i

Re: [HACKERS] SET ROLE and reserved roles

2016-05-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Apr 26, 2016 at 7:39 PM, Robert Haas wrote: > > On Mon, Apr 25, 2016 at 6:55 PM, Stephen Frost wrote: > >> Based on our discussion at PGConf.US and the comments up-thread from > >> Tom, I'll work up a patch to remove those checks around SET R

Re: [HACKERS] pg_dump broken for non-super user

2016-05-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, May 3, 2016 at 10:57 AM, Stephen Frost wrote: > > * Craig Ringer (cr...@2ndquadrant.com) wrote: > >> On 3 May 2016 at 19:04, Rushabh Lathia wrote: > >> > >> > With commit a9f0e8e5a2e779a888988cb64479a6723f668c84, now pg_dump, use a > >> > bit

Re: [HACKERS] SET ROLE and reserved roles

2016-05-03 Thread Robert Haas
On Tue, Apr 26, 2016 at 7:39 PM, Robert Haas wrote: > On Mon, Apr 25, 2016 at 6:55 PM, Stephen Frost wrote: >> Based on our discussion at PGConf.US and the comments up-thread from >> Tom, I'll work up a patch to remove those checks around SET ROLE and >> friends which were trying to prevent defau

Re: [HACKERS] pg_dump broken for non-super user

2016-05-03 Thread Stephen Frost
* Rushabh Lathia (rushabh.lat...@gmail.com) wrote: > With commit a9f0e8e5a2e779a888988cb64479a6723f668c84, now pg_dump, use a > bitmap > to represent what to include. With this commit if non-super user is unable > to perform the dump. [...] > pg_dump: [archiver (db)] query was: LOCK TABLE pg_catalo

Re: [HACKERS] pgindent fixups

2016-05-03 Thread Robert Haas
On Tue, May 3, 2016 at 9:40 AM, Tom Lane wrote: > Robert Haas writes: >> 1. Is pgindent supposed to touch DATA() lines? Because it does. > > It always has. > >> 2. CustomPathMethods is not in the buildfarm's typedefs.list. Why not? > > Probably because there are no variables, parameters, or fie

Re: [HACKERS] pg_dump broken for non-super user

2016-05-03 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > On 3 May 2016 at 19:04, Rushabh Lathia wrote: > > > With commit a9f0e8e5a2e779a888988cb64479a6723f668c84, now pg_dump, use a > > bitmap > > to represent what to include. With this commit if non-super user is unable > > to perform the dump. > > > >

Re: [HACKERS] Need help debugging why autovacuum seems "stuck" -- until I use superuser to vacuum freeze pg_database

2016-05-03 Thread Robert Haas
On Sun, May 1, 2016 at 10:39 PM, McCoy, Shawn wrote: > I have been debugging a problem on a 9.3.10 Postgres database cluster with > over 1200 databases. 10 workers, increased maintenance_work_mem, auto > vacuum settings to run more frequently than default. What I will notice is > that autovacuu

Re: [HACKERS] max_worker_processes missing some documentation

2016-05-03 Thread Robert Haas
On Mon, May 2, 2016 at 2:46 PM, Julien Rouhaud wrote: > I noticed that postgresql.conf.sample doesn't say that changing > max_worker_processes requires a restart. > > Patch attached. Good catch. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Compa

Re: [HACKERS] SPI_exec ERROR in pl/r of R 3.2.4 on PostgreSQL on Windows 7

2016-05-03 Thread dandl
Windows 7 vs 10 makes no odds here. What matters is whether your Postgres or R installations are x64 or x86. Any x64 Windows can run x64 or x86 software perfectly happily, but you can’t mix a DLL with an EXE of a different flavour. They have to match. Regards David M Bennett FACS

Re: [HACKERS] pg_dump broken for non-super user

2016-05-03 Thread Craig Ringer
On 3 May 2016 at 19:04, Rushabh Lathia wrote: > With commit a9f0e8e5a2e779a888988cb64479a6723f668c84, now pg_dump, use a > bitmap > to represent what to include. With this commit if non-super user is unable > to perform the dump. > Hm. I think we need a src/bin/pg_dump/t/ test for the TAP suite

Re: [HACKERS] Processes and caches in postgresql

2016-05-03 Thread Craig Ringer
On 3 May 2016 at 21:37, Merlin Moncure wrote: > > There is library out there, unfortunately GPL licensed, that attempts > to fully implement posix including fork(): http://midipix.org/. One > of these days I'd like to have a go at porting postgres to it. ... and here I thought you'd be keen t

Re: [HACKERS] full table delete query

2016-05-03 Thread David G. Johnston
On Tue, May 3, 2016 at 5:51 AM, hari.prasath wrote: > Hi all, > How postgresql handles full table delete in terms of loading the > full table in these scenarios > > consider one big table(tablename: bigtable) > and the query will be >> delete from bigtable; > > 1)which doesn't have any fore

[HACKERS] Re: Logical decoding timeline following fails to handle records split across segments

2016-05-03 Thread Craig Ringer
On 3 May 2016 at 22:03, Craig Ringer wrote: > A corrected and handily much, much simpler patch is attached. The logic > for finding the last timeline on a segment was massively more complex than > it needed to be, and that wasn't the only thing. > I'm aware that this is late in the piece, btw,

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-03 Thread Tomas Vondra
Hi, On 05/02/2016 09:18 PM, Robert Haas wrote: On Sat, Apr 30, 2016 at 1:35 PM, Tom Lane wrote: Julien Rouhaud writes: On 29/04/2016 18:05, Tom Lane wrote: Julien Rouhaud writes: The segfault is caused by quals_match_foreign_key() calling get_leftop() and get_rightop() on a ScalarArrayOpE

Re: [HACKERS] pg9.6 segfault using simple query (related to use fk for join estimates)

2016-05-03 Thread Tomas Vondra
Hi, On 04/30/2016 07:35 PM, Tom Lane wrote: Julien Rouhaud writes: On 29/04/2016 18:05, Tom Lane wrote: Julien Rouhaud writes: The segfault is caused by quals_match_foreign_key() calling get_leftop() and get_rightop() on a ScalarArrayOpExpr node. I'm not sure that assuming this compatibili

  1   2   >