Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Greg Stark
On Sat, Apr 2, 2016 at 3:31 PM, Tomas Vondra wrote: > So let me be clear: I do think the patch seems to be a significant > performance improvement for most of the queries, and I'm OK with accepting a > few regressions (particularly if we agree those are pathological

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Peter Geoghegan
On Sat, Apr 2, 2016 at 7:31 AM, Tomas Vondra wrote: > So, I do have the results from both machines - I've attached the basic > comparison spreadsheets, the complete summary is available here: > >https://github.com/tvondra/sort-benchmark > > The database log also

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Tom Lane
Alex Shulgin writes: > On Sun, Apr 3, 2016 at 3:43 AM, Alex Shulgin wrote: >> I'm not sure yet about the 1% rule for the last value, but would also love >> to see if we can avoid the arbitrary limit here. What happens with a last >> value which is

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Tom Lane
Alex Shulgin writes: > On Sun, Apr 3, 2016 at 7:18 AM, Tom Lane wrote: >> Well, we have to do *something* with the last (possibly only) value. >> Neither "include always" nor "omit always" seem sane to me. What other >> decision rule do you want

Re: [HACKERS] Re: [COMMITTERS] pgsql: Refer to a TOKEN_USER payload as a "token user, " not as a "user

2016-04-02 Thread Tom Lane
Stephen Frost writes: > * Noah Misch (n...@leadboat.com) wrote: >> The wording in GetTokenUser() and AddUserToTokenDacl() seems fine; let's >> standardize on that. Also, every GetTokenUser() failure has been yielding >> two >> messages, the second contributing no further

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Tom Lane
I wrote: > Attached is a libpq-portion-only version of a patch doing it this way. > I've not yet looked at the psql part of your patch. Here's an update for the psql side. regards, tom lane diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-02 Thread Dilip Kumar
On Fri, Apr 1, 2016 at 2:09 PM, Andres Freund wrote: > One interesting thing to do would be to use -P1 during the test and see > how much the performance varies over time. > I have run with -P option, I ran for 1200 second and set -P as 30 second, and what I observed is that

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Peter Geoghegan
On Thu, Feb 11, 2016 at 9:50 AM, Robert Haas wrote: >> Actually, what'd be really handy IMO is something to regurgitate the >> most recent error in verbose mode, without making a permanent session >> state change. Something like >> >> regression=# insert into bar

Re: [HACKERS] Re: [COMMITTERS] pgsql: Refer to a TOKEN_USER payload as a "token user, " not as a "user

2016-04-02 Thread Stephen Frost
* Noah Misch (n...@leadboat.com) wrote: > On Fri, Apr 01, 2016 at 11:07:01PM -0400, Tom Lane wrote: > > Stephen Frost writes: > > > * Noah Misch (n...@leadboat.com) wrote: > > >> I see some advantages of writing "TokenUser", as you propose. However, > > >> our > > >> error

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Peter Geoghegan
On Sat, Apr 2, 2016 at 3:22 PM, Peter Geoghegan wrote: > On Sat, Apr 2, 2016 at 3:20 PM, Greg Stark wrote: >> There are also some weird cases in this list where there's a >> significant regression at 32MB but not at 8MB. I would like to see >> 16MB and perhaps

Re: [HACKERS] PostgreSQL 9.6 Feature Freeze - April 8, 2016

2016-04-02 Thread Noah Misch
On Tue, Mar 15, 2016 at 11:35:31AM -0400, Robert Haas wrote: > At the recent PostgreSQL developer meeting in Brussels, a consensus > was reached that an early beta, leading to an on-time release, would > be very desirable. In particular, it was suggested that we should > attempt to release

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Alex Shulgin
On Sun, Apr 3, 2016 at 7:18 AM, Tom Lane wrote: > Alex Shulgin writes: > > On Sun, Apr 3, 2016 at 3:43 AM, Alex Shulgin > wrote: > >> I'm not sure yet about the 1% rule for the last value, but would also > love > >> to see if

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Peter Geoghegan
On Sat, Apr 2, 2016 at 3:20 PM, Greg Stark wrote: > There are also some weird cases in this list where there's a > significant regression at 32MB but not at 8MB. I would like to see > 16MB and perhaps 12MB and 24MB. They would help understand if these > are just quirks or there's a

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Peter Geoghegan
On Sat, Apr 2, 2016 at 3:20 PM, Greg Stark wrote: > These are the averages across all queries across all data sets for the > run-time for the patch versus master (not patched 64 which I think is > the replacement_sort_mem=64MB which appears to not be a win). So even > in the less

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Alex Shulgin
On Sat, Apr 2, 2016 at 8:57 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Apr 2, 2016 18:38, "Tom Lane" wrote: > >> I did not like the fact that the compute_scalar_stats logic >> would allow absolutely anything into the MCV list once num_hist falls >>

[HACKERS] pgbench more operators & functions

2016-04-02 Thread Fabien COELHO
Hello, Here is a simple patch which adds a bunch of operators (bitwise: & | ^ ~, comparisons: =/== <>/!= < <= > >=, logical: and/&& or/|| xor/^^ not/!) and functions (exp ln if) to pgbench. I've tried to be pg's SQL compatible where appropriate. Also attached is a simple test script. Some

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Alex Shulgin
On Sun, Apr 3, 2016 at 3:43 AM, Alex Shulgin wrote: > > I'm not sure yet about the 1% rule for the last value, but would also love > to see if we can avoid the arbitrary limit here. What happens with a last > value which is less than 1% popular in the current code

Re: [HACKERS] Re: [COMMITTERS] pgsql: Refer to a TOKEN_USER payload as a "token user, " not as a "user

2016-04-02 Thread Noah Misch
On Fri, Apr 01, 2016 at 11:07:01PM -0400, Tom Lane wrote: > Stephen Frost writes: > > * Noah Misch (n...@leadboat.com) wrote: > >> I see some advantages of writing "TokenUser", as you propose. However, our > >> error style guide says "Avoid mentioning called function names,

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Alex Shulgin
On Sat, Apr 2, 2016 at 11:41 PM, Tom Lane wrote: > "Shulgin, Oleksandr" writes: > > On Mon, Mar 14, 2016 at 7:55 PM, Tom Lane wrote: > >> Yeah, I don't much like that either. But I don't think we can avoid > >> some

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Tom Lane
Alex Shulgin writes: > What about regression tests? My assumption was that we won't be able to > add them with the usual expected file approach, but that we also don't need > it that hard. Everyone's in favor? It'd be nice to have a regression test, but I concur that

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

2016-04-02 Thread Masahiko Sawada
On Fri, Apr 1, 2016 at 9:10 AM, Noah Misch wrote: > On Thu, Mar 31, 2016 at 04:48:26PM +0900, Masahiko Sawada wrote: >> On Thu, Mar 31, 2016 at 2:02 PM, Noah Misch wrote: >> > On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote: >> >> As a result

Re: [HACKERS] pgbench - remove unused clientDone parameter

2016-04-02 Thread Fabien COELHO
Remove pgbench clientDone unused "ok" parameter. Seems useless, yeah, removed. Good! Actually it was introduced as an unused argument in 3da0dfb4b146. Strange indeed. I did not thought to dig into the history. I switchted the corresponding commitfest entry as committed. Thanks, --

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-02 Thread Amit Kapila
On Thu, Mar 31, 2016 at 3:48 PM, Andres Freund wrote: > > On 2016-03-31 15:07:22 +0530, Amit Kapila wrote: > > On Thu, Mar 31, 2016 at 4:39 AM, Andres Freund wrote: > > > > > > On 2016-03-28 22:50:49 +0530, Amit Kapila wrote: > > > > On Fri, Sep 11, 2015

Re: [HACKERS] fd.c: flush data problems on osx

2016-04-02 Thread Michael Paquier
On Mon, Mar 21, 2016 at 9:09 PM, Stas Kelvich wrote: > On 21 Mar 2016, at 14:53, Andres Freund wrote: >> Hm. I think we should rather just skip calling pg_flush_data in the >> directory case, that very likely isn't beneficial on any OS. > > Seems

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-02 Thread David Rowley
On 2 April 2016 at 05:52, Tom Lane wrote: > > David Rowley writes: > > On 12 March 2016 at 11:43, Tom Lane wrote: > >> It seems like the major intellectual complexity here is to figure out > >> how to detect inner-side-unique

Re: [HACKERS] [PATCH v11] GSSAPI encryption support

2016-04-02 Thread Michael Paquier
On Sat, Apr 2, 2016 at 7:34 AM, Robbie Harwood wrote: > - Attempt to address a crash Michael is observing by switching to using > the StringInfo/pqExpBuffer management functions over my own code as > much as possible. Michael, if this doesn't fix it, I'm out of ideas.

Re: [HACKERS] snapshot too old, configured by time

2016-04-02 Thread Kevin Grittner
On Sat, Apr 2, 2016 at 7:12 AM, Michael Paquier wrote: > On Fri, Apr 1, 2016 at 11:45 PM, Alvaro Herrera > wrote: >> Kevin Grittner wrote: >> >>> Attached is what I think you're talking about for the first patch. >>> AFAICS this should

Re: [HACKERS] snapshot too old, configured by time

2016-04-02 Thread Michael Paquier
On Fri, Apr 1, 2016 at 11:45 PM, Alvaro Herrera wrote: > Kevin Grittner wrote: > >> Attached is what I think you're talking about for the first patch. >> AFAICS this should generate identical executable code to unpatched. >> Then the patch to actually implement the

Re: [HACKERS] improving GROUP BY estimation

2016-04-02 Thread Dean Rasheed
On 31 March 2016 at 22:02, Tom Lane wrote: > I'm just concerned about what happens when the Dellera paper stops being > available. I don't mind including that URL as a backup to the written-out > argument I just suggested. > Here's an updated patch with references to both

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-02 Thread David Rowley
On 2 April 2016 at 23:26, David Rowley wrote: > I worked on this today to try and get it into shape. In the last patch I failed to notice that there's an alternative expected results file for one of the regression tests. The attached patch includes the fix to

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Tom Lane
"Shulgin, Oleksandr" writes: > On Apr 1, 2016 23:14, "Tom Lane" wrote: >> Haven't looked at 0002 yet. > [crosses fingers] hope you'll have a chance to do that before feature > freeze for 9.6 I studied this patch for awhile after rebasing it

Transactional enum additions - was Re: [HACKERS] Alter or rename enum value

2016-04-02 Thread Andrew Dunstan
On 03/29/2016 04:56 PM, Andrew Dunstan wrote: On 03/27/2016 10:20 AM, Tom Lane wrote: Andrew Dunstan writes: The more I think about this the more I bump up against the fact that almost anything we do might want to do to ameliorate the situation is going to be rolled

Re: [HACKERS] improving GROUP BY estimation

2016-04-02 Thread Tom Lane
Dean Rasheed writes: > Here's an updated patch with references to both papers, and a more > detailed justification for the formula, along with the other changes > discussed. Note that although equation (2) in the Dell'Era paper looks > different from the Yao formula,

[HACKERS] Small typo in a comment in pg_regress.c

2016-04-02 Thread Andreas 'ads' Scherbaum
Hi, stumbled over this while looking into the source. Patch attached. Regards, -- Andreas 'ads' Scherbaum German PostgreSQL User Group European PostgreSQL User Group - Board of Directors Volunteer Regional Contact, Germany - PostgreSQL Project diff --git

Re: [HACKERS] Small typo in a comment in pg_regress.c

2016-04-02 Thread Stephen Frost
Andreas, * Andreas 'ads' Scherbaum (adsm...@wars-nicht.de) wrote: > stumbled over this while looking into the source. Patch attached. Fix pushed, thanks! Stephen signature.asc Description: Digital signature

Re: [HACKERS] Batch update of indexes

2016-04-02 Thread Konstantin Knizhnik
On 04/02/2016 09:57 PM, Tom Lane wrote: Konstantin Knizhnik writes: Attached please find patch for "ALTER INDEX ... WHERE ..." clause. It is now able to handle all three possible situations: 1. Making index partial (add WHERE condition to the ordinary index) 2.

Re: [HACKERS] raw output from copy

2016-04-02 Thread Andrew Dunstan
On 04/01/2016 11:42 AM, Daniel Verite wrote: Andrew Dunstan wrote: If someone can make a good case that this is going to be of general use I'll happily go along, but I haven't seen one so far. About COPY FROM with a raw format, for instance just yesterday there was this user

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Shulgin, Oleksandr
On Apr 2, 2016 18:38, "Tom Lane" wrote: > > "Shulgin, Oleksandr" writes: > > On Apr 1, 2016 23:14, "Tom Lane" wrote: > >> Haven't looked at 0002 yet. > > > [crosses fingers] hope you'll have a chance to do that before feature

Re: Transactional enum additions - was Re: [HACKERS] Alter or rename enum value

2016-04-02 Thread Tom Lane
Andrew Dunstan writes: > Looking at this briefly. It looks like the check should be called from > enum_in() and enum_recv(). What error should be raised if the enum row's > xmin isn't committed? ERRCODE_FEATURE_NOT_SUPPORTED? or maybe > ERRCODE_DATA_EXCEPTION? I don't see

Re: [HACKERS] Batch update of indexes

2016-04-02 Thread Tom Lane
Konstantin Knizhnik writes: > Attached please find patch for "ALTER INDEX ... WHERE ..." clause. > It is now able to handle all three possible situations: > 1. Making index partial (add WHERE condition to the ordinary index) > 2. Extend partial index range (less

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Tom Lane
"Shulgin, Oleksandr" writes: > On Mon, Mar 14, 2016 at 7:55 PM, Tom Lane wrote: >> Yeah, I don't much like that either. But I don't think we can avoid >> some refactoring there; as designed, conversion of an error message into >> user-visible

Re: [HACKERS] standalone backend PANICs during recovery

2016-04-02 Thread Alvaro Herrera
Bernd Helmle wrote: > While investigating a problem on a streaming hot standby instance at a > customer site, i got the following when using a standalone backend: > > PANIC: btree_xlog_delete_get_latestRemovedXid: cannot operate with > inconsistent data > CONTEXT: xlog redo delete: index