Re: [HACKERS] fd.c: flush data problems on osx

2016-04-02 Thread Michael Paquier
On Mon, Mar 21, 2016 at 9:09 PM, Stas Kelvich wrote: > On 21 Mar 2016, at 14:53, Andres Freund wrote: >> Hm. I think we should rather just skip calling pg_flush_data in the >> directory case, that very likely isn't beneficial on any OS. > > Seems reasonable, changed. > >> I think we still need to

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-02 Thread David Rowley
On 2 April 2016 at 05:52, Tom Lane wrote: > > David Rowley writes: > > On 12 March 2016 at 11:43, Tom Lane wrote: > >> It seems like the major intellectual complexity here is to figure out > >> how to detect inner-side-unique at reasonable cost. I see that for > >> LEFT joins you're caching tha

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-02 Thread Amit Kapila
On Thu, Mar 31, 2016 at 3:48 PM, Andres Freund wrote: > > On 2016-03-31 15:07:22 +0530, Amit Kapila wrote: > > On Thu, Mar 31, 2016 at 4:39 AM, Andres Freund wrote: > > > > > > On 2016-03-28 22:50:49 +0530, Amit Kapila wrote: > > > > On Fri, Sep 11, 2015 at 8:01 PM, Amit Kapila < amit.kapil...@gm

Re: [HACKERS] snapshot too old, configured by time

2016-04-02 Thread Michael Paquier
On Fri, Apr 1, 2016 at 11:45 PM, Alvaro Herrera wrote: > Kevin Grittner wrote: > >> Attached is what I think you're talking about for the first patch. >> AFAICS this should generate identical executable code to unpatched. >> Then the patch to actually implement the feature would, instead >> of add

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-02 Thread David Rowley
On 2 April 2016 at 23:26, David Rowley wrote: > I worked on this today to try and get it into shape. In the last patch I failed to notice that there's an alternative expected results file for one of the regression tests. The attached patch includes the fix to update that file to match the new ex

Re: [HACKERS] improving GROUP BY estimation

2016-04-02 Thread Dean Rasheed
On 31 March 2016 at 22:02, Tom Lane wrote: > I'm just concerned about what happens when the Dellera paper stops being > available. I don't mind including that URL as a backup to the written-out > argument I just suggested. > Here's an updated patch with references to both papers, and a more deta

Re: [HACKERS] [PATCH v11] GSSAPI encryption support

2016-04-02 Thread Michael Paquier
On Sat, Apr 2, 2016 at 7:34 AM, Robbie Harwood wrote: > - Attempt to address a crash Michael is observing by switching to using > the StringInfo/pqExpBuffer management functions over my own code as > much as possible. Michael, if this doesn't fix it, I'm out of ideas. Nope, it doesn't. >

Re: [HACKERS] snapshot too old, configured by time

2016-04-02 Thread Kevin Grittner
On Sat, Apr 2, 2016 at 7:12 AM, Michael Paquier wrote: > On Fri, Apr 1, 2016 at 11:45 PM, Alvaro Herrera > wrote: >> Kevin Grittner wrote: >> >>> Attached is what I think you're talking about for the first patch. >>> AFAICS this should generate identical executable code to unpatched. >>> Then th

Re: [HACKERS] improving GROUP BY estimation

2016-04-02 Thread Tom Lane
Dean Rasheed writes: > Here's an updated patch with references to both papers, and a more > detailed justification for the formula, along with the other changes > discussed. Note that although equation (2) in the Dell'Era paper looks > different from the Yao formula, it's actually the same. Looks

[HACKERS] Small typo in a comment in pg_regress.c

2016-04-02 Thread Andreas 'ads' Scherbaum
Hi, stumbled over this while looking into the source. Patch attached. Regards, -- Andreas 'ads' Scherbaum German PostgreSQL User Group European PostgreSQL User Group - Board of Directors Volunteer Regional Contact, Germany - PostgreSQL Project diff --git a/src/

Re: [HACKERS] Small typo in a comment in pg_regress.c

2016-04-02 Thread Stephen Frost
Andreas, * Andreas 'ads' Scherbaum (adsm...@wars-nicht.de) wrote: > stumbled over this while looking into the source. Patch attached. Fix pushed, thanks! Stephen signature.asc Description: Digital signature

Transactional enum additions - was Re: [HACKERS] Alter or rename enum value

2016-04-02 Thread Andrew Dunstan
On 03/29/2016 04:56 PM, Andrew Dunstan wrote: On 03/27/2016 10:20 AM, Tom Lane wrote: Andrew Dunstan writes: The more I think about this the more I bump up against the fact that almost anything we do might want to do to ameliorate the situation is going to be rolled back. The only approach

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Tom Lane
"Shulgin, Oleksandr" writes: > On Apr 1, 2016 23:14, "Tom Lane" wrote: >> Haven't looked at 0002 yet. > [crosses fingers] hope you'll have a chance to do that before feature > freeze for 9.6 I studied this patch for awhile after rebasing it onto yesterday's commits. I did not like the fact tha

Re: Transactional enum additions - was Re: [HACKERS] Alter or rename enum value

2016-04-02 Thread Tom Lane
Andrew Dunstan writes: > Looking at this briefly. It looks like the check should be called from > enum_in() and enum_recv(). What error should be raised if the enum row's > xmin isn't committed? ERRCODE_FEATURE_NOT_SUPPORTED? or maybe > ERRCODE_DATA_EXCEPTION? I don't see anything that fits ver

Re: [HACKERS] raw output from copy

2016-04-02 Thread Andrew Dunstan
On 04/01/2016 11:42 AM, Daniel Verite wrote: Andrew Dunstan wrote: If someone can make a good case that this is going to be of general use I'll happily go along, but I haven't seen one so far. About COPY FROM with a raw format, for instance just yesterday there was this user question

Re: [HACKERS] Batch update of indexes

2016-04-02 Thread Tom Lane
Konstantin Knizhnik writes: > Attached please find patch for "ALTER INDEX ... WHERE ..." clause. > It is now able to handle all three possible situations: > 1. Making index partial (add WHERE condition to the ordinary index) > 2. Extend partial index range (less restricted index predicate) > 3. Ar

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Shulgin, Oleksandr
On Apr 2, 2016 18:38, "Tom Lane" wrote: > > "Shulgin, Oleksandr" writes: > > On Apr 1, 2016 23:14, "Tom Lane" wrote: > >> Haven't looked at 0002 yet. > > > [crosses fingers] hope you'll have a chance to do that before feature > > freeze for 9.6 > > I studied this patch for awhile after rebasing

Re: [HACKERS] Batch update of indexes

2016-04-02 Thread Konstantin Knizhnik
On 04/02/2016 09:57 PM, Tom Lane wrote: Konstantin Knizhnik writes: Attached please find patch for "ALTER INDEX ... WHERE ..." clause. It is now able to handle all three possible situations: 1. Making index partial (add WHERE condition to the ordinary index) 2. Extend partial index range (less

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Tom Lane
"Shulgin, Oleksandr" writes: > On Mon, Mar 14, 2016 at 7:55 PM, Tom Lane wrote: >> Yeah, I don't much like that either. But I don't think we can avoid >> some refactoring there; as designed, conversion of an error message into >> user-visible form is too tightly tied to receipt of the message.

Re: [HACKERS] standalone backend PANICs during recovery

2016-04-02 Thread Alvaro Herrera
Bernd Helmle wrote: > While investigating a problem on a streaming hot standby instance at a > customer site, i got the following when using a standalone backend: > > PANIC: btree_xlog_delete_get_latestRemovedXid: cannot operate with > inconsistent data > CONTEXT: xlog redo delete: index 1663/65

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Greg Stark
On Sat, Apr 2, 2016 at 3:31 PM, Tomas Vondra wrote: > So let me be clear: I do think the patch seems to be a significant > performance improvement for most of the queries, and I'm OK with accepting a > few regressions (particularly if we agree those are pathological cases, > unlikely to happen in

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Peter Geoghegan
On Sat, Apr 2, 2016 at 3:20 PM, Greg Stark wrote: > There are also some weird cases in this list where there's a > significant regression at 32MB but not at 8MB. I would like to see > 16MB and perhaps 12MB and 24MB. They would help understand if these > are just quirks or there's a consistent patt

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Peter Geoghegan
On Sat, Apr 2, 2016 at 3:20 PM, Greg Stark wrote: > These are the averages across all queries across all data sets for the > run-time for the patch versus master (not patched 64 which I think is > the replacement_sort_mem=64MB which appears to not be a win). So even > in the less successful cases

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Tom Lane
I wrote: > Attached is a libpq-portion-only version of a patch doing it this way. > I've not yet looked at the psql part of your patch. Here's an update for the psql side. regards, tom lane diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Peter Geoghegan
On Sat, Apr 2, 2016 at 7:31 AM, Tomas Vondra wrote: > So, I do have the results from both machines - I've attached the basic > comparison spreadsheets, the complete summary is available here: > >https://github.com/tvondra/sort-benchmark > > The database log also includes the logs for trace_sor

Re: [HACKERS] Using quicksort for every external sort run

2016-04-02 Thread Peter Geoghegan
On Sat, Apr 2, 2016 at 3:22 PM, Peter Geoghegan wrote: > On Sat, Apr 2, 2016 at 3:20 PM, Greg Stark wrote: >> There are also some weird cases in this list where there's a >> significant regression at 32MB but not at 8MB. I would like to see >> 16MB and perhaps 12MB and 24MB. They would help under

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Peter Geoghegan
On Thu, Feb 11, 2016 at 9:50 AM, Robert Haas wrote: >> Actually, what'd be really handy IMO is something to regurgitate the >> most recent error in verbose mode, without making a permanent session >> state change. Something like >> >> regression=# insert into bar values(1); >> ERROR: insert or u

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Alex Shulgin
On Sat, Apr 2, 2016 at 8:57 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Apr 2, 2016 18:38, "Tom Lane" wrote: > >> I did not like the fact that the compute_scalar_stats logic >> would allow absolutely anything into the MCV list once num_hist falls >> below 2. I think it's imp

Re: [HACKERS] Re: [COMMITTERS] pgsql: Refer to a TOKEN_USER payload as a "token user, " not as a "user

2016-04-02 Thread Noah Misch
On Fri, Apr 01, 2016 at 11:07:01PM -0400, Tom Lane wrote: > Stephen Frost writes: > > * Noah Misch (n...@leadboat.com) wrote: > >> I see some advantages of writing "TokenUser", as you propose. However, our > >> error style guide says "Avoid mentioning called function names, either; > >> instead s

Re: [HACKERS] Re: [COMMITTERS] pgsql: Refer to a TOKEN_USER payload as a "token user, " not as a "user

2016-04-02 Thread Stephen Frost
* Noah Misch (n...@leadboat.com) wrote: > On Fri, Apr 01, 2016 at 11:07:01PM -0400, Tom Lane wrote: > > Stephen Frost writes: > > > * Noah Misch (n...@leadboat.com) wrote: > > >> I see some advantages of writing "TokenUser", as you propose. However, > > >> our > > >> error style guide says "Avoi

Re: [HACKERS] PostgreSQL 9.6 Feature Freeze - April 8, 2016

2016-04-02 Thread Noah Misch
On Tue, Mar 15, 2016 at 11:35:31AM -0400, Robert Haas wrote: > At the recent PostgreSQL developer meeting in Brussels, a consensus > was reached that an early beta, leading to an on-time release, would > be very desirable. In particular, it was suggested that we should > attempt to release Postgre

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-02 Thread Dilip Kumar
On Fri, Apr 1, 2016 at 2:09 PM, Andres Freund wrote: > One interesting thing to do would be to use -P1 during the test and see > how much the performance varies over time. > I have run with -P option, I ran for 1200 second and set -P as 30 second, and what I observed is that when its low its low

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Alex Shulgin
On Sat, Apr 2, 2016 at 11:41 PM, Tom Lane wrote: > "Shulgin, Oleksandr" writes: > > On Mon, Mar 14, 2016 at 7:55 PM, Tom Lane wrote: > >> Yeah, I don't much like that either. But I don't think we can avoid > >> some refactoring there; as designed, conversion of an error message into > >> user-

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Alex Shulgin
On Sun, Apr 3, 2016 at 3:43 AM, Alex Shulgin wrote: > > I'm not sure yet about the 1% rule for the last value, but would also love > to see if we can avoid the arbitrary limit here. What happens with a last > value which is less than 1% popular in the current code anyway? > Tom, Now that I thi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Refer to a TOKEN_USER payload as a "token user, " not as a "user

2016-04-02 Thread Tom Lane
Stephen Frost writes: > * Noah Misch (n...@leadboat.com) wrote: >> The wording in GetTokenUser() and AddUserToTokenDacl() seems fine; let's >> standardize on that. Also, every GetTokenUser() failure has been yielding >> two >> messages, the second contributing no further detail. I'll reduce tha

Re: [HACKERS] Add schema-qualified relnames in constraint error messages.

2016-04-02 Thread Tom Lane
Alex Shulgin writes: > What about regression tests? My assumption was that we won't be able to > add them with the usual expected file approach, but that we also don't need > it that hard. Everyone's in favor? It'd be nice to have a regression test, but I concur that the LOCATION information is

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Tom Lane
Alex Shulgin writes: > On Sun, Apr 3, 2016 at 3:43 AM, Alex Shulgin wrote: >> I'm not sure yet about the 1% rule for the last value, but would also love >> to see if we can avoid the arbitrary limit here. What happens with a last >> value which is less than 1% popular in the current code anyway?

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Alex Shulgin
On Sun, Apr 3, 2016 at 7:18 AM, Tom Lane wrote: > Alex Shulgin writes: > > On Sun, Apr 3, 2016 at 3:43 AM, Alex Shulgin > wrote: > >> I'm not sure yet about the 1% rule for the last value, but would also > love > >> to see if we can avoid the arbitrary limit here. What happens with a > last >

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Tom Lane
Alex Shulgin writes: > On Sun, Apr 3, 2016 at 7:18 AM, Tom Lane wrote: >> Well, we have to do *something* with the last (possibly only) value. >> Neither "include always" nor "omit always" seem sane to me. What other >> decision rule do you want there? > Well, what implies that the last value i

[HACKERS] pgbench more operators & functions

2016-04-02 Thread Fabien COELHO
Hello, Here is a simple patch which adds a bunch of operators (bitwise: & | ^ ~, comparisons: =/== <>/!= < <= > >=, logical: and/&& or/|| xor/^^ not/!) and functions (exp ln if) to pgbench. I've tried to be pg's SQL compatible where appropriate. Also attached is a simple test script. Some

Re: [HACKERS] More stable query plans via more predictable column statistics

2016-04-02 Thread Alex Shulgin
On Sun, Apr 3, 2016 at 7:49 AM, Tom Lane wrote: > Alex Shulgin writes: > > On Sun, Apr 3, 2016 at 7:18 AM, Tom Lane wrote: > >> Well, we have to do *something* with the last (possibly only) value. > >> Neither "include always" nor "omit always" seem sane to me. What other > >> decision rule do