Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Thu, Apr 7, 2016 at 10:02 AM, Fujii Masao wrote: > > On Thu, Apr 7, 2016 at 1:22 PM, Amit Kapila wrote: > > > > But for that, I think we don't need to do anything extra. I mean > > write_nondefault_variables() will automatically write the

Re: [HACKERS] pg_ctl promote wait

2016-04-06 Thread Michael Paquier
On Wed, Mar 23, 2016 at 1:47 AM, David Steele wrote: > On 3/16/16 12:19 PM, David Steele wrote: >> Hi Peter, >> >> On 3/9/16 3:08 PM, Michael Paquier wrote: >> >>> Here are some comments about 0002 >> <...> >>> I think that we had better do something like the attached first.

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-06 Thread Andres Freund
Hi, On 2016-04-07 09:14:00 +0530, Amit Kapila wrote: > On Sat, Apr 2, 2016 at 5:25 PM, Amit Kapila wrote: > I have ran exactly same test on intel x86 m/c and the results are as below: Thanks for running these tests! > Client Count/Patch_ver (tps) 2 128 256 > HEAD –

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-04-06 Thread Karl O. Pinc
On Wed, 6 Apr 2016 22:26:13 -0500 "Karl O. Pinc" wrote: > On Wed, 23 Mar 2016 23:22:26 +0100 > Gilles Darold wrote: > > > Thanks for the reminder, here is the v3 of the patch after a deeper > > review and testing. It is now registered to the next commit

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Thu, Apr 7, 2016 at 1:22 PM, Amit Kapila wrote: > On Wed, Apr 6, 2016 at 8:11 PM, Fujii Masao wrote: >> >> On Wed, Apr 6, 2016 at 11:14 PM, Amit Kapila >> wrote: >> > On Wed, Apr 6, 2016 at 7:03 PM, Fujii Masao

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Wed, Apr 6, 2016 at 8:11 PM, Fujii Masao wrote: > > On Wed, Apr 6, 2016 at 11:14 PM, Amit Kapila wrote: > > On Wed, Apr 6, 2016 at 7:03 PM, Fujii Masao wrote: > >> > >> On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On April 7, 2016 2:26:41 AM GMT+02:00, Michael Paquier wrote: >On Thu, Apr 7, 2016 at 12:55 AM, Andres Freund >wrote: >> On 2016-04-06 16:49:17 +0100, Simon Riggs wrote: >>> Perhaps easy to solve, but how do we test it is solved? >> >> Maybe

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-06 Thread Amit Kapila
On Sat, Apr 2, 2016 at 5:25 PM, Amit Kapila wrote: > On Thu, Mar 31, 2016 at 3:48 PM, Andres Freund wrote: > > Here is the performance data (configuration of machine used to perform > this test is mentioned at end of mail): > > Non-default parameters

[HACKERS] Re: [COMMITTERS] pgsql: Use GRANT system to manage access to sensitive functions

2016-04-06 Thread Stephen Frost
* Fujii Masao (masao.fu...@gmail.com) wrote: > On Thu, Apr 7, 2016 at 10:45 AM, Stephen Frost wrote: > > Use GRANT system to manage access to sensitive functions > > > > Now that pg_dump will properly dump out any ACL changes made to > > functions which exist in pg_catalog,

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-04-06 Thread Karl O. Pinc
Hi Gilles, On Wed, 23 Mar 2016 23:22:26 +0100 Gilles Darold wrote: > Thanks for the reminder, here is the v3 of the patch after a deeper > review and testing. It is now registered to the next commit fest under > the System Administration topic. I am going to try

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-04-06 Thread Noah Misch
On Wed, Apr 06, 2016 at 09:17:22AM +0200, Magnus Hagander wrote: > On Wed, Apr 6, 2016 at 6:42 AM, Noah Misch wrote: > > On Tue, Apr 05, 2016 at 08:15:16PM +0200, Magnus Hagander wrote: > > > I've pushed this version, and also added the item from the Brussels > > > developer

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Robert Haas
On Wed, Apr 6, 2016 at 10:04 AM, Dilip Kumar wrote: > On Wed, Apr 6, 2016 at 3:22 PM, Andres Freund wrote: >> Which scale did you initialize with? I'm trying to reproduce the >> workload on hydra as precisely as possible... > > I tested with scale

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-06 Thread David Rowley
On 7 April 2016 at 08:01, David Rowley wrote: > On 7 April 2016 at 04:05, Tom Lane wrote: >> Starting to look at this again. I wonder, now that you have the generic >> caching mechanism for remembering whether join inner sides have been >>

Re: [HACKERS] postgres_fdw : altering foreign table not invalidating prepare statement execution plan.

2016-04-06 Thread Kyotaro HORIGUCHI
Hi, At Tue, 5 Apr 2016 19:46:04 +0900, Amit Langote wrote in <5703976c.30...@lab.ntt.co.jp> > On 2016/04/05 18:44, Kyotaro HORIGUCHI wrote: > > At Tue, 5 Apr 2016 14:24:52 +0900, Amit Langote wrote: > > With this patch, making any change on foreign servers or user

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Michael Paquier
On Thu, Apr 7, 2016 at 12:55 AM, Andres Freund wrote: > On 2016-04-06 16:49:17 +0100, Simon Riggs wrote: >> Perhaps easy to solve, but how do we test it is solved? > > Maybe something like > > -- drain > pg_logical_slot_get_changes(...); > -- generate message in different

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Tom Lane
Robbie Harwood writes: > Tom Lane writes: >> Wait a second. So the initial connection-request packet is necessarily >> unencrypted under this scheme? > Yes, by necessity. The username must be sent in the clear, even if only > as part of the GSSAPI

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2016-04-06 Thread Tomas Vondra
Hi, attached is the patch split into two parts, as proposed by Simon. 0001 just adds the stuff to relcache, 0002 actually uses it for estimation. On 04/04/2016 12:03 PM, Amit Langote wrote: On 2016/04/04 17:25, Simon Riggs wrote: The rel cache code you're adding uses a flag called

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Michael Paquier
On Thu, Apr 7, 2016 at 7:44 AM, Michael Paquier wrote: > On Thu, Apr 7, 2016 at 6:11 AM, Petr Jelinek wrote: >> On 06/04/16 22:50, Andrew Dunstan wrote: >>> I have spent way too much time on this and don't have it working yet. >>> I'm setting up a

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Petr Jelinek
On 07/04/16 00:50, Michael Paquier wrote: On Thu, Apr 7, 2016 at 7:44 AM, Michael Paquier wrote: On Thu, Apr 7, 2016 at 6:11 AM, Petr Jelinek wrote: On 06/04/16 22:50, Andrew Dunstan wrote: I have spent way too much time on this and don't

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Michael Paquier
On Thu, Apr 7, 2016 at 6:11 AM, Petr Jelinek wrote: > On 06/04/16 22:50, Andrew Dunstan wrote: >> I have spent way too much time on this and don't have it working yet. >> I'm setting up a sacrificial VM from scratch in a last ditch attempt to >> get it working. >> >> Things

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2016-04-06 Thread Tom Lane
Pavel Stehule writes: > 1. We want this patch - it increase a functionality of autocomplete TBH, I do not think that is an agreed-to statement. I concur with Peter's comments upthread questioning how much use-case there is for interactive completion of IF (NOT) EXISTS.

Re: [HACKERS] [patch] Proposal for \crosstabview in psql

2016-04-06 Thread Alvaro Herrera
I've been looking at this patch. First thing was to rebase on top of recent psql code restructuring; second, pgindent; third, reordered the code in crosstabview.c more sensibly (had to add prototypes). New version attached. Then I looked at the docs to try to figure out exactly how it works.

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Robbie Harwood
Tom Lane writes: > Robbie Harwood writes: >> I need to flush this any time we might be doing encryption because it >> needs to be in a separate request to _secure_write() from what follows >> it. We don't know whether we should be doing encryption until

Re: [HACKERS] Combining Aggregates

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 22:28, David Rowley wrote: > On 7 April 2016 at 09:25, Simon Riggs wrote: > > On 5 April 2016 at 19:33, Robert Haas wrote: > > > >> > >> Committed 0002+0003 with those changes, some minor cosmetic

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Alvaro Herrera
Petr Jelinek wrote: > It's fun to set it up yes. I do have the machine with buildfarm client ready > still (although now also traveling so slightly complicated to get to it) but > I didn't activate it yet as I don't want it to just report failures forever. Maybe you should just activate it

Re: [HACKERS] Combining Aggregates

2016-04-06 Thread David Rowley
On 7 April 2016 at 09:25, Simon Riggs wrote: > On 5 April 2016 at 19:33, Robert Haas wrote: > >> >> Committed 0002+0003 with those changes, some minor cosmetic stuff, and >> of course the obligatory catversion bump. Oh, and fixed an OID >> conflict

Re: [HACKERS] Combining Aggregates

2016-04-06 Thread Simon Riggs
On 5 April 2016 at 19:33, Robert Haas wrote: > Committed 0002+0003 with those changes, some minor cosmetic stuff, and > of course the obligatory catversion bump. Oh, and fixed an OID > conflict with the patch Magnus just committed. Is that everything now? I don't see

Re: [HACKERS] Truncating/vacuuming relations on full tablespaces

2016-04-06 Thread Jim Nasby
On 4/6/16 11:06 AM, Robert Haas wrote: This is too late for 9.6 at this point and certainly requires discussion anyway, so please add it to the next CommitFest. If the goal here is to free up space via truncation when there's a real emergency, perhaps there's some other steps that should be

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Petr Jelinek
On 06/04/16 22:50, Andrew Dunstan wrote: On 03/29/2016 09:38 PM, Robert Haas wrote: On Tue, Mar 29, 2016 at 9:29 PM, Andrew Dunstan wrote: I am currently travelling, but my intention is to deal with the remaining patches when I'm back home this weekend, unless someone

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Tom Lane
Robbie Harwood writes: > I need to flush this any time we might be doing encryption because it > needs to be in a separate request to _secure_write() from what follows > it. We don't know whether we should be doing encryption until > connection parameters are parsed; to put

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-06 Thread David Rowley
On 7 April 2016 at 08:01, David Rowley wrote: > On 7 April 2016 at 04:05, Tom Lane wrote: >> Starting to look at this again. I wonder, now that you have the generic >> caching mechanism for remembering whether join inner sides have been >>

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-06 Thread Peter Geoghegan
On Wed, Apr 6, 2016 at 1:50 PM, Peter Geoghegan wrote: > Personally, I like documenting assertions, and will sometimes write > assertions that the compiler could easily optimize away. Maybe going > *that* far is more a matter of personal style, but I think an > assertion about

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Andrew Dunstan
On 03/29/2016 09:38 PM, Robert Haas wrote: On Tue, Mar 29, 2016 at 9:29 PM, Andrew Dunstan wrote: I am currently travelling, but my intention is to deal with the remaining patches when I'm back home this weekend, unless someone beats me to it. Cool. Progress report:

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-06 Thread Peter Geoghegan
On Wed, Apr 6, 2016 at 6:15 AM, Anastasia Lubennikova wrote: >> * I would like to see index_reform_tuple() assert that the new, >> truncated index tuple is definitely <= the original (I worry about the >> 1/3 page restriction issue). Maybe you should also change the

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2016-04-06 Thread Pavel Stehule
Hi 2016-04-04 7:58 GMT+02:00 Kyotaro HORIGUCHI : > Thank you for testing. That is a silly mistake, sorry. > > The attached is the fixed version. > > # Can I add a suffix to format-patche's output files? > > At Sat, 2 Apr 2016 07:18:32 +0200, Pavel Stehule

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-06 Thread David Rowley
On 7 April 2016 at 04:05, Tom Lane wrote: > David Rowley writes: >> In the last patch I failed to notice that there's an alternative >> expected results file for one of the regression tests. >> The attached patch includes the fix to update that

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Robbie Harwood
Stephen Frost writes: > Just an initial pass over the patch. Thanks! In the interest of brevity, if I haven't replied to something, I plan to fix it. >> /* >> - * Flush message so client will see it, except for AUTH_REQ_OK, which >> need >> - * not be sent

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Petr Jelinek
On 06/04/16 17:55, Andres Freund wrote: On 2016-04-06 16:49:17 +0100, Simon Riggs wrote: Perhaps easy to solve, but how do we test it is solved? Maybe something like -- drain pg_logical_slot_get_changes(...); -- generate message in different database, to ensure it's not processed -- in this

Re: [HACKERS] Choosing parallel_degree

2016-04-06 Thread Julien Rouhaud
On 06/04/2016 07:38, Amit Kapila wrote: > On Tue, Apr 5, 2016 at 11:55 PM, Julien Rouhaud >> >> In alter_table.sgml, I didn't comment the lock level needed to modify >> parallel_degree since it requires an access exclusive lock for now. >> While thinking about it, I think it's safe to use a share

Re: [HACKERS] insufficient qualification of some objects in dump files

2016-04-06 Thread Peter Eisentraut
On 04/05/2016 09:46 PM, Robert Haas wrote: On Fri, Mar 18, 2016 at 11:22 AM, Robert Haas wrote: On Fri, Mar 18, 2016 at 2:13 AM, Michael Paquier wrote: On Fri, Mar 18, 2016 at 5:38 AM, Tom Lane wrote: Given the lack of

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-06 Thread Teodor Sigaev
I've tested the v2, v3 and v3.1 of the patch, to see if there are any differences. The v2 no longer applies, so I tested it on ee943004. The following table shows the total duration of the data load, and also sizes of the two GIN indexes. duration (sec) subject body

Re: [HACKERS] [PATH] Jsonb, insert a new value into an array at arbitrary position

2016-04-06 Thread Teodor Sigaev
Thank you, pushed with Petr's notice Dmitry Dolgov wrote: On 6 April 2016 at 03:29, Andrew Dunstan > wrote: Yeah, keeping it but rejecting update of an existing key is my preference too. cheers andrew Yes, it sounds quite

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 15:17, Andres Freund wrote: > On 2016-04-06 14:30:21 +0100, Simon Riggs wrote: > > On 6 April 2016 at 14:15, Craig Ringer wrote: > > ... > > > > Nice summary > > > > Failover slots are optional. And they work on master. > > > >

Re: [HACKERS] Truncating/vacuuming relations on full tablespaces

2016-04-06 Thread Robert Haas
On Wed, Apr 6, 2016 at 3:32 AM, Asif Naeem wrote: >> Oh, I see. I think it's probably not a good idea to skip truncating >> those maps, but perhaps the option could be defined as making no new >> entries, rather than ignoring them altogether, so that they never >> grow. It

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-06 Thread Tom Lane
David Rowley writes: > In the last patch I failed to notice that there's an alternative > expected results file for one of the regression tests. > The attached patch includes the fix to update that file to match the > new expected EXPLAIN output. Starting to look at

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On 2016-04-06 16:49:17 +0100, Simon Riggs wrote: > Perhaps easy to solve, but how do we test it is solved? Maybe something like -- drain pg_logical_slot_get_changes(...); -- generate message in different database, to ensure it's not processed -- in this database \c template1 SELECT

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 15:29, Andres Freund wrote: > On 2016-04-06 10:24:52 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2016-04-06 10:15:59 -0400, Tom Lane wrote: > > >> Well, that's something worth thinking about. I assume that > > >>

Re: [HACKERS] [CommitFest App] Feature request -- review e-mail additions

2016-04-06 Thread Magnus Hagander
On Wed, Apr 6, 2016 at 2:11 PM, Magnus Hagander wrote: > On Wed, Mar 30, 2016 at 7:47 PM, Alvaro Herrera > wrote: > >> José Luis Tallón wrote: >> > > * Auto-CC the patch author on this e-mail >> > I guess this should speed up reactions / make

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-06 Thread Anastasia Lubennikova
06.04.2016 16:15, Anastasia Lubennikova : 06.04.2016 03:05, Peter Geoghegan: * There is some stray whitespace within RelationGetIndexAttrBitmap(). I think you should have updated it with code, though. I don't think it's necessary for HOT updates to work, but I think it could be necessary so

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Wed, Apr 6, 2016 at 11:14 PM, Amit Kapila wrote: > On Wed, Apr 6, 2016 at 7:03 PM, Fujii Masao wrote: >> >> On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila >> wrote: >> > >> >> BTW, we can move SyncRepUpdateConfig() just

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On 2016-04-06 10:24:52 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-04-06 10:15:59 -0400, Tom Lane wrote: > >> Well, that's something worth thinking about. I assume that > >> pg_logical_slot_get_changes could be executed in a database different from > >> the one

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On 2016-04-06 16:20:29 +0200, Andres Freund wrote: > On 2016-04-06 10:15:59 -0400, Tom Lane wrote: > > > In some ways it seems like the argument to pg_logical_emit_message(...) > > > should > > > be 'bytea'. That'd be much more convenient for application use. But then > > > it's a pain when using

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Tom Lane
Andres Freund writes: > On 2016-04-06 10:15:59 -0400, Tom Lane wrote: >> Well, that's something worth thinking about. I assume that >> pg_logical_slot_get_changes could be executed in a database different from >> the one where a change was originated? > You can execute it,

Re: [HACKERS] [PATCH] Phrase search ported to 9.6

2016-04-06 Thread Dmitry Ivanov
> > It seems that everything is settled now, so here's the patch introducing > > the '<->' and '' operators. I've made the necessary changes to docs & > > regression tests. > > I noticed that I had accidently trimmed whitespaces in docs, this is a > better one. After a brief but reasonable

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On 2016-04-06 10:15:59 -0400, Tom Lane wrote: > > In some ways it seems like the argument to pg_logical_emit_message(...) > > should > > be 'bytea'. That'd be much more convenient for application use. But then > > it's a pain when using it via the text-format SQL interface calls, where > > we've

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Andres Freund
On 2016-04-06 14:30:21 +0100, Simon Riggs wrote: > On 6 April 2016 at 14:15, Craig Ringer wrote: > ... > > Nice summary > > Failover slots are optional. And they work on master. > > While the other approach could also work, it will work later and still > require a slot

Re: [HACKERS] [PATCH] Phrase search ported to 9.6

2016-04-06 Thread Dmitry Ivanov
> > > It seems that everything is settled now, so here's the patch introducing > > > the '<->' and '' operators. I've made the necessary changes to docs & > > > regression tests. > > > > I noticed that I had accidently trimmed whitespaces in docs, this is a > > better one. > > After a brief but

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Tom Lane
Craig Ringer writes: > Interesting issue. Mainly because the "Å¥" char it complains about > (utf-8 0xc5 0xa5) is accepted in the SELECT that generates the record. Uh, no, actually it's the SELECT that's failing. > The regress script in question sets: > SET client_encoding

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Wed, Apr 6, 2016 at 7:03 PM, Fujii Masao wrote: > > On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila wrote: > > > >> BTW, we can move SyncRepUpdateConfig() just after ProcessConfigFile() > >> from pg_stat_get_wal_senders() and every backends always

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Dilip Kumar
On Wed, Apr 6, 2016 at 3:22 PM, Andres Freund wrote: > Which scale did you initialize with? I'm trying to reproduce the > workload on hydra as precisely as possible... > I tested with scale factor 300, shared buffer 8GB. My test script is attached with the mail

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Craig Ringer
Committed. https://commitfest.postgresql.org/9/468/ Buildfarm error: http://www.postgresql.org/message-id/cab7npqrod2mxqy_5+czjvhw0whrrz6p8jv_rsblcrxrtwlh...@mail.gmail.com Interesting issue. Mainly because the "ť" char it complains about (utf-8 0xc5 0xa5) is accepted in the SELECT that

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila wrote: > On Wed, Apr 6, 2016 at 11:17 AM, Fujii Masao wrote: >> >> On Tue, Apr 5, 2016 at 11:40 PM, Amit Kapila >> wrote: >> >> >> >> > 2. >> >> > pg_stat_get_wal_senders() >> >>

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 14:15, Craig Ringer wrote: ... Nice summary Failover slots are optional. And they work on master. While the other approach could also work, it will work later and still require a slot on the master. => I don't see why having Failover Slots in 9.6

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 14:00:20 +0100, Simon Riggs wrote: > On 6 April 2016 at 13:48, Michael Paquier wrote: > > > > > Yeah... We have reached a clear consensus about the way things should > > be done after quite a lot of discussions that has gone for a couple of > > months.

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 13:50:24 +0100, Simon Riggs wrote: > On 6 April 2016 at 13:27, Andres Freund wrote: > > > On 2016-04-06 13:11:40 +0100, Simon Riggs wrote: > > > On 6 April 2016 at 10:09, Andres Freund wrote: > > > > On 2016-04-06 10:04:42 +0100, Simon Riggs

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-06 Thread Anastasia Lubennikova
06.04.2016 03:05, Peter Geoghegan: On Tue, Apr 5, 2016 at 1:31 PM, Peter Geoghegan wrote: My new understanding: The extra "included" columns are stored in the index, but do not affect its sort order at all. They are no more part of the key than, say, the heap TID that the key

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Craig Ringer
A few thoughts on failover slots vs the alternative of pushing catalog_xmin up to the master via a replica's slot and creating independent slots on replicas. Failover slots: --- + Failover slots are very easy for applications. They "just work" and are transparent for failover. This is great

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 13:48, Michael Paquier wrote: > > Yeah... We have reached a clear consensus about the way things should > be done after quite a lot of discussions that has gone for a couple of > months. And Andres' design on the matter is what is getting approval >

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 13:27, Andres Freund wrote: > On 2016-04-06 13:11:40 +0100, Simon Riggs wrote: > > On 6 April 2016 at 10:09, Andres Freund wrote: > > > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > > > The issue there is that we continue to issue

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Michael Paquier
On Wed, Apr 6, 2016 at 9:27 PM, Andres Freund wrote: > On 2016-04-06 13:11:40 +0100, Simon Riggs wrote: >> On 6 April 2016 at 10:09, Andres Freund wrote: >> > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: >> > The issue there is that we continue to

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 13:11:40 +0100, Simon Riggs wrote: > On 6 April 2016 at 10:09, Andres Freund wrote: > > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > > The issue there is that we continue to issue checkpoints if the only > > activity since the last checkpoint was emitting a

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Robert Haas
On Wed, Apr 6, 2016 at 8:02 AM, Simon Riggs wrote: >> > We can, if you wish, revert this patch. If we do, we will have nothing, >> > since I object to the other patch(es). >> >> I don't think you have an absolute veto over other patches > > Huh? My understanding is I have

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 10:09, Andres Freund wrote: > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > > On 6 April 2016 at 09:45, Andres Freund wrote: > > > > > On 2016-04-06 09:18:54 +0100, Simon Riggs wrote: > > > > Rather than take that option, I went to

Re: [HACKERS] [CommitFest App] Feature request -- review e-mail additions

2016-04-06 Thread Magnus Hagander
On Wed, Mar 30, 2016 at 7:47 PM, Alvaro Herrera wrote: > José Luis Tallón wrote: > > > Just wanted to suggest two minor mods to the review e-mails > > auto-generated by the app: > > > > * Prepend a [review] tag to the e-mail subject > > ... so that e-mails sent

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 12:24, Robert Haas wrote: > On Wed, Apr 6, 2016 at 4:18 AM, Simon Riggs wrote: > >> FWIW, I vote also for reverting this patch. This has been committed > >> without any real discussions.. > > > > Michael, its a shame to hear you

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Wed, Apr 6, 2016 at 11:17 AM, Fujii Masao wrote: > > On Tue, Apr 5, 2016 at 11:40 PM, Amit Kapila wrote: > >> > >> > 2. > >> > pg_stat_get_wal_senders() > >> > { > >> > .. > >> > /* > >> > ! * Allocate and update the config data of synchronous

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Craig Ringer
On 6 April 2016 at 17:43, Simon Riggs wrote: > On 25 January 2016 at 14:25, Craig Ringer wrote: > > >> I'd like to get failover slots in place for 9.6 since the're fairly >> self-contained and meet an immediate need: allowing replication using slots

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Robert Haas
On Wed, Apr 6, 2016 at 4:18 AM, Simon Riggs wrote: >> FWIW, I vote also for reverting this patch. This has been committed >> without any real discussions.. > > Michael, its a shame to hear you say that, so let me give full context. > > The patches under review in the CF are

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Stephen Frost
Robbie, Just an initial pass over the patch. * Robbie Harwood (rharw...@redhat.com) wrote: > Here's v12, both here and on my github: > https://github.com/frozencemetery/postgres/tree/feature/gssencrypt12 I've started taking a look at this as it's a capability I've wanted us to support for a

[HACKERS] Detrimental performance impact of ringbuffers on performance

2016-04-06 Thread Andres Freund
Hi, While benchmarking on hydra (c.f. http://archives.postgresql.org/message-id/20160406104352.5bn3ehkcsceja65c%40alap3.anarazel.de), which has quite slow IO, I was once more annoyed by how incredibly long the vacuum at the the end of a pgbench -i takes. The issue is that, even for an entirely

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Andres Freund
On 2016-04-06 11:52:28 +0200, Andres Freund wrote: > Hi, > > On 2016-04-03 16:47:49 +0530, Dilip Kumar wrote: > > > Summary Of the Run: > > - > > 1. Throughout one run if we observe TPS every 30 seconds its stable in one > > run. > > 2. With Head 64 client run vary

Re: [HACKERS] Yet another small patch - reorderbuffer.c:1099

2016-04-06 Thread Andres Freund
On 2016-04-05 11:38:27 -0300, Alvaro Herrera wrote: > IMO the code is wrong. I'm a bit confused how an intentionally duplicated block makes code wrong... But whatever, I found it to be clerarer that way, but apparently I'm alone. > The current arrangement looks bizantine to me, for no reason.

Re: [HACKERS] Yet another small patch - reorderbuffer.c:1099

2016-04-06 Thread Aleksander Alekseev
> > IMO the code is wrong. There should be a single block comment > > saying something like "Remove the node from its containing list. > > In the FOO case, the list corresponds to BAR and therefore we > > delete it because BAZ. In the QUUX case the list is PLUGH and we > > delete in because

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Andres Freund
Hi, On 2016-04-03 16:47:49 +0530, Dilip Kumar wrote: > Summary Of the Run: > - > 1. Throughout one run if we observe TPS every 30 seconds its stable in one > run. > 2. With Head 64 client run vary between ~250,000 to ~45. you can see > below results. > > run1:

Re: [HACKERS] Speedup twophase transactions

2016-04-06 Thread Stas Kelvich
> On Apr 2, 2016, at 3:14 AM, Michael Paquier wrote: > > On Fri, Apr 1, 2016 at 10:53 PM, Stas Kelvich > wrote: >> I wrote: >>> While testing the patch, I found a bug in the recovery conflict code >>> path. You can do the following to

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Simon Riggs
On 25 January 2016 at 14:25, Craig Ringer wrote: > I'd like to get failover slots in place for 9.6 since the're fairly > self-contained and meet an immediate need: allowing replication using slots > (physical or logical) to follow a failover event. > I'm a bit confused

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Andres Freund
On 2016-04-05 12:56:46 +0530, Dilip Kumar wrote: > On Mon, Apr 4, 2016 at 2:28 PM, Andres Freund wrote: > > > Hm, interesting. I suspect that's because of the missing backoff in my > > experimental patch. If you apply the attached patch ontop of that > > (requires

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Alexander Korotkov
On Tue, Apr 5, 2016 at 5:45 PM, Andres Freund wrote: > On 2016-04-05 17:36:49 +0300, Alexander Korotkov wrote: > > Could the reason be that we're increasing concurrency for LWLock state > > atomic variable by placing queue spinlock there? > > Don't think so, it's the same

Re: [HACKERS] large regression for parallel COPY

2016-04-06 Thread Andres Freund
On 2016-04-05 17:12:11 -0400, Robert Haas wrote: > On Wed, Mar 30, 2016 at 4:10 PM, Andres Freund wrote: > > Indeed. On SSDs I see about a 25-35% gain, on HDDs about 5%. If I > > increase the size of backend_flush_after to 64 (like it's for bgwriter) > > I however do get about

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 09:23, Fujii Masao wrote: > Okay, I pushed the patch! > Many thanks to all involved in the development of this feature! > Very good. I think the description in the commit message that we don't support "quorum commit" is sufficient to cover my concerns

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Michael Paquier
On Wed, Apr 6, 2016 at 5:23 PM, Fujii Masao wrote: > Okay, I pushed the patch! > Many thanks to all involved in the development of this feature! I think that I am crying... Really cool to see this milestone accomplished. -- Michael -- Sent via pgsql-hackers mailing

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > On 6 April 2016 at 09:45, Andres Freund wrote: > > > On 2016-04-06 09:18:54 +0100, Simon Riggs wrote: > > > Rather than take that option, I went to the trouble of writing a patch > > that > > > does the same thing but

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 09:45, Andres Freund wrote: > On 2016-04-06 09:18:54 +0100, Simon Riggs wrote: > > Rather than take that option, I went to the trouble of writing a patch > that > > does the same thing but simpler, less invasive and more maintainable. > > Primarily, I did

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 09:18:54 +0100, Simon Riggs wrote: > Rather than take that option, I went to the trouble of writing a patch that > does the same thing but simpler, less invasive and more maintainable. > Primarily, I did that for you, to avoid you having wasted your time and to > allow you to

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Wed, Apr 6, 2016 at 5:07 PM, Fujii Masao wrote: > On Wed, Apr 6, 2016 at 4:08 PM, Michael Paquier > wrote: >> On Wed, Apr 6, 2016 at 3:29 PM, Fujii Masao wrote: >>> On Wed, Apr 6, 2016 at 2:18 PM, Kyotaro HORIGUCHI >>>

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 5 April 2016 at 01:18, Michael Paquier wrote: > On Mon, Apr 4, 2016 at 4:50 PM, Andres Freund wrote: > > On 2016-04-04 08:44:47 +0100, Simon Riggs wrote: > >> That patch does exactly the same thing as the patch you prefer, just > >> does it

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Kyotaro HORIGUCHI
Sorry, my code was wrong in the case that the total numer of synchronous standby exceeds required number and the wansender is at priority 1. Sorry for the noise. At Wed, 06 Apr 2016 17:01:51 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Wed, Apr 6, 2016 at 5:01 PM, Kyotaro HORIGUCHI wrote: > At Wed, 6 Apr 2016 15:29:12 +0900, Fujii Masao wrote > in

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Wed, Apr 6, 2016 at 4:08 PM, Michael Paquier wrote: > On Wed, Apr 6, 2016 at 3:29 PM, Fujii Masao wrote: >> On Wed, Apr 6, 2016 at 2:18 PM, Kyotaro HORIGUCHI >> wrote: >>> At Tue, 5 Apr 2016 20:17:21 +0900,

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Michael Paquier
On Wed, Apr 6, 2016 at 4:08 PM, Michael Paquier wrote: > Here are few things I have noticed: > + for (i = 0; i < max_wal_senders; i++) > + { > + walsnd = >walsnds[i]; > No volatile pointer to prevent code reordering? > > */ > typedef struct WalSnd > { > +

  1   2   >