Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Thu, Apr 7, 2016 at 11:56 AM, Fujii Masao wrote: > > On Thu, Apr 7, 2016 at 2:48 PM, Amit Kapila wrote: > > On Thu, Apr 7, 2016 at 10:02 AM, Fujii Masao wrote: > >> > >> On Thu, Apr 7, 2016 at 1:22 PM, Amit Kapila > >> wrote: > >> > > >> > But for that, I think we don't need to do anything e

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-04-06 Thread Karl O. Pinc
On Thu, 7 Apr 2016 01:13:51 -0500 "Karl O. Pinc" wrote: > On Wed, 6 Apr 2016 23:37:09 -0500 > "Karl O. Pinc" wrote: > > > On Wed, 6 Apr 2016 22:26:13 -0500 > > "Karl O. Pinc" wrote: > > > On Wed, 23 Mar 2016 23:22:26 +0100 > > > Gilles Darold wrote: > > > > > > > Thanks for the reminde

Re: [HACKERS] Speedup twophase transactions

2016-04-06 Thread Michael Paquier
On Wed, Apr 6, 2016 at 6:47 PM, Stas Kelvich wrote: >> On Apr 2, 2016, at 3:14 AM, Michael Paquier >> wrote: >> >> On Fri, Apr 1, 2016 at 10:53 PM, Stas Kelvich >> wrote: >>> I wrote: While testing the patch, I found a bug in the recovery conflict code path. You can do the following

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Thu, Apr 7, 2016 at 2:48 PM, Amit Kapila wrote: > On Thu, Apr 7, 2016 at 10:02 AM, Fujii Masao wrote: >> >> On Thu, Apr 7, 2016 at 1:22 PM, Amit Kapila >> wrote: >> > >> > But for that, I think we don't need to do anything extra. I mean >> > write_nondefault_variables() will automatically wr

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-04-06 Thread Karl O. Pinc
On Wed, 6 Apr 2016 23:37:09 -0500 "Karl O. Pinc" wrote: > On Wed, 6 Apr 2016 22:26:13 -0500 > "Karl O. Pinc" wrote: > > On Wed, 23 Mar 2016 23:22:26 +0100 > > Gilles Darold wrote: > > > > > Thanks for the reminder, here is the v3 of the patch after a > > > deeper review and testing. It is no

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Thu, Apr 7, 2016 at 10:02 AM, Fujii Masao wrote: > > On Thu, Apr 7, 2016 at 1:22 PM, Amit Kapila wrote: > > > > But for that, I think we don't need to do anything extra. I mean > > write_nondefault_variables() will automatically write the non-default value > > of variable and then during back

Re: [HACKERS] pg_ctl promote wait

2016-04-06 Thread Michael Paquier
On Wed, Mar 23, 2016 at 1:47 AM, David Steele wrote: > On 3/16/16 12:19 PM, David Steele wrote: >> Hi Peter, >> >> On 3/9/16 3:08 PM, Michael Paquier wrote: >> >>> Here are some comments about 0002 >> <...> >>> I think that we had better do something like the attached first. >>> Thoughts? >> >> It

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-06 Thread Andres Freund
Hi, On 2016-04-07 09:14:00 +0530, Amit Kapila wrote: > On Sat, Apr 2, 2016 at 5:25 PM, Amit Kapila wrote: > I have ran exactly same test on intel x86 m/c and the results are as below: Thanks for running these tests! > Client Count/Patch_ver (tps) 2 128 256 > HEAD – Commit 2143f5e1 2832 35001 26

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-04-06 Thread Karl O. Pinc
On Wed, 6 Apr 2016 22:26:13 -0500 "Karl O. Pinc" wrote: > On Wed, 23 Mar 2016 23:22:26 +0100 > Gilles Darold wrote: > > > Thanks for the reminder, here is the v3 of the patch after a deeper > > review and testing. It is now registered to the next commit fest > > under the System Administration t

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Thu, Apr 7, 2016 at 1:22 PM, Amit Kapila wrote: > On Wed, Apr 6, 2016 at 8:11 PM, Fujii Masao wrote: >> >> On Wed, Apr 6, 2016 at 11:14 PM, Amit Kapila >> wrote: >> > On Wed, Apr 6, 2016 at 7:03 PM, Fujii Masao >> > wrote: >> >> >> >> On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila >> >> wrote:

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Wed, Apr 6, 2016 at 8:11 PM, Fujii Masao wrote: > > On Wed, Apr 6, 2016 at 11:14 PM, Amit Kapila wrote: > > On Wed, Apr 6, 2016 at 7:03 PM, Fujii Masao wrote: > >> > >> On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila > >> wrote: > >> > > >> >> BTW, we can move SyncRepUpdateConfig() just after Pr

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On April 7, 2016 2:26:41 AM GMT+02:00, Michael Paquier wrote: >On Thu, Apr 7, 2016 at 12:55 AM, Andres Freund >wrote: >> On 2016-04-06 16:49:17 +0100, Simon Riggs wrote: >>> Perhaps easy to solve, but how do we test it is solved? >> >> Maybe something like >> >> -- drain >> pg_logical_slot_get

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-04-06 Thread Amit Kapila
On Sat, Apr 2, 2016 at 5:25 PM, Amit Kapila wrote: > On Thu, Mar 31, 2016 at 3:48 PM, Andres Freund wrote: > > Here is the performance data (configuration of machine used to perform > this test is mentioned at end of mail): > > Non-default parameters > > max_

[HACKERS] Re: [COMMITTERS] pgsql: Use GRANT system to manage access to sensitive functions

2016-04-06 Thread Stephen Frost
* Fujii Masao (masao.fu...@gmail.com) wrote: > On Thu, Apr 7, 2016 at 10:45 AM, Stephen Frost wrote: > > Use GRANT system to manage access to sensitive functions > > > > Now that pg_dump will properly dump out any ACL changes made to > > functions which exist in pg_catalog, switch to using the GRA

Re: [HACKERS] Patch to implement pg_current_logfile() function

2016-04-06 Thread Karl O. Pinc
Hi Gilles, On Wed, 23 Mar 2016 23:22:26 +0100 Gilles Darold wrote: > Thanks for the reminder, here is the v3 of the patch after a deeper > review and testing. It is now registered to the next commit fest under > the System Administration topic. I am going to try reviewing your patch. I don't f

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-04-06 Thread Noah Misch
On Wed, Apr 06, 2016 at 09:17:22AM +0200, Magnus Hagander wrote: > On Wed, Apr 6, 2016 at 6:42 AM, Noah Misch wrote: > > On Tue, Apr 05, 2016 at 08:15:16PM +0200, Magnus Hagander wrote: > > > I've pushed this version, and also added the item from the Brussels > > > developer meeting to actually re

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Robert Haas
On Wed, Apr 6, 2016 at 10:04 AM, Dilip Kumar wrote: > On Wed, Apr 6, 2016 at 3:22 PM, Andres Freund wrote: >> Which scale did you initialize with? I'm trying to reproduce the >> workload on hydra as precisely as possible... > > I tested with scale factor 300, shared buffer 8GB. > > My test script

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-06 Thread David Rowley
On 7 April 2016 at 08:01, David Rowley wrote: > On 7 April 2016 at 04:05, Tom Lane wrote: >> Starting to look at this again. I wonder, now that you have the generic >> caching mechanism for remembering whether join inner sides have been >> proven unique, is it still worth having the is_unique_jo

Re: [HACKERS] postgres_fdw : altering foreign table not invalidating prepare statement execution plan.

2016-04-06 Thread Kyotaro HORIGUCHI
Hi, At Tue, 5 Apr 2016 19:46:04 +0900, Amit Langote wrote in <5703976c.30...@lab.ntt.co.jp> > On 2016/04/05 18:44, Kyotaro HORIGUCHI wrote: > > At Tue, 5 Apr 2016 14:24:52 +0900, Amit Langote wrote: > > With this patch, making any change on foreign servers or user > > mappings corresponding to e

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Michael Paquier
On Thu, Apr 7, 2016 at 12:55 AM, Andres Freund wrote: > On 2016-04-06 16:49:17 +0100, Simon Riggs wrote: >> Perhaps easy to solve, but how do we test it is solved? > > Maybe something like > > -- drain > pg_logical_slot_get_changes(...); > -- generate message in different database, to ensure it's

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Tom Lane
Robbie Harwood writes: > Tom Lane writes: >> Wait a second. So the initial connection-request packet is necessarily >> unencrypted under this scheme? > Yes, by necessity. The username must be sent in the clear, even if only > as part of the GSSAPI handshake (i.e., the GSSAPI username will appe

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2016-04-06 Thread Tomas Vondra
Hi, attached is the patch split into two parts, as proposed by Simon. 0001 just adds the stuff to relcache, 0002 actually uses it for estimation. On 04/04/2016 12:03 PM, Amit Langote wrote: On 2016/04/04 17:25, Simon Riggs wrote: The rel cache code you're adding uses a flag called "rd_fkeyva

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Michael Paquier
On Thu, Apr 7, 2016 at 7:44 AM, Michael Paquier wrote: > On Thu, Apr 7, 2016 at 6:11 AM, Petr Jelinek wrote: >> On 06/04/16 22:50, Andrew Dunstan wrote: >>> I have spent way too much time on this and don't have it working yet. >>> I'm setting up a sacrificial VM from scratch in a last ditch attem

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Petr Jelinek
On 07/04/16 00:50, Michael Paquier wrote: On Thu, Apr 7, 2016 at 7:44 AM, Michael Paquier wrote: On Thu, Apr 7, 2016 at 6:11 AM, Petr Jelinek wrote: On 06/04/16 22:50, Andrew Dunstan wrote: I have spent way too much time on this and don't have it working yet. I'm setting up a sacrificial VM

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Michael Paquier
On Thu, Apr 7, 2016 at 6:11 AM, Petr Jelinek wrote: > On 06/04/16 22:50, Andrew Dunstan wrote: >> I have spent way too much time on this and don't have it working yet. >> I'm setting up a sacrificial VM from scratch in a last ditch attempt to >> get it working. >> >> Things to note so far: >> >>

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2016-04-06 Thread Tom Lane
Pavel Stehule writes: > 1. We want this patch - it increase a functionality of autocomplete TBH, I do not think that is an agreed-to statement. I concur with Peter's comments upthread questioning how much use-case there is for interactive completion of IF (NOT) EXISTS. If it were a small and un

Re: [HACKERS] [patch] Proposal for \crosstabview in psql

2016-04-06 Thread Alvaro Herrera
I've been looking at this patch. First thing was to rebase on top of recent psql code restructuring; second, pgindent; third, reordered the code in crosstabview.c more sensibly (had to add prototypes). New version attached. Then I looked at the docs to try to figure out exactly how it works. I'm

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Robbie Harwood
Tom Lane writes: > Robbie Harwood writes: >> I need to flush this any time we might be doing encryption because it >> needs to be in a separate request to _secure_write() from what follows >> it. We don't know whether we should be doing encryption until >> connection parameters are parsed; to p

Re: [HACKERS] Combining Aggregates

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 22:28, David Rowley wrote: > On 7 April 2016 at 09:25, Simon Riggs wrote: > > On 5 April 2016 at 19:33, Robert Haas wrote: > > > >> > >> Committed 0002+0003 with those changes, some minor cosmetic stuff, and > >> of course the obligatory catversion bump. Oh, and fixed an OID

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Alvaro Herrera
Petr Jelinek wrote: > It's fun to set it up yes. I do have the machine with buildfarm client ready > still (although now also traveling so slightly complicated to get to it) but > I didn't activate it yet as I don't want it to just report failures forever. Maybe you should just activate it regard

Re: [HACKERS] Combining Aggregates

2016-04-06 Thread David Rowley
On 7 April 2016 at 09:25, Simon Riggs wrote: > On 5 April 2016 at 19:33, Robert Haas wrote: > >> >> Committed 0002+0003 with those changes, some minor cosmetic stuff, and >> of course the obligatory catversion bump. Oh, and fixed an OID >> conflict with the patch Magnus just committed. > > > Is

Re: [HACKERS] Combining Aggregates

2016-04-06 Thread Simon Riggs
On 5 April 2016 at 19:33, Robert Haas wrote: > Committed 0002+0003 with those changes, some minor cosmetic stuff, and > of course the obligatory catversion bump. Oh, and fixed an OID > conflict with the patch Magnus just committed. Is that everything now? I don't see anything about combining

Re: [HACKERS] Truncating/vacuuming relations on full tablespaces

2016-04-06 Thread Jim Nasby
On 4/6/16 11:06 AM, Robert Haas wrote: This is too late for 9.6 at this point and certainly requires discussion anyway, so please add it to the next CommitFest. If the goal here is to free up space via truncation when there's a real emergency, perhaps there's some other steps that should be ta

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Petr Jelinek
On 06/04/16 22:50, Andrew Dunstan wrote: On 03/29/2016 09:38 PM, Robert Haas wrote: On Tue, Mar 29, 2016 at 9:29 PM, Andrew Dunstan wrote: I am currently travelling, but my intention is to deal with the remaining patches when I'm back home this weekend, unless someone beats me to it. Cool.

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Tom Lane
Robbie Harwood writes: > I need to flush this any time we might be doing encryption because it > needs to be in a separate request to _secure_write() from what follows > it. We don't know whether we should be doing encryption until > connection parameters are parsed; to put it another way, > port

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-06 Thread David Rowley
On 7 April 2016 at 08:01, David Rowley wrote: > On 7 April 2016 at 04:05, Tom Lane wrote: >> Starting to look at this again. I wonder, now that you have the generic >> caching mechanism for remembering whether join inner sides have been >> proven unique, is it still worth having the is_unique_jo

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-06 Thread Peter Geoghegan
On Wed, Apr 6, 2016 at 1:50 PM, Peter Geoghegan wrote: > Personally, I like documenting assertions, and will sometimes write > assertions that the compiler could easily optimize away. Maybe going > *that* far is more a matter of personal style, but I think an > assertion about the new index tuple

Re: [HACKERS] VS 2015 support in src/tools/msvc

2016-04-06 Thread Andrew Dunstan
On 03/29/2016 09:38 PM, Robert Haas wrote: On Tue, Mar 29, 2016 at 9:29 PM, Andrew Dunstan wrote: I am currently travelling, but my intention is to deal with the remaining patches when I'm back home this weekend, unless someone beats me to it. Cool. Progress report: I have spent way too

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-06 Thread Peter Geoghegan
On Wed, Apr 6, 2016 at 6:15 AM, Anastasia Lubennikova wrote: >> * I would like to see index_reform_tuple() assert that the new, >> truncated index tuple is definitely <= the original (I worry about the >> 1/3 page restriction issue). Maybe you should also change the name of >> index_reform_tuple()

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2016-04-06 Thread Pavel Stehule
Hi 2016-04-04 7:58 GMT+02:00 Kyotaro HORIGUCHI : > Thank you for testing. That is a silly mistake, sorry. > > The attached is the fixed version. > > # Can I add a suffix to format-patche's output files? > > At Sat, 2 Apr 2016 07:18:32 +0200, Pavel Stehule > wrote in < > cafj8pradf3rmq3y33aer1c7w

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-06 Thread David Rowley
On 7 April 2016 at 04:05, Tom Lane wrote: > David Rowley writes: >> In the last patch I failed to notice that there's an alternative >> expected results file for one of the regression tests. >> The attached patch includes the fix to update that file to match the >> new expected EXPLAIN output. >

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Robbie Harwood
Stephen Frost writes: > Just an initial pass over the patch. Thanks! In the interest of brevity, if I haven't replied to something, I plan to fix it. >> /* >> - * Flush message so client will see it, except for AUTH_REQ_OK, which >> need >> - * not be sent until we are ready for

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Petr Jelinek
On 06/04/16 17:55, Andres Freund wrote: On 2016-04-06 16:49:17 +0100, Simon Riggs wrote: Perhaps easy to solve, but how do we test it is solved? Maybe something like -- drain pg_logical_slot_get_changes(...); -- generate message in different database, to ensure it's not processed -- in this d

Re: [HACKERS] Choosing parallel_degree

2016-04-06 Thread Julien Rouhaud
On 06/04/2016 07:38, Amit Kapila wrote: > On Tue, Apr 5, 2016 at 11:55 PM, Julien Rouhaud >> >> In alter_table.sgml, I didn't comment the lock level needed to modify >> parallel_degree since it requires an access exclusive lock for now. >> While thinking about it, I think it's safe to use a share u

Re: [HACKERS] insufficient qualification of some objects in dump files

2016-04-06 Thread Peter Eisentraut
On 04/05/2016 09:46 PM, Robert Haas wrote: On Fri, Mar 18, 2016 at 11:22 AM, Robert Haas wrote: On Fri, Mar 18, 2016 at 2:13 AM, Michael Paquier wrote: On Fri, Mar 18, 2016 at 5:38 AM, Tom Lane wrote: Given the lack of any other complaints about this, I'm okay with the approach as presented

Re: [HACKERS] GIN data corruption bug(s) in 9.6devel

2016-04-06 Thread Teodor Sigaev
I've tested the v2, v3 and v3.1 of the patch, to see if there are any differences. The v2 no longer applies, so I tested it on ee943004. The following table shows the total duration of the data load, and also sizes of the two GIN indexes. duration (sec) subject body

Re: [HACKERS] [PATH] Jsonb, insert a new value into an array at arbitrary position

2016-04-06 Thread Teodor Sigaev
Thank you, pushed with Petr's notice Dmitry Dolgov wrote: On 6 April 2016 at 03:29, Andrew Dunstan mailto:and...@dunslane.net>> wrote: Yeah, keeping it but rejecting update of an existing key is my preference too. cheers andrew Yes, it sounds quite reasonable. Here is a new v

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 15:17, Andres Freund wrote: > On 2016-04-06 14:30:21 +0100, Simon Riggs wrote: > > On 6 April 2016 at 14:15, Craig Ringer wrote: > > ... > > > > Nice summary > > > > Failover slots are optional. And they work on master. > > > > While the other approach could also work, it wil

Re: [HACKERS] Truncating/vacuuming relations on full tablespaces

2016-04-06 Thread Robert Haas
On Wed, Apr 6, 2016 at 3:32 AM, Asif Naeem wrote: >> Oh, I see. I think it's probably not a good idea to skip truncating >> those maps, but perhaps the option could be defined as making no new >> entries, rather than ignoring them altogether, so that they never >> grow. It seems that both of tho

Re: [HACKERS] Performance improvement for joins where outer side is unique

2016-04-06 Thread Tom Lane
David Rowley writes: > In the last patch I failed to notice that there's an alternative > expected results file for one of the regression tests. > The attached patch includes the fix to update that file to match the > new expected EXPLAIN output. Starting to look at this again. I wonder, now tha

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On 2016-04-06 16:49:17 +0100, Simon Riggs wrote: > Perhaps easy to solve, but how do we test it is solved? Maybe something like -- drain pg_logical_slot_get_changes(...); -- generate message in different database, to ensure it's not processed -- in this database \c template1 SELECT pg_logical_emi

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 15:29, Andres Freund wrote: > On 2016-04-06 10:24:52 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2016-04-06 10:15:59 -0400, Tom Lane wrote: > > >> Well, that's something worth thinking about. I assume that > > >> pg_logical_slot_get_changes could be executed in

Re: [HACKERS] [CommitFest App] Feature request -- review e-mail additions

2016-04-06 Thread Magnus Hagander
On Wed, Apr 6, 2016 at 2:11 PM, Magnus Hagander wrote: > On Wed, Mar 30, 2016 at 7:47 PM, Alvaro Herrera > wrote: > >> José Luis Tallón wrote: >> > > * Auto-CC the patch author on this e-mail >> > I guess this should speed up reactions / make communication a bit >> more >> > fluid. >> >> Yes

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-06 Thread Anastasia Lubennikova
06.04.2016 16:15, Anastasia Lubennikova : 06.04.2016 03:05, Peter Geoghegan: * There is some stray whitespace within RelationGetIndexAttrBitmap(). I think you should have updated it with code, though. I don't think it's necessary for HOT updates to work, but I think it could be necessary so that

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Wed, Apr 6, 2016 at 11:14 PM, Amit Kapila wrote: > On Wed, Apr 6, 2016 at 7:03 PM, Fujii Masao wrote: >> >> On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila >> wrote: >> > >> >> BTW, we can move SyncRepUpdateConfig() just after ProcessConfigFile() >> >> from pg_stat_get_wal_senders() and every bac

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On 2016-04-06 10:24:52 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-04-06 10:15:59 -0400, Tom Lane wrote: > >> Well, that's something worth thinking about. I assume that > >> pg_logical_slot_get_changes could be executed in a database different from > >> the one where a change was o

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On 2016-04-06 16:20:29 +0200, Andres Freund wrote: > On 2016-04-06 10:15:59 -0400, Tom Lane wrote: > > > In some ways it seems like the argument to pg_logical_emit_message(...) > > > should > > > be 'bytea'. That'd be much more convenient for application use. But then > > > it's a pain when using

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Tom Lane
Andres Freund writes: > On 2016-04-06 10:15:59 -0400, Tom Lane wrote: >> Well, that's something worth thinking about. I assume that >> pg_logical_slot_get_changes could be executed in a database different from >> the one where a change was originated? > You can execute it, but you'll get an erro

Re: [HACKERS] [PATCH] Phrase search ported to 9.6

2016-04-06 Thread Dmitry Ivanov
> > It seems that everything is settled now, so here's the patch introducing > > the '<->' and '' operators. I've made the necessary changes to docs & > > regression tests. > > I noticed that I had accidently trimmed whitespaces in docs, this is a > better one. After a brief but reasonable discus

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Andres Freund
On 2016-04-06 10:15:59 -0400, Tom Lane wrote: > > In some ways it seems like the argument to pg_logical_emit_message(...) > > should > > be 'bytea'. That'd be much more convenient for application use. But then > > it's a pain when using it via the text-format SQL interface calls, where > > we've g

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Andres Freund
On 2016-04-06 14:30:21 +0100, Simon Riggs wrote: > On 6 April 2016 at 14:15, Craig Ringer wrote: > ... > > Nice summary > > Failover slots are optional. And they work on master. > > While the other approach could also work, it will work later and still > require a slot on the master. > > > =

Re: [HACKERS] [PATCH] Phrase search ported to 9.6

2016-04-06 Thread Dmitry Ivanov
> > > It seems that everything is settled now, so here's the patch introducing > > > the '<->' and '' operators. I've made the necessary changes to docs & > > > regression tests. > > > > I noticed that I had accidently trimmed whitespaces in docs, this is a > > better one. > > After a brief but r

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Tom Lane
Craig Ringer writes: > Interesting issue. Mainly because the "Å¥" char it complains about > (utf-8 0xc5 0xa5) is accepted in the SELECT that generates the record. Uh, no, actually it's the SELECT that's failing. > The regress script in question sets: > SET client_encoding = 'utf8'; > but we're a

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Wed, Apr 6, 2016 at 7:03 PM, Fujii Masao wrote: > > On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila wrote: > > > >> BTW, we can move SyncRepUpdateConfig() just after ProcessConfigFile() > >> from pg_stat_get_wal_senders() and every backends always parse the value > >> of s_s_names when the setting

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Dilip Kumar
On Wed, Apr 6, 2016 at 3:22 PM, Andres Freund wrote: > Which scale did you initialize with? I'm trying to reproduce the > workload on hydra as precisely as possible... > I tested with scale factor 300, shared buffer 8GB. My test script is attached with the mail (perf_pgbench_ro.sh). I have don

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-06 Thread Craig Ringer
Committed. https://commitfest.postgresql.org/9/468/ Buildfarm error: http://www.postgresql.org/message-id/cab7npqrod2mxqy_5+czjvhw0whrrz6p8jv_rsblcrxrtwlh...@mail.gmail.com Interesting issue. Mainly because the "ť" char it complains about (utf-8 0xc5 0xa5) is accepted in the SELECT that generates

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Wed, Apr 6, 2016 at 8:59 PM, Amit Kapila wrote: > On Wed, Apr 6, 2016 at 11:17 AM, Fujii Masao wrote: >> >> On Tue, Apr 5, 2016 at 11:40 PM, Amit Kapila >> wrote: >> >> >> >> > 2. >> >> > pg_stat_get_wal_senders() >> >> > { >> >> > .. >> >> > /* >> >> > ! * Allocate and update the config data

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 14:15, Craig Ringer wrote: ... Nice summary Failover slots are optional. And they work on master. While the other approach could also work, it will work later and still require a slot on the master. => I don't see why having Failover Slots in 9.6 would prevent us from havi

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 14:00:20 +0100, Simon Riggs wrote: > On 6 April 2016 at 13:48, Michael Paquier wrote: > > > > > Yeah... We have reached a clear consensus about the way things should > > be done after quite a lot of discussions that has gone for a couple of > > months. And Andres' design on the matt

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 13:50:24 +0100, Simon Riggs wrote: > On 6 April 2016 at 13:27, Andres Freund wrote: > > > On 2016-04-06 13:11:40 +0100, Simon Riggs wrote: > > > On 6 April 2016 at 10:09, Andres Freund wrote: > > > > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > > > > The issue there is that w

Re: [HACKERS] WIP: Covering + unique indexes.

2016-04-06 Thread Anastasia Lubennikova
06.04.2016 03:05, Peter Geoghegan: On Tue, Apr 5, 2016 at 1:31 PM, Peter Geoghegan wrote: My new understanding: The extra "included" columns are stored in the index, but do not affect its sort order at all. They are no more part of the key than, say, the heap TID that the key points to. They ar

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Craig Ringer
A few thoughts on failover slots vs the alternative of pushing catalog_xmin up to the master via a replica's slot and creating independent slots on replicas. Failover slots: --- + Failover slots are very easy for applications. They "just work" and are transparent for failover. This is great espe

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 13:48, Michael Paquier wrote: > > Yeah... We have reached a clear consensus about the way things should > be done after quite a lot of discussions that has gone for a couple of > months. And Andres' design on the matter is what is getting approval > from everybody who has worke

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 13:27, Andres Freund wrote: > On 2016-04-06 13:11:40 +0100, Simon Riggs wrote: > > On 6 April 2016 at 10:09, Andres Freund wrote: > > > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > > > The issue there is that we continue to issue checkpoints if the only > > > activity si

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Michael Paquier
On Wed, Apr 6, 2016 at 9:27 PM, Andres Freund wrote: > On 2016-04-06 13:11:40 +0100, Simon Riggs wrote: >> On 6 April 2016 at 10:09, Andres Freund wrote: >> > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: >> > The issue there is that we continue to issue checkpoints if the only >> > activity s

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 13:11:40 +0100, Simon Riggs wrote: > On 6 April 2016 at 10:09, Andres Freund wrote: > > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > > The issue there is that we continue to issue checkpoints if the only > > activity since the last checkpoint was emitting a standby > > snapshot

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Robert Haas
On Wed, Apr 6, 2016 at 8:02 AM, Simon Riggs wrote: >> > We can, if you wish, revert this patch. If we do, we will have nothing, >> > since I object to the other patch(es). >> >> I don't think you have an absolute veto over other patches > > Huh? My understanding is I have the same powers as other

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 10:09, Andres Freund wrote: > On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > > On 6 April 2016 at 09:45, Andres Freund wrote: > > > > > On 2016-04-06 09:18:54 +0100, Simon Riggs wrote: > > > > Rather than take that option, I went to the trouble of writing a > patch > > > t

Re: [HACKERS] [CommitFest App] Feature request -- review e-mail additions

2016-04-06 Thread Magnus Hagander
On Wed, Mar 30, 2016 at 7:47 PM, Alvaro Herrera wrote: > José Luis Tallón wrote: > > > Just wanted to suggest two minor mods to the review e-mails > > auto-generated by the app: > > > > * Prepend a [review] tag to the e-mail subject > > ... so that e-mails sent to -hackers will read " [H

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 12:24, Robert Haas wrote: > On Wed, Apr 6, 2016 at 4:18 AM, Simon Riggs wrote: > >> FWIW, I vote also for reverting this patch. This has been committed > >> without any real discussions.. > > > > Michael, its a shame to hear you say that, so let me give full context. > > > > T

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Amit Kapila
On Wed, Apr 6, 2016 at 11:17 AM, Fujii Masao wrote: > > On Tue, Apr 5, 2016 at 11:40 PM, Amit Kapila wrote: > >> > >> > 2. > >> > pg_stat_get_wal_senders() > >> > { > >> > .. > >> > /* > >> > ! * Allocate and update the config data of synchronous replication, > >> > ! * and then get the currently

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Craig Ringer
On 6 April 2016 at 17:43, Simon Riggs wrote: > On 25 January 2016 at 14:25, Craig Ringer wrote: > > >> I'd like to get failover slots in place for 9.6 since the're fairly >> self-contained and meet an immediate need: allowing replication using slots >> (physical or logical) to follow a failover

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Robert Haas
On Wed, Apr 6, 2016 at 4:18 AM, Simon Riggs wrote: >> FWIW, I vote also for reverting this patch. This has been committed >> without any real discussions.. > > Michael, its a shame to hear you say that, so let me give full context. > > The patches under review in the CF are too invasive and not wo

Re: [HACKERS] [PATCH v12] GSSAPI encryption support

2016-04-06 Thread Stephen Frost
Robbie, Just an initial pass over the patch. * Robbie Harwood (rharw...@redhat.com) wrote: > Here's v12, both here and on my github: > https://github.com/frozencemetery/postgres/tree/feature/gssencrypt12 I've started taking a look at this as it's a capability I've wanted us to support for a *lon

[HACKERS] Detrimental performance impact of ringbuffers on performance

2016-04-06 Thread Andres Freund
Hi, While benchmarking on hydra (c.f. http://archives.postgresql.org/message-id/20160406104352.5bn3ehkcsceja65c%40alap3.anarazel.de), which has quite slow IO, I was once more annoyed by how incredibly long the vacuum at the the end of a pgbench -i takes. The issue is that, even for an entirely s

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Andres Freund
On 2016-04-06 11:52:28 +0200, Andres Freund wrote: > Hi, > > On 2016-04-03 16:47:49 +0530, Dilip Kumar wrote: > > > Summary Of the Run: > > - > > 1. Throughout one run if we observe TPS every 30 seconds its stable in one > > run. > > 2. With Head 64 client run vary bet

Re: [HACKERS] Yet another small patch - reorderbuffer.c:1099

2016-04-06 Thread Andres Freund
On 2016-04-05 11:38:27 -0300, Alvaro Herrera wrote: > IMO the code is wrong. I'm a bit confused how an intentionally duplicated block makes code wrong... But whatever, I found it to be clerarer that way, but apparently I'm alone. > The current arrangement looks bizantine to me, for no reason.

Re: [HACKERS] Yet another small patch - reorderbuffer.c:1099

2016-04-06 Thread Aleksander Alekseev
> > IMO the code is wrong. There should be a single block comment > > saying something like "Remove the node from its containing list. > > In the FOO case, the list corresponds to BAR and therefore we > > delete it because BAZ. In the QUUX case the list is PLUGH and we > > delete in because THUD.

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Andres Freund
Hi, On 2016-04-03 16:47:49 +0530, Dilip Kumar wrote: > Summary Of the Run: > - > 1. Throughout one run if we observe TPS every 30 seconds its stable in one > run. > 2. With Head 64 client run vary between ~250,000 to ~45. you can see > below results. > > run1: 434

Re: [HACKERS] Speedup twophase transactions

2016-04-06 Thread Stas Kelvich
> On Apr 2, 2016, at 3:14 AM, Michael Paquier wrote: > > On Fri, Apr 1, 2016 at 10:53 PM, Stas Kelvich > wrote: >> I wrote: >>> While testing the patch, I found a bug in the recovery conflict code >>> path. You can do the following to reproduce it: >>> 1) Start a master with a standby >>> 2) pr

Re: [HACKERS] WIP: Failover Slots

2016-04-06 Thread Simon Riggs
On 25 January 2016 at 14:25, Craig Ringer wrote: > I'd like to get failover slots in place for 9.6 since the're fairly > self-contained and meet an immediate need: allowing replication using slots > (physical or logical) to follow a failover event. > I'm a bit confused about this now. We seem

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Andres Freund
On 2016-04-05 12:56:46 +0530, Dilip Kumar wrote: > On Mon, Apr 4, 2016 at 2:28 PM, Andres Freund wrote: > > > Hm, interesting. I suspect that's because of the missing backoff in my > > experimental patch. If you apply the attached patch ontop of that > > (requires infrastructure from pinunpin), h

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-06 Thread Alexander Korotkov
On Tue, Apr 5, 2016 at 5:45 PM, Andres Freund wrote: > On 2016-04-05 17:36:49 +0300, Alexander Korotkov wrote: > > Could the reason be that we're increasing concurrency for LWLock state > > atomic variable by placing queue spinlock there? > > Don't think so, it's the same cache-line either way. >

Re: [HACKERS] large regression for parallel COPY

2016-04-06 Thread Andres Freund
On 2016-04-05 17:12:11 -0400, Robert Haas wrote: > On Wed, Mar 30, 2016 at 4:10 PM, Andres Freund wrote: > > Indeed. On SSDs I see about a 25-35% gain, on HDDs about 5%. If I > > increase the size of backend_flush_after to 64 (like it's for bgwriter) > > I however do get about 15% for HDDs as well

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 09:23, Fujii Masao wrote: > Okay, I pushed the patch! > Many thanks to all involved in the development of this feature! > Very good. I think the description in the commit message that we don't support "quorum commit" is sufficient to cover my concerns about what others might

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Michael Paquier
On Wed, Apr 6, 2016 at 5:23 PM, Fujii Masao wrote: > Okay, I pushed the patch! > Many thanks to all involved in the development of this feature! I think that I am crying... Really cool to see this milestone accomplished. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 10:04:42 +0100, Simon Riggs wrote: > On 6 April 2016 at 09:45, Andres Freund wrote: > > > On 2016-04-06 09:18:54 +0100, Simon Riggs wrote: > > > Rather than take that option, I went to the trouble of writing a patch > > that > > > does the same thing but simpler, less invasive and m

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Simon Riggs
On 6 April 2016 at 09:45, Andres Freund wrote: > On 2016-04-06 09:18:54 +0100, Simon Riggs wrote: > > Rather than take that option, I went to the trouble of writing a patch > that > > does the same thing but simpler, less invasive and more maintainable. > > Primarily, I did that for you, to avoid

Re: [HACKERS] [COMMITTERS] pgsql: Avoid archiving XLOG_RUNNING_XACTS on idle server

2016-04-06 Thread Andres Freund
On 2016-04-06 09:18:54 +0100, Simon Riggs wrote: > Rather than take that option, I went to the trouble of writing a patch that > does the same thing but simpler, less invasive and more maintainable. > Primarily, I did that for you, to avoid you having wasted your time and to > allow you to backpatc

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-06 Thread Fujii Masao
On Wed, Apr 6, 2016 at 5:07 PM, Fujii Masao wrote: > On Wed, Apr 6, 2016 at 4:08 PM, Michael Paquier > wrote: >> On Wed, Apr 6, 2016 at 3:29 PM, Fujii Masao wrote: >>> On Wed, Apr 6, 2016 at 2:18 PM, Kyotaro HORIGUCHI >>> wrote: At Tue, 5 Apr 2016 20:17:21 +0900, Fujii Masao wrote i

  1   2   >