Re: [HACKERS] password_encryption, default and 'plain' support

2017-05-05 Thread Michael Paquier
On Thu, May 4, 2017 at 8:37 PM, Heikki Linnakangas wrote: > On 05/03/2017 08:40 PM, Tom Lane wrote: >> >> The other question I can think to ask is what will happen during >> pg_upgrade, given an existing installation with one or more passwords >> stored plain. If the answer is

Re: [HACKERS] [patch] Build pgoutput with MSVC

2017-05-05 Thread Michael Paquier
On Fri, May 5, 2017 at 6:10 PM, MauMau wrote: > The pgoutput is not built with MSVC. The attached patch fixes this. > I confirmed that a few INSERTs were replicated correctly. > > Should I add this matter in the PostgreSQL 10 Open Items page? Yes, with Peter as committer

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-05-05 Thread Michael Paquier
On Fri, May 5, 2017 at 5:33 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > > On Fri, May 5, 2017 at 10:56 AM, Michael Paquier <michael.paqu...@gmail.com> > wrote: >> >> On Wed, May 3, 2017 at 12:25 AM, Peter Eisentraut >> >> >>

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-05-04 Thread Michael Paquier
On Wed, May 3, 2017 at 12:25 AM, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 5/2/17 10:08, Michael Paquier wrote: >> On Tue, May 2, 2017 at 9:30 PM, Peter Eisentraut >> <peter.eisentr...@2ndquadrant.com> wrote: >>> On 5/2/17 03:11, Petr Jeli

Re: [HACKERS] password_encryption, default and 'plain' support

2017-05-03 Thread Michael Paquier
On Wed, May 3, 2017 at 9:57 PM, Magnus Hagander <mag...@hagander.net> wrote: > > > On Wed, May 3, 2017 at 2:25 PM, Michael Paquier <michael.paqu...@gmail.com> > wrote: >> >> On Wed, May 3, 2017 at 8:38 PM, Magnus Hagander <mag...@hagander.net> >> wr

Re: [HACKERS] scram and \password

2017-05-03 Thread Michael Paquier
encryptPasswordConn and that would be a couple of extra lines in any applications. But honestly, people will appreciate a way to rely on what the backend uses automatically. > On 04/27/2017 07:03 AM, Michael Paquier wrote: >> I think that it is a mistake to move SASLprep out of >

Re: [HACKERS] password_encryption, default and 'plain' support

2017-05-03 Thread Michael Paquier
On Wed, May 3, 2017 at 8:38 PM, Magnus Hagander wrote: > On Wed, May 3, 2017 at 1:31 PM, Heikki Linnakangas wrote: >> In various threads on SCRAM, we've skirted around the question of whether >> we should still allow storing passwords in plaintext. I've

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-05-02 Thread Michael Paquier
On Tue, May 2, 2017 at 9:30 PM, Peter Eisentraut wrote: > On 5/2/17 03:11, Petr Jelinek wrote: >> logical decoding can theoretically >> do HOT pruning (even if the chance is really small) so it's not safe to >> start logical replication either. > > This seems a

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-05-02 Thread Michael Paquier
On Tue, May 2, 2017 at 9:27 PM, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 5/2/17 03:43, Michael Paquier wrote: >>> I don't think the code covers all because a) the SQL queries are not >>> covered at all that I can see and b) logical decoding c

Re: [HACKERS] Regarding B-Tree Lookup

2017-05-02 Thread Michael Paquier
On Tue, May 2, 2017 at 6:12 PM, Mahi Gurram wrote: > I'm building some custom extension on top of postgres 9.6.1. As part of > that, I would like to read Heap Tuple directly from my extension using > Primary Key. > > By default, postgres table index(B-Tree) its

Re: [HACKERS] [Proposal]: Extends VisualStudio to automatically precompile EmbeddedSQL

2017-05-02 Thread Michael Paquier
On Tue, May 2, 2017 at 6:35 PM, zosrothko wrote: > Hi > > I made an extension of VisualStudio that precompiles automaticaly C or > C++ source with PostgreSQL Embedded SQL. The extension is made of the 3 > files joined and I have no idea where they should be placed in the >

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-05-02 Thread Michael Paquier
On Tue, May 2, 2017 at 4:11 PM, Petr Jelinek <petr.jeli...@2ndquadrant.com> wrote: > On 02/05/17 05:35, Michael Paquier wrote: >> On Tue, May 2, 2017 at 7:07 AM, Peter Eisentraut >> <peter.eisentr...@2ndquadrant.com> wrote: >>> On 4/25/17 21:47, Michael Paquie

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-05-01 Thread Michael Paquier
On Tue, May 2, 2017 at 7:07 AM, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 4/25/17 21:47, Michael Paquier wrote: >> Attached is an updated patch to reflect that. > > I edited this a bit, here is a new version. Thanks, looks fine for me. >

Re: [HACKERS] .pgpass's behavior has changed

2017-04-28 Thread Michael Paquier
On Fri, Apr 28, 2017 at 4:54 PM, Kyotaro HORIGUCHI wrote: > I noticed that the precedence between host and hostaddr in a > connection string is reversed in regard to .pgpass lookup in > devel. > > For example the following connection string uses a .pgpass entry >

Re: [HACKERS] [PostgreSQL 10] default of hot_standby should be "on"?

2017-04-27 Thread Michael Paquier
On Thu, Apr 27, 2017 at 4:33 PM, Masahiko Sawada wrote: > On Thu, Apr 27, 2017 at 1:58 PM, Huong Dangminh > wrote: >>> On Thu, Apr 27, 2017 at 11:48 AM, Masahiko Sawada >>> wrote: >>> > Thank you for updating the patch.

Re: [HACKERS] scram and \password

2017-04-26 Thread Michael Paquier
On Thu, Apr 27, 2017 at 1:25 PM, Ashesh Vashi wrote: > - Do we need to provide the method here? > We have connection object itself, it can decide from the type of connection, > which method to be used. Providing the method is not mandatory. If you look upthread...

Re: [HACKERS] [PostgreSQL 10] default of hot_standby should be "on"?

2017-04-26 Thread Michael Paquier
On Thu, Apr 27, 2017 at 11:48 AM, Masahiko Sawada wrote: > Thank you for updating the patch. Also maybe we can update line in > PostgresNode.pm where hot_standby is set to on explicitly. I would refrain from doing that, having some parameters listed in the tests makes the

Re: [HACKERS] scram and \password

2017-04-26 Thread Michael Paquier
On Wed, Apr 26, 2017 at 7:22 PM, Heikki Linnakangas wrote: > We talked about the alternative where PQencryptPasswordConn() would not look > at password_encryption, but would always use the strongest possible > algorithm supported by the server. That would avoid querying the

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-26 Thread Michael Paquier
On Thu, Apr 27, 2017 at 2:25 AM, Tom Lane wrote: > Simon Riggs writes: >> I've added code following Michael and Tom's comments to the previous >> patch. New patch attached. > > Couple of minor suggestions: > > * Rather than deleting the comment for

Re: [HACKERS] Logical replication in the same cluster

2017-04-26 Thread Michael Paquier
On Thu, Apr 27, 2017 at 7:02 AM, Tom Lane wrote: > Petr Jelinek writes: >> On 26/04/17 18:59, Bruce Momjian wrote: >>> ... it just hangs. My server logs say: > >> Yes that's result of how logical replication slots work, the transaction >> that

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 10:56 AM, Bruce Momjian wrote: > First, I don't think RFC references belong in the release notes, let > alone RFC links. > > Second, there seems to be some confusion over what SCRAM-SHA-256 gives > us over MD5. I think there are a few benefits: > > o

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 4:53 AM, Simon Riggs wrote: > On 25 April 2017 at 16:28, Tom Lane wrote: >> Simon Riggs writes: >>> I can't see any reason now why overwriteOK should exist at all. I'm >>> guessing that the whole

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 3:17 AM, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 4/21/17 00:11, Michael Paquier wrote: >> Hmm. I have been actually looking at this solution and I am having >> doubts regarding its robustness. In short this would need to be

Re: [HACKERS] Re: logical replication and PANIC during shutdown checkpoint in publisher

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 4:26 AM, Peter Eisentraut wrote: > On 4/20/17 11:30, Peter Eisentraut wrote: >> We have a possible solution but need to work out a patch. Let's say >> next check-in on Monday. > > Update: We have a patch that looks promising, but we

Re: [HACKERS] scram and \password

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 12:26 AM, Heikki Linnakangas wrote: > Yeah, there is that. But we simply cannot change the signature of an > existing function. It would not only produce compile-time errors when > building old applications, which would arguably be a good thing, but it >

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 12:20 AM, Bruce Momjian <br...@momjian.us> wrote: > On Tue, Apr 25, 2017 at 02:39:40PM +0900, Michael Paquier wrote: >> >> Add SCRAM-SHA-256 >> support for password negotiation and storage (Michael >> Paquier, Heikki Linnakangas) >>

Re: [HACKERS] question: data file update when pg_basebackup in progress

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 1:45 AM, David G. Johnston wrote: > The first write to a page after a checkpoint is always recorded in the WAL > as a full page write. Every WAL file since the checkpoint must also be > copied to the backed up system. The replay of those WAL

Re: [HACKERS] Patch - Tcl 8.6 version support for PostgreSQL

2017-04-25 Thread Michael Paquier
On Tue, Apr 25, 2017 at 2:57 PM, Andres Freund wrote: > Any chance of formulating these in a version agnostic way, instead of > copying the same stanza for every version? E.g. using a wildcard or > such... Using glob() would be enough for this purpose. -- Michael -- Sent

Re: [HACKERS] PG 10 release notes

2017-04-24 Thread Michael Paquier
release-10.html Here is some feedback based on a first read. + +Issue fsync on the output files generated by pg_dump and pg_dumpall(Michael Paquier) + Missing a space here between pg_dumpall and my name. This is accessed via ts_headline() and to_tsvector. RIGHT SECTION? This shoul

Re: [HACKERS] scram and \password

2017-04-24 Thread Michael Paquier
On Sat, Apr 22, 2017 at 7:20 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > Do we really want to add a new function or have a hard failure? Any > application calling PQencryptPassword may trap itself silently if the > server uses scram as hba key or if the defa

Re: [HACKERS] identity columns

2017-04-23 Thread Michael Paquier
On Mon, Apr 24, 2017 at 10:03 AM, Vitaly Burovoy wrote: > On 4/23/17, Robert Haas wrote: >> On Thu, Apr 20, 2017 at 12:05 AM, Vitaly Burovoy >> wrote: >> But why do we need it? Instead of: >> >> ADD GENERATED { ALWAYS |

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-23 Thread Michael Paquier
On Sun, Apr 23, 2017 at 10:15 AM, Petr Jelinek <petr.jeli...@2ndquadrant.com> wrote: > On 21/04/17 06:11, Michael Paquier wrote: >> On Fri, Apr 21, 2017 at 12:29 AM, Peter Eisentraut >> <peter.eisentr...@2ndquadrant.com> wrote: >> Hmm. I have been actually looking

Re: [HACKERS] TAP tests - installcheck vs check

2017-04-23 Thread Michael Paquier
On Mon, Apr 24, 2017 at 7:29 AM, Andrew Dunstan wrote: > > AFAICT, unlike the pg_regress checks, which in the installcheck case run > against a running instance of postgres, for TAP tests the only > difference is that that for the check case a temp install is done,

Re: [HACKERS] A note about debugging TAP failures

2017-04-23 Thread Michael Paquier
On Sun, Apr 23, 2017 at 10:05 PM, Craig Ringer <craig.rin...@2ndquadrant.com> wrote: > On 23 Apr. 2017 10:32, "Michael Paquier" <michael.paqu...@gmail.com> wrote: > On Sun, Apr 23, 2017 at 7:48 AM, Daniel Gustafsson <dan...@yesql.se> wrote: >> Skipping

Re: [HACKERS] A note about debugging TAP failures

2017-04-22 Thread Michael Paquier
On Sun, Apr 23, 2017 at 7:48 AM, Daniel Gustafsson wrote: > Skipping the tempdir and instead using ${testname}_data_${name} without a > random suffix, we can achieve this with something along the lines of the > attached PoC. It works as now (retain of failure, remove on success

Re: [HACKERS] Small patch for pg_basebackup argument parsing

2017-04-22 Thread Michael Paquier
On Sat, Apr 22, 2017 at 11:12 PM, Pierre Ducroquet wrote: > Following your advice, I went through the source tree and cleaned up most > instances of that pattern. > I have attached the corresponding patch to this mail. > If you think this patch is indeed interesting, what

Re: [HACKERS] scram and \password

2017-04-21 Thread Michael Paquier
On Sat, Apr 22, 2017 at 5:04 AM, Heikki Linnakangas wrote: > I'll continue reviewing the rest of the patch on Monday, but one glaring > issue is that we cannot add an argument to the existing libpq > PQencryptPassword() function, because that's part of the public API. It > would

Re: [HACKERS] On-disk format of SCRAM verifiers

2017-04-21 Thread Michael Paquier
On Fri, Apr 21, 2017 at 9:25 PM, Stephen Frost wrote: > * Heikki Linnakangas (hlinn...@iki.fi) wrote: >> I think we should adopt that exact format, so that our verifiers are >> compatible with RFC 5803. It doesn't make any immediate difference, >> but since there is a standard

Re: [HACKERS] On-disk format of SCRAM verifiers

2017-04-21 Thread Michael Paquier
On Fri, Apr 21, 2017 at 10:02 PM, Simon Riggs wrote: > On 21 April 2017 at 10:20, Heikki Linnakangas wrote: >> But looking more closely, I think I misunderstood RFC 5803. It *does* in >> fact specify a single string format to store the verifier in. And the

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-20 Thread Michael Paquier
On Fri, Apr 21, 2017 at 12:29 AM, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 4/20/17 07:52, Petr Jelinek wrote: >> On 20/04/17 05:57, Michael Paquier wrote: >>> 2nd thoughts here... Ah now I see your point. True that there is no >>> way t

Re: [HACKERS] DROP SUBSCRIPTION, query cancellations and slot handling

2017-04-20 Thread Michael Paquier
On Fri, Apr 21, 2017 at 11:34 AM, Petr Jelinek wrote: > On 20/04/17 23:30, Peter Eisentraut wrote: >> On 4/20/17 10:19, Petr Jelinek wrote: >>> Hmm well since this only affects the synchronization of table >>> states/names, I guess we could just simply do that before

Re: [HACKERS] DROP SUBSCRIPTION, query cancellations and slot handling

2017-04-20 Thread Michael Paquier
On Thu, Apr 20, 2017 at 8:47 PM, Petr Jelinek wrote: > Or you can drop the slot manually on upstream. Sure, but the point here is that if for example users have client_min_messages set at least at warning, they may have no idea that an underlying slot has been

Re: [HACKERS] DROP SUBSCRIPTION, query cancellations and slot handling

2017-04-20 Thread Michael Paquier
On Thu, Apr 20, 2017 at 4:22 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > I am adding an open item. Just adding something... When a subscription is created, if the step synchronizing tables fails then CREATE SUBSCRIPTION fails but the slot remains present on the publisher

[HACKERS] DROP SUBSCRIPTION, query cancellations and slot handling

2017-04-20 Thread Michael Paquier
Hi, I have noticed the following behavior with DROP SUBSCRIPTION followed by a cancel request. If the remote replication slot is dropped, the subscription may still be present locally: =# CREATE SUBSCRIPTION mysub CONNECTION 'port=5432 user=mpaquier dbname=mpaquier' PUBLICATION mypub,

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-19 Thread Michael Paquier
On Thu, Apr 20, 2017 at 12:40 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Thu, Apr 20, 2017 at 4:57 AM, Peter Eisentraut > <peter.eisentr...@2ndquadrant.com> wrote: >> I think the problem with a signal-based solution is that there is no >> feedback. I

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-19 Thread Michael Paquier
On Thu, Apr 20, 2017 at 4:57 AM, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 4/19/17 01:45, Michael Paquier wrote: >> On Tue, Apr 18, 2017 at 3:27 AM, Peter Eisentraut >> <peter.eisentr...@2ndquadrant.com> wrote: >>> I'd imagine the

Re: [HACKERS] Continuous buildfarm failures on hamster with bin-check

2017-04-19 Thread Michael Paquier
On Wed, Apr 19, 2017 at 7:03 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Apr 19, 2017 at 12:48 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> Alvaro Herrera <alvhe...@2ndquadrant.com> writes: >>> Tom Lane wrote: >>>> FWIW, I

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-18 Thread Michael Paquier
On Wed, Apr 19, 2017 at 1:52 PM, Masahiko Sawada wrote: > On Wed, Apr 19, 2017 at 12:34 PM, Noah Misch wrote: >> On Sun, Apr 16, 2017 at 07:25:28PM +0900, Fujii Masao wrote: >>> On Sun, Apr 16, 2017 at 1:19 PM, Noah Misch wrote: >>> >

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-18 Thread Michael Paquier
On Tue, Apr 18, 2017 at 3:27 AM, Peter Eisentraut wrote: > I'd imagine the postmaster would tell the walsender that it has started > shutdown, and then the walsender would reject $certain_things. But I > don't see an existing way for the walsender to know that

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-04-18 Thread Michael Paquier
On Wed, Apr 19, 2017 at 12:36 PM, David Rowley wrote: > In favour of "location" -> "lsn": Tom, Stephen, David Steel > In favour of "lsn" -> "location": Peter, Kyotaro I vote for "location" -> "lsn". I would expect complains about the current inconsistency at some

Re: [HACKERS] Failed recovery with new faster 2PC code

2017-04-18 Thread Michael Paquier
On Wed, Apr 19, 2017 at 11:09 AM, Jeff Janes wrote: > Would this bug have been seen in a replica server in the absence of crashes, > or was it only vulnerable during crash recovery rather than streaming > replication? This issue could have been seen on a streaming standby

Re: [HACKERS] Continuous buildfarm failures on hamster with bin-check

2017-04-18 Thread Michael Paquier
On Wed, Apr 19, 2017 at 12:48 AM, Tom Lane wrote: > Alvaro Herrera writes: >> Tom Lane wrote: >>> FWIW, I'm a bit suspicious of relocating the temp stats directory as >>> being a reliable fix for this. > >> It's an SD card (the kind typically used in

Re: [HACKERS] Continuous buildfarm failures on hamster with bin-check

2017-04-18 Thread Michael Paquier
On Tue, Apr 18, 2017 at 9:35 PM, Andrew Dunstan <andrew.duns...@2ndquadrant.com> wrote: > > > On 04/18/2017 08:23 AM, Michael Paquier wrote: >> On Tue, Apr 18, 2017 at 9:13 PM, Andrew Dunstan >> <andrew.duns...@2ndquadrant.com> wrote: >>> Yeah, but t

Re: [HACKERS] Continuous buildfarm failures on hamster with bin-check

2017-04-18 Thread Michael Paquier
On Tue, Apr 18, 2017 at 9:13 PM, Andrew Dunstan wrote: > Yeah, but the way you have done it could also to lead to errors unless > you're very careful, as I found on axolotl (which died recently, > unfortunately). There I had to set the stats_temp directory to a >

Re: [HACKERS] Failed recovery with new faster 2PC code

2017-04-18 Thread Michael Paquier
On Tue, Apr 18, 2017 at 7:54 PM, Simon Riggs wrote: > Yeh, this is better. Pushed. I have been outraced on this one, the error is obvious once you see it ;) Thanks for the investigation and the fix! I have spent a couple of hours reviewing the interactions between the

Re: [HACKERS] PANIC in pg_commit_ts slru after crashes

2017-04-18 Thread Michael Paquier
On Tue, Apr 18, 2017 at 7:05 PM, Simon Riggs wrote: > I've added a recheck in ProcessTwoPhaseBuffer() after we acquire the lock. > > If its worth acquiring the lock its worth checking we don't have a race. I see. No objections to that. -- Michael -- Sent via

Re: [HACKERS] Continuous buildfarm failures on hamster with bin-check

2017-04-18 Thread Michael Paquier
On Tue, Apr 18, 2017 at 4:15 PM, Andres Freund <and...@anarazel.de> wrote: > Hi, > > On 2017-04-18 16:07:38 +0900, Michael Paquier wrote: >> Some of you may have noticed that hamster is heavily red on the >> buildfarm. I have done a bit of investigation, and I am able

[HACKERS] Continuous buildfarm failures on hamster with bin-check

2017-04-18 Thread Michael Paquier
Hi all, Some of you may have noticed that hamster is heavily red on the buildfarm. I have done a bit of investigation, and I am able to reproduce the failure manually. But actually after looking at the logs the error has obviously showed up: 2017-04-16 05:07:19.650 JST [18282] LOG: database

Re: [HACKERS] PANIC in pg_commit_ts slru after crashes

2017-04-17 Thread Michael Paquier
On Tue, Apr 18, 2017 at 12:33 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Sun, Apr 16, 2017 at 6:59 PM, Michael Paquier <michael.paqu...@gmail.com> > wrote: >> >> >> >> Jeff, does this patch make the situation better? The fix is rather >> simp

Re: [HACKERS] PANIC in pg_commit_ts slru after crashes

2017-04-16 Thread Michael Paquier
On Sun, Apr 16, 2017 at 6:02 PM, Simon Riggs wrote: > On 15 April 2017 at 21:30, Jeff Janes wrote: >> On Fri, Apr 14, 2017 at 9:33 PM, Pavan Deolasee >> wrote: >> >>> >>> Since all those offsets fall on a page boundary, my

Re: [HACKERS] Logical replication launcher uses wal_retrieve_retry_interval

2017-04-14 Thread Michael Paquier
On Fri, Apr 14, 2017 at 9:19 PM, Petr Jelinek wrote: > I am not quite sure adding more GUCs is all that great option. When > writing the patches I was wondering if we should perhaps rename the > wal_receiver_timeout and wal_retrieve_retry_interval to something that >

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-14 Thread Michael Paquier
On Fri, Apr 14, 2017 at 8:28 PM, Craig Ringer wrote: > There's no point advertising scram-512 if only -256 can work for 'bob' > because that's what we have in pg_authid. The possibility to have multiple verifiers has other benefits than that, password rolling being

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-14 Thread Michael Paquier
On Fri, Apr 14, 2017 at 8:03 PM, Stephen Frost wrote: > Some of those were specifically left around to test those code paths. > I'm not sure if these were those or not though, Andrew was the one who > reviewed the various pg_dumpall calls to add --no-sync in places. Well,

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-14 Thread Michael Paquier
On Thu, Apr 6, 2017 at 7:48 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Apr 5, 2017 at 10:24 PM, Stephen Frost <sfr...@snowman.net> wrote: >> * Michael Paquier (michael.paqu...@gmail.com) wrote: >>> 1) Initialize the old cluster and start it. >

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-13 Thread Michael Paquier
On Fri, Apr 14, 2017 at 1:37 AM, Heikki Linnakangas <hlinn...@iki.fi> wrote: > On 04/13/2017 05:53 AM, Michael Paquier wrote: >> +* Parse the list of SASL authentication mechanisms in the >> +* AuthenticationSASL message, and select the best mechanism that we >

Re: [HACKERS] identity columns

2017-04-13 Thread Michael Paquier
On Thu, Apr 13, 2017 at 7:40 PM, Vitaly Burovoy wrote: > On 4/6/17, Vitaly Burovoy wrote: >> On 4/6/17, Peter Eisentraut wrote: >>> As I tried to mention earlier, it is very difficult to implement the IF >>>

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-13 Thread Michael Paquier
On Fri, Apr 14, 2017 at 2:47 AM, Fujii Masao wrote: > I'm thinking that it's less confusing to report always 0 as the priority of > async standby whatever the setting of synchronous_standby_names is. > Thought? Or we could have priority being reported to NULL for async

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-13 Thread Michael Paquier
On Fri, Apr 14, 2017 at 3:03 AM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Thu, Apr 13, 2017 at 12:36 PM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> On Thu, Apr 13, 2017 at 12:28 PM, Fujii Masao <masao.fu...@gmail.com> wrote: >>> On Thu, Ap

Re: [HACKERS] Small patch for pg_basebackup argument parsing

2017-04-13 Thread Michael Paquier
On Fri, Apr 14, 2017 at 6:32 AM, Pierre Ducroquet wrote: > Yesterday while doing a few pg_basebackup, I realized that the integer > parameters were not properly checked against invalid input. > It is not a critical issue, but this could be misleading for an user who > writes

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-12 Thread Michael Paquier
On Tue, Apr 11, 2017 at 5:38 PM, Kyotaro HORIGUCHI wrote: > Sorry, what I have just sent was broken. You can use PROVE_TESTS when running make check to select a subset of tests you want to run. I use that all the time when working on patches dedicated to certain

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-12 Thread Michael Paquier
On Thu, Apr 13, 2017 at 12:28 PM, Fujii Masao wrote: > On Thu, Apr 13, 2017 at 5:25 AM, Peter Eisentraut > wrote: >> On 4/12/17 09:55, Fujii Masao wrote: >>> To fix this issue, we should terminate walsender for logical replication >>>

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-12 Thread Michael Paquier
On Thu, Apr 13, 2017 at 6:37 AM, Álvaro Hernández Tortosa wrote: > By looking at the them, and unless I'm missing something, I don't see > how the extra information for the future implementation of channel binding > would be added (without changing the protocol). Relevant

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-12 Thread Michael Paquier
On Thu, Apr 13, 2017 at 2:34 AM, Heikki Linnakangas wrote: > On 04/11/2017 02:32 PM, Álvaro Hernández Tortosa wrote: >> >> So I still see your proposal more awkward and less clear, mixing >> things that are separate. But again, your choice :) > > > So, here's my more

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Michael Paquier
On Thu, Apr 13, 2017 at 3:21 AM, Robert Haas wrote: > On Wed, Apr 12, 2017 at 2:09 PM, Heikki Linnakangas wrote: >> Yes, we need to nail down the protocol and \password before beta. I am >> working on them now. > > Good to hear. FWIW, there are patches

Re: [HACKERS] TAP tests take a long time

2017-04-11 Thread Michael Paquier
On Wed, Apr 12, 2017 at 9:12 AM, Craig Ringer wrote: > Well, you can get a lot of information on timings in crake's latest > builds for example, or nightjar's. > > Here's an interesting fact: almost all the time taken up in the scripts > test set seems to be consumed

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-04-11 Thread Michael Paquier
On Wed, Apr 12, 2017 at 3:40 AM, Tom Lane wrote: > I notice looking at pg_stat_activity that the logical replication launcher > sets its application_name to "logical replication launcher". This seems > inconsistent (no other standard background process sets application_name),

Re: [HACKERS] Reversed sync check in pg_receivewal

2017-04-11 Thread Michael Paquier
On Tue, Apr 11, 2017 at 9:41 PM, Magnus Hagander wrote: > This bug seems to have snuck in there with the introduction of walmethods. > AFAICT we are testing the result of sync() backwards, so whenever a partial > segment exists for pg_receivewal, it will fail. It will then

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-11 Thread Michael Paquier
On Tue, Apr 11, 2017 at 9:53 PM, Álvaro Hernández Tortosa wrote: > I know this is a lost battle. But please bear with me for a minute. I admire your courage. > But just a bit more is needed to make it really a big announcement and > provide real value to (I guess,

Re: [HACKERS] contrib/bloom wal-check not run by default

2017-04-10 Thread Michael Paquier
On Tue, Apr 11, 2017 at 12:14 AM, Peter Eisentraut wrote: > Why $subject? > > Does it just need to be wired into the makefiles a bit better? Looks like an oversight to me. I would suggest changing the Makefile like that: diff --git a/contrib/bloom/Makefile

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-10 Thread Michael Paquier
On Mon, Apr 10, 2017 at 5:11 PM, Heikki Linnakangas wrote: > Here are some characters that seem plausible to be misinterpreted and > ignored by SASLprep: > EUC-JP and EUC-JISX0213: > > U+00AD (C2 AD): 足 (meaning "foot", per Unihan database) > U+FE00-FE0F (EF B8 8X): 鏝 (meaning

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-10 Thread Michael Paquier
On Tue, Apr 11, 2017 at 4:41 AM, Heikki Linnakangas wrote: > On 04/10/2017 09:33 PM, Álvaro Hernández Tortosa wrote: >> - If the username used is the one sent in the startup message, rather >> than leaving it empty in the client-first-message, I would force it to >> be the same

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-10 Thread Michael Paquier
On Tue, Apr 11, 2017 at 1:45 AM, Tom Lane wrote: > Magnus Hagander writes: > Are these votes for getting rid of both win32.mak and bcc32.mak? > >> PFA a patch that does this. Did I miss something? :) > > Perhaps we should get rid of the

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2017-04-10 Thread Michael Paquier
On Mon, Apr 10, 2017 at 5:47 PM, Magnus Hagander wrote: > Based on that we seem to agree here, should we add this as an open item? > Clearly if we want to change this, we should do so before 10. This really is a new feature, so as the focus is to stabilize things I think

Re: [HACKERS] Compiler warning in costsize.c

2017-04-10 Thread Michael Paquier
On Tue, Apr 11, 2017 at 4:02 AM, Robert Haas wrote: > On Mon, Apr 10, 2017 at 2:09 PM, Tom Lane wrote: >> ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: >>> Why bother with the 'rte' variable at all if it's only used for the >>>

Re: [HACKERS] Compiler warning in costsize.c

2017-04-10 Thread Michael Paquier
On Mon, Apr 10, 2017 at 9:05 PM, Tom Lane wrote: > I wonder if we shouldn't just do > > RangeTblEntry *rte PG_USED_FOR_ASSERTS_ONLY; > ListCell *lc; > > /* Should only be applied to base relations that are subqueries */ > Assert(rel->relid >

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-10 Thread Michael Paquier
On Mon, Apr 10, 2017 at 8:35 PM, Tom Lane wrote: > Magnus Hagander writes: >> Are these votes for getting rid of both win32.mak and bcc32.mak? > > I'm for it. +1. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] scram and \password

2017-04-09 Thread Michael Paquier
On Mon, Apr 10, 2017 at 12:53 PM, Noah Misch wrote: > On Wed, Apr 05, 2017 at 08:11:25PM +0300, Heikki Linnakangas wrote: >> >Heikki, are you going to do something about these? We're running out of >> >time. >> >> Sorry I've been procrastinating. I'm on it now. (We need to do

Re: [HACKERS] [COMMITTERS] pgsql: Sync pg_dump and pg_dumpall output

2017-04-09 Thread Michael Paquier
On Mon, Apr 10, 2017 at 1:01 PM, Noah Misch wrote: > This PostgreSQL 10 open item is past due for your status update. Kindly send > a status update within 24 hours, and include a date for your subsequent status > update. Refer to the policy on open item ownership: >

Re: [HACKERS] SCRAM authentication, take three

2017-04-09 Thread Michael Paquier
On Sat, Apr 8, 2017 at 9:28 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Apr 7, 2017 at 6:32 PM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> On Sat, Apr 8, 2017 at 1:59 AM, Robert Haas <robertmh...@gmail.com> wrote: >>> On Fri, Apr

Re: [HACKERS] logical replication and SIGHUP

2017-04-09 Thread Michael Paquier
On Mon, Apr 10, 2017 at 11:41 AM, Noah Misch wrote: > On Thu, Apr 06, 2017 at 02:21:29AM +0900, Fujii Masao wrote: >> Both launcher and worker don't handle SIGHUP signal and cannot >> reload the configuration. I think that this is a bug. Will add this as >> an open item barring

Re: [HACKERS] 2017-03 CF Closed

2017-04-08 Thread Michael Paquier
On Sat, Apr 8, 2017 at 11:27 PM, Andres Freund wrote: > Thanks for your work on managing the fest! +1. Great work! -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Running make check-world in buildfarm (was Re: [COMMITTERS] pgsql: Use SASLprep to normalize passwords for SCRAM authentication.)

2017-04-08 Thread Michael Paquier
On Sat, Apr 8, 2017 at 7:33 PM, Heikki Linnakangas wrote: > Hmm. It looks like none of the buildfarm members are running the > authentication tests. Nor recovery tests, nor subscription tests. We're > missing a trick here, at least some of the buildfarm members really ought to >

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Michael Paquier
On Sat, Apr 8, 2017 at 1:59 AM, Robert Haas wrote: > On Fri, Apr 7, 2017 at 3:59 AM, Heikki Linnakangas wrote: >> I think the "SCRAM" part is more important than "SHA-256", so -1 on that. > > I agree. The point here isn't that we're using a better hashing

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-07 Thread Michael Paquier
On Fri, Apr 7, 2017 at 8:58 PM, Heikki Linnakangas <hlinn...@iki.fi> wrote: > On 04/07/2017 05:30 AM, Michael Paquier wrote: >> I am really wondering if this should not reflect the real range >> reported by the RFC. I understand that you have grouped things to save

Re: [HACKERS] Compiler warning in costsize.c

2017-04-07 Thread Michael Paquier
On Fri, Apr 7, 2017 at 12:38 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Apr 4, 2017 at 9:42 PM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> On Wed, Apr 5, 2017 at 2:54 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>>

Re: [HACKERS] Compiler warning in costsize.c

2017-04-07 Thread Michael Paquier
On Tue, Apr 4, 2017 at 9:42 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Apr 5, 2017 at 2:54 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> (I'm personally not that much in love with PG_USED_FOR_ASSERTS_ONLY, >> because it tends to confuse pgindent.) >

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-06 Thread Michael Paquier
On Fri, Apr 7, 2017 at 1:01 PM, Tom Lane wrote: > Still, it's not very clear why we need to cater for building just libpq > rather than the whole distribution, and a user of win32.mak presumably > has the option to do the latter. The core argument for bcc32.mak, > I think, is

[HACKERS] Compilation warning with MSVC in pg_depend.c

2017-04-06 Thread Michael Paquier
Hi all, I am getting the following warning with MSVC 2010 for what is a result of 3217327 (identity columns): c:\users\michael\git\postgres\src\backend\catalog\pg_depend.c(615): warning C4715: 'getOwnedSequence' : not all control paths return a value

[HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-06 Thread Michael Paquier
Hi all, While looking at some SCRAM stuff, I have bumped into bcc32.mak and win32.mak in src/interfaces/libpq. To put it short: those files are not up to date. The code of SCRAM is in the tree for a bit of time now, and should have updated those files to list and clean up objects, but nobody has

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-06 Thread Michael Paquier
On Fri, Apr 7, 2017 at 2:47 AM, Heikki Linnakangas wrote: > On 04/06/2017 08:42 PM, Heikki Linnakangas wrote: >>> There is for example this portion in the new tables: >>> +static const Codepoint prohibited_output_chars[] = >>> +{ >>> + 0xD800, 0xF8FF, /* C.3, C.5 */

<    3   4   5   6   7   8   9   10   11   12   >