Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-08-14 Thread Michael Paquier
On Sun, Aug 6, 2017 at 3:22 AM, Robert Haas wrote: > All of (1)-(3) are legitimate user choices, although not everyone will > make them. (4) is unfortunately the procedure recommended by our > documentation, which is where the problem comes in. I think it's > pretty lame

Re: [HACKERS] Regressions failures with libxml2 on ArchLinux

2017-08-14 Thread Michael Paquier
On Sat, Aug 12, 2017 at 8:46 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > I have not investigated much this problem yet, but has somebody else > seen those diffs? Coming back to that... Downgrading down to 2.9.4+4+g3169602-1 is proving to put things back the way they

[HACKERS] Simplify ACL handling for large objects and removal of superuser() checks

2017-08-13 Thread Michael Paquier
2a2b2beddc5b01ffe6de7e442e20e00b4e518859 Mon Sep 17 00:00:00 2001 From: Michael Paquier <mich...@paquier.xyz> Date: Mon, 14 Aug 2017 12:01:58 +0900 Subject: [PATCH 3/3] Move ACL checks for large objects when opening them Up to now, ACL checks for large objects happened at the the level of the SQL-callable functions,

Re: [HACKERS] More race conditions in logical replication

2017-08-12 Thread Michael Paquier
On Sun, Aug 13, 2017 at 10:25 AM, Noah Misch wrote: > Committed. These counts broke three times in the v10 release cycle. It's too > bad this defect doesn't cause an error when building the docs. That's another argument for generating the table dynamically. Thanks for the

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-08-12 Thread Michael Paquier
On Sun, Aug 13, 2017 at 1:09 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Michael Paquier <michael.paqu...@gmail.com> writes: >> On Wed, Aug 9, 2017 at 3:39 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> Michael Paquier <michael.paqu...@gmail.com> writes: >

[HACKERS] Regressions failures with libxml2 on ArchLinux

2017-08-12 Thread Michael Paquier
Hi all, It's been one month since I have done some serious development with Archlinux (I was abroad and away from the laptop dedicated to that), and surprise, I can see failures in the PG regression tests, like the following short extract (result compared to expected/xml.out): SELECT

Re: [HACKERS] More race conditions in logical replication

2017-08-12 Thread Michael Paquier
On Thu, Aug 10, 2017 at 4:22 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Aug 8, 2017 at 8:11 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> > wrote: >> Here's a patch. It turned to be a bit larger than I initially expected. > > Álvaro, 030273b7

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Michael Paquier
On Fri, Aug 11, 2017 at 9:31 PM, Álvaro Hernández Tortosa <a...@8kdata.com> wrote: > On 11/08/17 13:18, Michael Paquier wrote: >> On Fri, Aug 11, 2017 at 3:50 PM, Álvaro Hernández Tortosa >> <a...@8kdata.com> wrote: >>>> Relatedly, the SCRAM specifi

Re: [HACKERS] SCRAM protocol documentation

2017-08-11 Thread Michael Paquier
On Fri, Aug 11, 2017 at 3:50 PM, Álvaro Hernández Tortosa wrote: > On 11/08/17 03:57, Peter Eisentraut wrote: >> The SCRAM protocol documentation >> (https://www.postgresql.org/docs/devel/static/sasl-authentication.html) >> states >> >> "To avoid confusion, the client should use

Re: [HACKERS] Server crash (FailedAssertion) due to catcache refcount mis-handling

2017-08-10 Thread Michael Paquier
On Thu, Aug 10, 2017 at 9:00 PM, Robert Haas wrote: > On Thu, Aug 10, 2017 at 2:50 AM, Andreas Seltenreich > wrote: >> Will do. Won't miss this chance to try out discostu's extension >> pg_rage_terminator[1] :-) >> [1]

Re: [HACKERS] Creating backup history files for backups taken from standbys

2017-08-10 Thread Michael Paquier
On Thu, Aug 10, 2017 at 9:45 AM, Masahiko Sawada wrote: > Thank you for the patch. Regarding to creating the backup history file > on stanbys, is there any difference from the patch posted on earlier > thread? That's a rebased version on top of what has been applied,

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix inadequacies in recently added wait events

2017-08-10 Thread Michael Paquier
On Thu, Aug 10, 2017 at 2:52 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > With a minimal maintenance effort we can be careful enough. I think > that a comment for example in pgstat.c about the usage uniqueness > would be an adapted answer. By the way, let's discus

Re: [HACKERS] More race conditions in logical replication

2017-08-10 Thread Michael Paquier
On Tue, Aug 8, 2017 at 8:11 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Mon, Aug 7, 2017 at 8:14 PM, Alvaro Herrera >> wrote: >> > BTW, I noticed that the PG_WAIT_LOCK value that we're using for wait >> > event here (and in the

[HACKERS] Creating backup history files for backups taken from standbys

2017-08-10 Thread Michael Paquier
Hi all, In the recent thread related to a bug in pg_stop_backup when waiting for WAL segments to be archived, it has been mentioned that it would be nice to get backup history files generated as well on standbys:

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-08-10 Thread Michael Paquier
On Wed, Aug 9, 2017 at 3:39 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Michael Paquier <michael.paqu...@gmail.com> writes: >> On Tue, Aug 8, 2017 at 11:33 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> Michael Paquier <michael.paqu...@gmail.com> writes: >

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-08-09 Thread Michael Paquier
On Fri, Aug 4, 2017 at 8:19 AM, Masahiko Sawada wrote: > On Fri, Jul 28, 2017 at 2:24 PM, Noah Misch wrote: >> This item appears under "decisions to recheck mid-beta". If anyone is going >> to push for a change here, now is the time. > > It has been 1

Re: [HACKERS] Remove 1MB size limit in tsvector

2017-08-09 Thread Michael Paquier
On Wed, Aug 9, 2017 at 6:38 PM, Robert Haas wrote: > The patch doesn't really conform to our coding standards, though, so > you need to clean it up (or, if you're not sure what you need to do, > you need to have someone who knows how PostgreSQL code needs to look > review

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix inadequacies in recently added wait events

2017-08-09 Thread Michael Paquier
On Wed, Aug 9, 2017 at 9:25 PM, Alvaro Herrera wrote: > Tom Lane wrote: >> I think generating whatever we can from a single authoritative file >> is indeed a good idea. > > Yay. Indeed. >> But I had the impression that people also wanted to enforce a rule >> about

Re: [HACKERS] Cache lookup errors with functions manipulation object addresses

2017-08-09 Thread Michael Paquier
On Wed, Aug 9, 2017 at 2:47 PM, Aleksander Alekseev wrote: > I believe this patch is "Ready for Committer". > > The new status of this patch is: Ready for Committer Thanks for the lookup, but I think that this is still hasty as no discussion has happened about a couple

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-08-09 Thread Michael Paquier
On Tue, Aug 8, 2017 at 11:33 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Michael Paquier <michael.paqu...@gmail.com> writes: >> I got the same thought, wondering as well if get_slot_xmins should be >> renamed check_slot_xmins with the is() tests moved inside it as well.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix inadequacies in recently added wait events

2017-08-09 Thread Michael Paquier
On Tue, Aug 8, 2017 at 11:35 PM, Alvaro Herrera wrote: > Thread moved to -hackers. > > Thomas Munro wrote: >> On Wed, Aug 9, 2017 at 7:39 AM, Alvaro Herrera >> wrote: > >> > While at it, fix numerous other problems in the vicinity: > >> All of

Re: [HACKERS] FYI: branch for v11 devel is planned for next week

2017-08-08 Thread Michael Paquier
On Mon, Aug 7, 2017 at 5:33 AM, Tom Lane wrote: > The release team discussed this a couple weeks ago, but I don't > think anybody mentioned it on -hackers: v10 seems to be in good > enough shape that it's okay to make the REL_10_STABLE branch soon, > and open HEAD for v11

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-08-05 Thread Michael Paquier
On Sat, Aug 5, 2017 at 4:51 PM, Robert Haas wrote: > On Sat, Aug 5, 2017 at 9:11 AM, Robert Haas wrote: >> If we apply these patches to 9.6, then pg_stop_backup() on a standby >> will start writing backup history files and removing no-longer-needed

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-08-05 Thread Michael Paquier
On Sat, Aug 5, 2017 at 5:27 AM, Stephen Frost wrote: > Unfortunately the day got away from me due to some personal... adventures > (having to do with lack of air conditioning first and then lack of gas, > amongst a lot of other things going on right now...). I just got things

Re: [HACKERS] pg_upgrade failed with error - ERROR: column "a" in child table must be marked NOT NULL

2017-08-04 Thread Michael Paquier
On Fri, Aug 4, 2017 at 5:50 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Michael Paquier <michael.paqu...@gmail.com> writes: >> So I think that the attached patch is able to do the legwork. > > I've pushed this into HEAD. It seems like enough of a behavioral > change

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2017-08-04 Thread Michael Paquier
On Thu, Aug 3, 2017 at 11:26 PM, Daniel Gustafsson <dan...@yesql.se> wrote: >> On 03 Aug 2017, at 19:27, Michael Paquier <michael.paqu...@gmail.com> wrote: >> There were no APIs to get the TLS finish message last time I looked at OSX >> stuff, which mattered for tls-u

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2017-08-03 Thread Michael Paquier
On Thu, Aug 3, 2017 at 12:02 PM, Daniel Gustafsson wrote: > In https://postgr.es/m/69db7657-3f9d-4d30-8a4b-e06034251...@yesql.se I > presented a WIP patch for adding support for the Apple Secure Transport SSL > library on macOS as, an alternative to OpenSSL. That patch got put

Re: [HACKERS] Cache lookup errors with functions manipulation object addresses

2017-08-03 Thread Michael Paquier
On Fri, Jul 21, 2017 at 8:53 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > I can see your point. The v1 proposed above adds quite a lot of error > code churn to deal with the lack of missing_ok flag in > getObjectDescription, getObjectIdentity and getObjectIdentityParts.

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-08-03 Thread Michael Paquier
On Mon, Jul 31, 2017 at 7:27 PM, Alvaro Herrera wrote: > Robert Haas wrote: > >> An alternative approach is to have some kind of other identifier, >> let's call it a distributed transaction ID (DXID) which is mapped by >> each node onto a local XID. > > Postgres-XL seems

Re: [HACKERS] Clarification in pg10's pgupgrade.html step 10 (upgrading standby servers)

2017-08-03 Thread Michael Paquier
On Mon, Jul 31, 2017 at 6:13 PM, Robert Haas wrote: > On Fri, Jul 28, 2017 at 10:35 AM, Andreas Joseph Krogh > wrote: >> I'm reading https://www.postgresql.org/docs/10/static/pgupgrade.html to try >> to understand how to upgrade standby-servers using

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-08-03 Thread Michael Paquier
On Thu, Aug 3, 2017 at 4:29 AM, Stephen Frost wrote: > I'll provide another update tomorrow. Hopefully Michael is able to produce > a 9.6 patch, otherwise I'll do it. I have sent an updated version of the patch, with something that can be used for 9.6 as well. It would be

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-08-03 Thread Michael Paquier
On Wed, Aug 2, 2017 at 7:58 PM, Stephen Frost <sfr...@snowman.net> wrote: > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> Do you need a back-patchable version for 9.6? I could get one out of >> my pocket if necessary. > > I was just trying to find a bit

Re: [HACKERS] pgbench: Skipping the creating primary keys after initialization

2017-08-02 Thread Michael Paquier
On Wed, Aug 2, 2017 at 5:50 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Aug 2, 2017 at 10:17 AM, Tom Lane wrote: >>> Or in other words, this looks to me quite a bit like the hackery >>> that resulted in pgbench's -S and -N

Re: [HACKERS] On Complex Source Code Reading Strategy

2017-08-02 Thread Michael Paquier
On Wed, Aug 2, 2017 at 7:24 AM, Zeray Kalayu wrote: > Lastly, I strongly believe that Code is the ultimate truth and being > able to understand complex and high quality code effectively and > strategically is of paramount importance. Documentation to understand how a system

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-08-02 Thread Michael Paquier
On Mon, Jul 31, 2017 at 9:13 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Thu, Jul 27, 2017 at 10:27 AM, Stephen Frost wrote: >> > * Noah Misch (n...@leadboat.com) wrote: >> >> This PostgreSQL 10 open item is past due for

Re: [HACKERS] [PATCH v3] pg_progress() SQL function to monitor progression of long running SQL queries/utilities

2017-08-01 Thread Michael Paquier
On Tue, Aug 1, 2017 at 7:17 PM, Andres Freund <and...@anarazel.de> wrote: > On 2017-08-01 19:11:55 +0200, Michael Paquier wrote: >> I think that Depesz makes use of a non-default format for its >> explain.depesz.com, or he would have a hard time maintaining a >> depars

Re: [HACKERS] [PATCH v3] pg_progress() SQL function to monitor progression of long running SQL queries/utilities

2017-08-01 Thread Michael Paquier
On Tue, Aug 1, 2017 at 6:35 PM, Remi Colinet wrote: > I'am skeptical about the use of JSON, XML, and others in such output. You should not. > Does anyone use these formats (XML, JSON, YAML) for EXPLAIN output? > I suspect only TEXT format is being used. I think that

Re: [HACKERS] How to run PG TAP tests on windows?

2017-08-01 Thread Michael Paquier
On Tue, Aug 1, 2017 at 10:24 AM, Abbas Butt wrote: > Can anyone point out to a tutorial or a list of steps required to run PG TAP > tests on windows? Only MSVC has a special handling:

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-07-28 Thread Michael Paquier
On Fri, Jul 28, 2017 at 7:28 AM, Masahiko Sawada wrote: > That also requires to share the same XID space with all remote nodes. You are putting your finger on the main bottleneck with global consistency that XC and XL has because of that. And the source feeding the XIDs is

Re: [HACKERS] pg_upgrade failed with error - ERROR: column "a" in child table must be marked NOT NULL

2017-07-27 Thread Michael Paquier
On Wed, Jul 26, 2017 at 4:50 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Jul 26, 2017 at 4:02 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> Not sure what's involved there code-wise, though, nor whether we'd want >> to back-patch. > > I'

Re: [HACKERS] A couple of postgresql.conf.sample discrepancies

2017-07-27 Thread Michael Paquier
On Thu, Jul 27, 2017 at 10:27 AM, Aleksander Alekseev wrote: >> Here's a script that reminds you about GUCs you forgot to put in >> postgresql.conf.sample. It probably needs some work. Does this >> already happen somewhere else? I guess not, because it found two >>

Re: [HACKERS] pg_upgrade failed with error - ERROR: column "a" in child table must be marked NOT NULL

2017-07-26 Thread Michael Paquier
On Wed, Jul 26, 2017 at 4:02 PM, Tom Lane wrote: > Not sure what's involved there code-wise, though, nor whether we'd want > to back-patch. I'll try to look at the code around that to come up with a clear conclusion in the next couple of days, likely more as that's a vacation

Re: [HACKERS] pg_upgrade failed with error - ERROR: column "a" in child table must be marked NOT NULL

2017-07-26 Thread Michael Paquier
On Wed, Jul 26, 2017 at 12:09 PM, tushar wrote: > v9.5/9.6 > > create these objects - > CREATE TABLE constraint_rename_test (a int CONSTRAINT con1 CHECK (a > 0), b > int, c int); > CREATE TABLE constraint_rename_test2 (a int CONSTRAINT con1 CHECK (a > 0), d > int)

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-24 Thread Michael Paquier
On Mon, Jul 24, 2017 at 6:45 PM, Stephen Frost wrote: > What the change would do is make the pg_stop_backup() caller block until > the last WAL is archvied, and perhaps that ends up taking hours, and > then the connection is dropped for whatever reason and the backup fails >

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-21 Thread Michael Paquier
On Fri, Jul 21, 2017 at 8:22 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Thu, Jul 20, 2017 at 9:31 PM, Robert Haas <robertmh...@gmail.com> wrote: >> Still, it can't be worse than the status quo, where instead of int64 >> we're using int and int32, so m

Re: [HACKERS] Adding -E switch to pg_dumpall

2017-07-21 Thread Michael Paquier
On Thu, Jul 20, 2017 at 11:17 AM, Fabien COELHO wrote: > Ok for me. I switched the status to "Ready for committers". Thanks for the review, Fabien. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Cache lookup errors with functions manipulation object addresses

2017-07-21 Thread Michael Paquier
On Thu, Jul 20, 2017 at 6:26 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Michael Paquier wrote: >> On Thu, Jul 20, 2017 at 4:04 PM, Alvaro Herrera >> <alvhe...@2ndquadrant.com> wrote: >> > I think the addition of checks everywhere for NULL return i

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-21 Thread Michael Paquier
On Thu, Jul 20, 2017 at 9:31 PM, Robert Haas wrote: > I was initially surprised that your testing managed to pass, but then > I noticed that this sanity test is using && where it should really be > using ||; it will only fail if ALL of the data types are wrong. Oops. Oh,

Re: [HACKERS] PgFDW connection invalidation by ALTER SERVER/ALTER USER MAPPING

2017-07-20 Thread Michael Paquier
On Fri, Jul 21, 2017 at 12:23 AM, Alvaro Herrera wrote: > Kyotaro HORIGUCHI wrote: >> Finally, I added a new TAP test library PsqlSession. It offers >> interactive psql sessions. Then added a simple test to >> postgres_fdw using it. > > Hmm, I think this can be very

Re: [HACKERS] Cache lookup errors with functions manipulation object addresses

2017-07-20 Thread Michael Paquier
On Thu, Jul 20, 2017 at 4:04 PM, Alvaro Herrera wrote: > I think the addition of checks everywhere for NULL return is worse. > Let's add a missing_ok flag instead, so that most callers can just trust > that they get a non null value if they don't want to deal with that >

Re: [HACKERS] Cache lookup errors with functions manipulation object addresses

2017-07-20 Thread Michael Paquier
On Wed, Jul 19, 2017 at 7:29 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jul 19, 2017 at 2:25 AM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> Would we want to improve the error handling of such objects? > > +1 for such an improvement

Re: [HACKERS] xlogfilename

2017-07-20 Thread Michael Paquier
On Thu, Jul 20, 2017 at 10:58 AM, DEV_OPS wrote: > I think you may reference to function: pg_xlogfile_name in > src/backend/access/transam/xlogfuncs.c, it use XLogFileName defined in > src/include/access/xlog_internal.h > > #define XLogFileName(fname, tli, logSegNo) \ >

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-20 Thread Michael Paquier
On Fri, Jul 7, 2017 at 9:33 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Fri, Jul 7, 2017 at 4:31 PM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> > wrote: >> On Fri, Jul 7, 2017 at 7:49 AM, Michael Paquier >> <michael.paqu...@gmail.com> wr

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2017-07-19 Thread Michael Paquier
On Wed, Jul 19, 2017 at 8:59 PM, Fabrízio de Royes Mello wrote: > On Wed, Jul 19, 2017 at 3:54 PM, Robins Tharakan wrote: >> You may want to consider this patch (attached) which additionally has the >> pg_dumpall changes. >> It would be great if you

Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

2017-07-19 Thread Michael Paquier
On Wed, Jul 19, 2017 at 8:56 PM, Tom Lane wrote: > I wonder if it'd be worth the trouble to stick something like this into > xlog.c: > > /* > * For reliability's sake, it's critical that pg_control updates > * be atomic writes. That generally means

Re: [HACKERS] Using non-sequential timelines in order to help with possible collisions

2017-07-19 Thread Michael Paquier
On Wed, Jul 19, 2017 at 8:05 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Michael Paquier <michael.paqu...@gmail.com> writes: >> On Wed, Jul 19, 2017 at 7:00 PM, Robert Haas <robertmh...@gmail.com> wrote: >>> It's interesting that you bring this up. I've also

Re: [HACKERS] Using non-sequential timelines in order to help with possible collisions

2017-07-19 Thread Michael Paquier
On Wed, Jul 19, 2017 at 7:00 PM, Robert Haas wrote: > On Wed, Jul 19, 2017 at 11:23 AM, Brian Faherty > wrote: >> I was working with replication and recovery the other day and noticed that >> there were scenarios where I could cause multiple

[HACKERS] Cache lookup errors with functions manipulation object addresses

2017-07-19 Thread Michael Paquier
Hi all, Per an offline report from Moshe Jacobson, it is possible to trigger easily cache lookup errors using pg_describe_object with invalid object IDs and pg_describe_object(). I had a closer look at things in this area, to notice that there are other user-facing failures as many things use the

Re: [HACKERS] Proposal for CSN based snapshots

2017-07-18 Thread Michael Paquier
On Wed, Jun 21, 2017 at 1:48 PM, Alexander Kuzmenkov wrote: > Glad to see you working on this! I've been studying this topic too. Attached > you can find a recently rebased version of Heikki's v4 patch. > I also fixed a bug that appeared on report-receipts isolation

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2017-07-18 Thread Michael Paquier
On Tue, Jul 18, 2017 at 3:45 AM, David Fetter wrote: > The one I run into frequently is in a proprietary fork, RDS Postgres. > It'll happily dump out COMMENT ON EXTENSION plpgsq IS ... > which is great as far as it goes, but errors out when you try to > reload it. > > While

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-18 Thread Michael Paquier
On Tue, Jul 18, 2017 at 4:18 AM, Amit Kapila wrote: > Thanks. Do you have any suggestion for back-branches? As of now, it > fails badly with below kind of error: > > test=> SELECT * FROM t_u_hash; > ERROR: could not open file "base/16384/16392": No such file or

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-17 Thread Michael Paquier
On Mon, Jul 17, 2017 at 6:02 PM, Robert Haas wrote: > It seems to me that this area might benefit from a broader rethink. > It's not very nice to impose a restriction that init forks can only be > constructed using log_newpage(); on the other hand, it is also not > very

Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

2017-07-17 Thread Michael Paquier
On Sun, Jul 16, 2017 at 10:27 PM, Greg Stark wrote: > On 15 July 2017 at 23:00, Tom Lane wrote: > >> While it's too late in the v10 cycle to do anything very meaningful >> about this now, I am tempted to strengthen the deprecation notice's >> wording from

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2017-07-17 Thread Michael Paquier
On Thu, Jul 6, 2017 at 3:48 PM, Heikki Linnakangas wrote: > On 07/03/2017 06:30 PM, Chapman Flack wrote: >> Although it's moot in the straightforward approach of re-zeroing in >> the loop, it would still help my understanding of the system to know >> if there is some subtlety

Re: [HACKERS] Unportable use of select for timeouts in PostgresNode.pm

2017-07-17 Thread Michael Paquier
On Mon, Jul 17, 2017 at 4:48 PM, Andrew Dunstan wrote: > This is a usage that is known not to work in Windows - IIRC we > eliminated such calls from our C programs at the time of the Windows > port - and it seems to me very likely to be the cause of the hang. >

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2017-07-17 Thread Michael Paquier
On Fri, Jun 23, 2017 at 6:08 AM, Chapman Flack wrote: > Well, gzip was doing pretty well; it could get a 16 MB segment file down > to under 27 kB, or less than 14 bytes for each of 2000 pages, when a page > header is what, 20 bytes, it looks like? I'm not sure how much

Re: [HACKERS] Adding -E switch to pg_dumpall

2017-07-15 Thread Michael Paquier
On Fri, Jul 14, 2017 at 12:39 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > While looking at a user problem, I got surprised that pg_dumpall does > not have a -E switch. This has been discussed a bit in the past like > here: > https://www.postgre

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-15 Thread Michael Paquier
On Sat, Jul 15, 2017 at 6:27 AM, Ashutosh Sharma wrote: > I do agree with Amit. I think hash index is slightly trickier (in > terms of how it is organised) than other indexes and that could be the > reason for maintaining common code for hashbuild and hashbuildempty. Well,

Re: [HACKERS] SCRAM auth and Pgpool-II

2017-07-15 Thread Michael Paquier
On Sat, Jul 15, 2017 at 12:15 AM, Stephen Frost wrote: > While this might be possible by having some kind of special trusted > connection between the pooler and PG, I actually don't particularly like > the notion of inventing a bunch of complicated logic and pain so that a >

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-14 Thread Michael Paquier
(catching up finally with this thread) On Mon, Jul 10, 2017 at 11:57 AM, Kyotaro HORIGUCHI wrote: > At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila > wrote in

[HACKERS] Adding -E switch to pg_dumpall

2017-07-14 Thread Michael Paquier
Hi all, While looking at a user problem, I got surprised that pg_dumpall does not have a -E switch. This has been discussed a bit in the past like here: https://www.postgresql.org/message-id/75e4c42d37e6a74e9fb57c2e9f1300d601070...@tiger.nexperience.com Now it is possible to enforce the encoding

Re: [HACKERS] Authentication mechanisms categorization

2017-07-14 Thread Michael Paquier
On Fri, Jul 14, 2017 at 12:16 PM, Álvaro Hernández Tortosa wrote: > If the parameter authmethod would rather be "authmethods", i.e., a list, > I think it would be significantly more flexible. Yes, but the handling of a list becomes messier if there are some other connection

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-14 Thread Michael Paquier
On Fri, Jul 14, 2017 at 12:34 AM, Stephen Frost <sfr...@snowman.net> wrote: > Michael, all, > > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> On Thu, Jul 13, 2017 at 7:13 AM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >> > Sorry, I

Re: [HACKERS] Authentication mechanisms categorization

2017-07-14 Thread Michael Paquier
On Sat, Jul 8, 2017 at 2:19 PM, Álvaro Hernández Tortosa wrote: > There has been some prior discussion, that we recently continued at > pgday.ru, about what to do if a client wants to use a "strong" > authentication mechanism but a rogue server forces the client to use a >

Re: [HACKERS] PostgreSQL - Weak DH group

2017-07-13 Thread Michael Paquier
On Thu, Jul 13, 2017 at 5:32 PM, Heikki Linnakangas wrote: > I rebased the patch, did some other clean up of error reporting, and added a > GUC along those lines, as well as docs. How does this look? > > It's late in the release cycle, but it would be nice to sneak this into v10.

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-13 Thread Michael Paquier
On Thu, Jul 13, 2017 at 7:13 AM, Masahiko Sawada wrote: > Sorry, I missed lots of typo in the last patch. All comments from you > are incorporated into the attached latest patch and I've checked it > whether there is other typos. Please review it. Thanks for providing a

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-12 Thread Michael Paquier
On Tue, Jul 11, 2017 at 9:28 AM, Masahiko Sawada wrote: > Attached updated version patch. Please review it. Cool, thanks. +useless. If the second parameter wait_for_archive is true and +the backup is taken on a standby, pg_stop_backup waits for WAL +to

Re: [HACKERS] Minor style cleanup in tab-complete.c

2017-07-11 Thread Michael Paquier
On Wed, Jul 12, 2017 at 11:19 AM, Thomas Munro wrote: > From the triviality department: I noticed some branches in > tab-complete.c's gargantuan if statement, mostly brand new, that break > from the established brace style. Should we fix that like this? For

[HACKERS] Re: retry shm attach for windows (WAS: Re: OK, so culicidae is *still* broken)

2017-07-10 Thread Michael Paquier
On Mon, Jul 10, 2017 at 3:36 PM, Noah Misch wrote: > I recommend pushing your patch so the August back-branch releases have it. > One can see by inspection that your patch has negligible effect on systems > healthy today. I have a reasonable suspicion it will help some

Re: [HACKERS] SCRAM auth and Pgpool-II

2017-07-09 Thread Michael Paquier
On Sat, Jul 8, 2017 at 1:54 PM, Álvaro Hernández Tortosa wrote: > There's definitely an important concern here that should be addressed: > how poolers/proxies/middleware/etc can deal with SCRAM, specifically in the > context of channel binding. > > If there is to be a

Re: [HACKERS] SCRAM auth and Pgpool-II

2017-07-09 Thread Michael Paquier
On Sat, Jul 8, 2017 at 9:30 AM, Stephen Frost <sfr...@snowman.net> wrote: > Michael, > > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> On Sat, Jul 8, 2017 at 1:24 AM, Robert Haas <robertmh...@gmail.com> wrote: >> > IIUC, things will get even wor

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-09 Thread Michael Paquier
On Sat, Jul 8, 2017 at 12:50 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Fri, Jul 7, 2017 at 3:48 PM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> So I would suggest the following things to address this issue: >> 1) Generate a backup

Re: [HACKERS] pg_receivewal and messages printed in non-verbose mode

2017-07-09 Thread Michael Paquier
On Sun, Jul 9, 2017 at 9:30 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sun, Jul 9, 2017 at 5:58 AM, Magnus Hagander <mag...@hagander.net> wrote: >> I wonder if we should actually just remove the second message? AFAICT no >> other tools log that informati

Re: [HACKERS] Authentification method on client side checking

2017-07-09 Thread Michael Paquier
On Mon, Jul 10, 2017 at 9:29 AM, Álvaro Hernández Tortosa wrote: > Precisely yesterday I initiated a similar thread: > https://www.postgresql.org/message-id/d4098ef4-2910-c8bf-f1e3-f178ba77c381%408kdata.com > > I think that a) the mere auth mechanism is not enough

Re: [HACKERS] pg_receivewal and messages printed in non-verbose mode

2017-07-09 Thread Michael Paquier
On Sun, Jul 9, 2017 at 5:58 AM, Magnus Hagander wrote: > I wonder if we should actually just remove the second message? AFAICT no > other tools log that information. Is there any particular reason why we want > that logging in pg_receivewal when we don't have it in other

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-09 Thread Michael Paquier
On Sun, Jul 9, 2017 at 8:00 PM, Stephen Frost wrote: > * Noah Misch (n...@leadboat.com) wrote: >> [Action required within three days. This is a generic notification.] >> >> The above-described topic is currently a PostgreSQL 10 open item. Stephen, >> since you committed the

Re: [HACKERS] SCRAM auth and Pgpool-II

2017-07-07 Thread Michael Paquier
On Sat, Jul 8, 2017 at 1:24 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jul 5, 2017 at 9:32 PM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> On Thu, Jul 6, 2017 at 10:03 AM, Tatsuo Ishii <is...@sraoss.co.jp> wrote: >>> For Pgpool-II

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-07 Thread Michael Paquier
On Fri, Jul 7, 2017 at 4:31 PM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > On Fri, Jul 7, 2017 at 7:49 AM, Michael Paquier > <michael.paqu...@gmail.com> wrote: > I don't have any more inputs on this patch and it looks good to me. > So, I'm moving the status to read

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-07 Thread Michael Paquier
On Fri, Jul 7, 2017 at 4:06 PM, Masahiko Sawada wrote: > I agree with this idea. I've tried to make it wait for archiving but > it seems to me that there are other two issues we need to deal with: > the timeline ID on standby server is always reset after created a > restart

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-07 Thread Michael Paquier
On Wed, Jul 5, 2017 at 4:57 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > Why not refactoring a bit do_pg_stop_backup() so as the wait phase > happens even if a backup is started in recovery? Now wait_for_archive > is ignored because no wait is happening and the stop po

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-07-07 Thread Michael Paquier
On Tue, Jul 4, 2017 at 1:34 PM, Noah Misch wrote: > Bundling code cleanup into commits that also do something else is strictly > worse than bundling whitespace cleanup, which is itself bad: > https://postgr.es/m/flat/20160113144826.gb3379...@tornado.leadboat.com FWIW, I agree

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-06 Thread Michael Paquier
On Thu, Jul 6, 2017 at 8:51 PM, Dilip Kumar wrote: > On Thu, Jul 6, 2017 at 4:18 PM, Kuntal Ghosh > wrote: >> But, I'm little concerned/doubt regarding the following part of the code. >> +/* >> + * Converts an int64 from network byte order to

Re: [HACKERS] SCRAM auth and Pgpool-II

2017-07-05 Thread Michael Paquier
On Thu, Jul 6, 2017 at 10:03 AM, Tatsuo Ishii wrote: > For Pgpool-II, things would go as follows: > > 1) clients sends user name to Pgpool-II. > 2) Pgpool-II forwards it to PostgreSQL servers. > 3) Each PostgreSQL server sends their own salt to Pgpool-II. > 4) Pgpool-II is

Re: [HACKERS] Re: pg_ctl wait exit code (was Re: [COMMITTERS] pgsql: Additional tests for subtransactions in recovery)

2017-07-05 Thread Michael Paquier
On Thu, Jul 6, 2017 at 2:41 AM, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 7/2/17 20:28, Michael Paquier wrote: >>> I was going to hold this back for PG11, but since we're now doing some >>> other tweaks in pg_ctl, it might be useful to add this to

Re: [HACKERS] Re: Error-like LOG when connecting with SSL for password authentication

2017-07-05 Thread Michael Paquier
On Thu, Jul 6, 2017 at 12:45 AM, Ryan Murphy wrote: > I tried to apply your patch to HEAD and it failed. > But I think that's because some version of it has already been committed, > correct? I see some of your ECONNRESET and "SSL connection has been closed >

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-05 Thread Michael Paquier
On Wed, Jul 5, 2017 at 10:19 AM, Masahiko Sawada wrote: > I feel that since we cannot switch the WAL forcibly on the standby > server we need to find a new way to do so. I'm not sure but it might > be a hard work and be late for PG10. Or you meant that you have a idea > for

Re: [HACKERS] Small patch for pg_basebackup argument parsing

2017-07-05 Thread Michael Paquier
On Wed, Jul 5, 2017 at 2:57 PM, Ryan Murphy wrote: > I tried to apply your patch to test it (though reading Robert's last comment > it seems we wish to have it adjusted before committing)... but in any case I > was not able to apply your patch to the tip of the master

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-07-05 Thread Michael Paquier
On Wed, Jun 7, 2017 at 1:06 AM, Man Trieu wrote: > 2017-06-07 0:31 GMT+09:00 Bruce Momjian : >> >> On Wed, Jun 7, 2017 at 12:10:25AM +0900, Dang Minh Huong wrote: >> > > On Jun 4, 29 Heisei, at 00:48, Bruce Momjian wrote: >> >

Re: [HACKERS] Incorrect mentions to pg_xlog in walmethods.c/h

2017-07-04 Thread Michael Paquier
On Wed, Jul 5, 2017 at 2:29 PM, Ryan Murphy wrote: > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: not tested > Spec compliant: not tested > Documentation:not

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-04 Thread Michael Paquier
On Tue, Jul 4, 2017 at 4:41 PM, Kuntal Ghosh wrote: > I've not yet started the patch and it may take some time for me to > understand and write > the patch in a correct way. Since, you've almost written the patch, > IMHO, please go ahead > and submit the patch. I'll

<    1   2   3   4   5   6   7   8   9   10   >