Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-04 Thread Michael Paquier
On Mon, Jul 3, 2017 at 7:40 PM, Amit Kapila wrote: > There is already such a test, see create_index.sql. > CREATE UNLOGGED TABLE unlogged_hash_table (id int4); > CREATE INDEX unlogged_hash_index ON unlogged_hash_table USING hash (id > int4_ops); > > Do you have something else in mind? > > I think

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-04 Thread Michael Paquier
On Tue, Jul 4, 2017 at 3:25 PM, Kuntal Ghosh wrote: > On Mon, Jul 3, 2017 at 6:50 PM, Michael Paquier > wrote: >> pg_basebackup/ with fe_recvint64() has its own way to do things, as >> does the large object things in libpq. I would think that at least on >> HEAD things

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-03 Thread Michael Paquier
On Tue, Jul 4, 2017 at 4:27 AM, Tom Lane wrote: > Peter Eisentraut writes: >> On 7/3/17 09:53, Tom Lane wrote: >>> Hm. Before we add a bunch of code to deal with that, are we sure we >>> *want* it to copy such files? Seems like that's expending a lot of >>> data-transfer work for zero added val

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

2017-07-03 Thread Michael Paquier
On Mon, Jul 3, 2017 at 11:00 PM, Tom Lane wrote: > Michael Paquier writes: >> On Mon, Jul 3, 2017 at 9:25 PM, Greg Stark wrote: >>> On 2 July 2017 at 18:33, Tom Lane wrote: >>>> system("cp -a ...") call in favor of something more portable. > >>

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

2017-07-03 Thread Michael Paquier
On Mon, Jul 3, 2017 at 10:02 PM, Alvaro Herrera wrote: > Tom Lane wrote: > >> (Other unfinished work: teaching the MSVC scripts to use this, >> and teaching pg_upgrade's test script to use it.) > > Maybe it'd be simpler to rewrite pg_upgrade tests using PostgresNode > instead? You are looking for

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-03 Thread Michael Paquier
On Mon, Jul 3, 2017 at 8:22 PM, Kuntal Ghosh wrote: > pg_rewind throws the following error when there is a file of large > size available in the Slave server's data directory. Oops. > I guess we've to change the data type to bigint. Also, we need some > implementation of ntohl() for 8-byte data

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

2017-07-03 Thread Michael Paquier
On Mon, Jul 3, 2017 at 9:25 PM, Greg Stark wrote: > On 2 July 2017 at 18:33, Tom Lane wrote: >> system("cp -a ...") call in favor of something more portable. > > If we're ok with using Perl there's File::Copy::Recursive::dircopy() > which does exactly that. This stuff needs to support perl down

Re: [HACKERS] Error-like LOG when connecting with SSL for password authentication

2017-07-03 Thread Michael Paquier
On Mon, Jul 3, 2017 at 9:02 PM, Heikki Linnakangas wrote: > Thanks, I've pushed the backend read part of this patch. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-07-02 Thread Michael Paquier
On Sat, Jul 1, 2017 at 7:20 AM, Alvaro Herrera wrote: > Michael Paquier wrote: >> Considering how crazy the conditions to make the information fetched >> by users inconsistent are met, I agree with that. > > Pushed. Thanks Álvaro for pushing the patch. I had a second look

Re: [HACKERS] Race-like failure in recovery/t/009_twophase.pl

2017-07-02 Thread Michael Paquier
(catching up test threads) On Mon, Jul 3, 2017 at 7:02 AM, Tom Lane wrote: > I'm now inclined to think that the correct fix is to ensure that we > run synchronous rep in both directions, rather than to insert delays > to substitute for that. Just setting synchronous_standby_names for > node pari

Re: [HACKERS] PostgresNode::poll_query_until hacking

2017-07-02 Thread Michael Paquier
On Sun, Jul 2, 2017 at 4:53 AM, Tom Lane wrote: > The attached proposed patch changes the TAP test infrastructure's > poll_query_until function in two ways: > > 1. An optional argument is added to allow specifying the query result > value we're waiting for, overriding the normal "t". This allows

[HACKERS] Re: pg_ctl wait exit code (was Re: [COMMITTERS] pgsql: Additional tests for subtransactions in recovery)

2017-07-02 Thread Michael Paquier
On Sat, Jul 1, 2017 at 4:47 AM, Peter Eisentraut wrote: > On 5/1/17 12:19, Peter Eisentraut wrote: >> However: Failure to complete promotion within the waiting time does not >> lead to an error exit, so you will not get a failure if the promotion >> does not finish. This is probably a mistake. L

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-02 Thread Michael Paquier
On Sat, Jul 1, 2017 at 3:59 AM, Stephen Frost wrote: > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: >> On 6/30/17 04:08, Masahiko Sawada wrote: >> >> I'm not sure. I think this can be considered a bug in the implementation >> >> for >> >> 10, and as such is "open for fixing". Howe

Re: [HACKERS] Incorrect mentions to pg_xlog in walmethods.c/h

2017-06-30 Thread Michael Paquier
On Sat, Jul 1, 2017 at 3:41 AM, Peter Eisentraut wrote: > On 6/27/17 01:26, Michael Paquier wrote: >> I have noticed $subject. A patch is attached. Those comments are not >> completely wrong either as pg_basebackup can generate pg_xlog as well, >> still I would recommend to

Re: [HACKERS] What does it mean by XLOG_BACKUP_RECORD?

2017-06-28 Thread Michael Paquier
On Thu, Jun 29, 2017 at 10:28 AM, Masahiko Sawada wrote: > While reading source codes I found the following comment in xlog.c. > > /* > * Have we passed our safe starting point? Note that minRecoveryPoint is > * known to be incorrectly set if ControlFile->backupEndRequired, until > * the XLOG_B

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-28 Thread Michael Paquier
On Thu, Jun 29, 2017 at 7:52 AM, Alvaro Herrera wrote: > Michael Paquier wrote: >> On Thu, Jun 8, 2017 at 3:31 AM, Robert Haas wrote: >> I take that you are referring to the two lookups in >> WalSndWaitForWal(), one in exec_replication_command(), one in >> WalSndL

Re: [HACKERS] Fast promotion not used when doing a recovery_target PITR restore?

2017-06-28 Thread Michael Paquier
On Wed, Jun 28, 2017 at 6:13 AM, Andres Freund wrote: > You seem to completely argue besides my point that the replication path > is *more* robust by now? And there's plenty scenarios where a faster > startup is quite crucial for performance. The difference between an > immediate shutdown + recov

Re: [HACKERS] Fast promotion not used when doing a recovery_target PITR restore?

2017-06-27 Thread Michael Paquier
On Wed, Jun 28, 2017 at 3:44 AM, Andres Freund wrote: > I'm far from convinced by this. By now WAL replay with checkpointer, > bgwriter, etc. active is actually *more* tested than the cases without > it. The likelihood of bugs is higher in the less frequently exercised > paths, and given that rep

Re: [HACKERS] pg_basebackup fails on Windows when using tablespace mapping

2017-06-27 Thread Michael Paquier
On Tue, Jun 27, 2017 at 7:46 PM, Ashutosh Sharma wrote: > I am still seeing the issue with the attached patch. I had a quick > look into the patch. It seems to me like you have canonicalized the > tablespace path to convert win32 slashes to unix type of slashes but > that is not being passed to st

[HACKERS] Incorrect mentions to pg_xlog in walmethods.c/h

2017-06-26 Thread Michael Paquier
Hi all, I have noticed $subject. A patch is attached. Those comments are not completely wrong either as pg_basebackup can generate pg_xlog as well, still I would recommend to just mention "pg_wal". Thanks, -- Michael walmethods-comments.patch Description: Binary data -- Sent via pgsql-hackers

Re: [HACKERS] pg_basebackup fails on Windows when using tablespace mapping

2017-06-26 Thread Michael Paquier
On Tue, Jun 27, 2017 at 12:13 PM, Michael Paquier wrote: > At quick glance, I think that this should definitely be a client-only > fix. One reason is that pg_basebackup should be able to work with past > servers. A second is that this impacts as well the backend code, and > read

Re: [HACKERS] pg_basebackup fails on Windows when using tablespace mapping

2017-06-26 Thread Michael Paquier
On Tue, Jun 27, 2017 at 1:57 AM, nb wrote: > Trying to take a `pg_basebackup -T OLDDIR=NEWDIR [etc]` on master (server > running the cluster) fails on Windows with error "pg_basebackup: directory > "OLDDIR" exists but is not empty". Yes, I can see this error easily. > This fixed the issue, but a

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-06-26 Thread Michael Paquier
On Mon, Jun 26, 2017 at 4:11 PM, Masahiko Sawada wrote: > Thank you for the patches! I checked additional patches for brin and > spgist. They look good to me. Last versions are still missing something: brin_mask() and spg_mask() can be updated so as mask_unused_space() is called for meta pages. E

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-06-26 Thread Michael Paquier
On Tue, Jun 27, 2017 at 3:44 AM, Tom Lane wrote: > Looks good as far as it goes, but I wonder whether any of the other > get_slot_xmins calls need polling too. Don't feel a need to add such > calls until someone exhibits a failure there, but I won't be very > surprised if someone does. I got the

Re: [HACKERS] Reducing pg_ctl's reaction time

2017-06-25 Thread Michael Paquier
On Mon, Jun 26, 2017 at 7:13 AM, Tom Lane wrote: > The attached proposed patch adjusts pg_ctl to check every 100msec, > instead of every second, for the postmaster to be done starting or > stopping. This cuts the runtime of the recovery TAP tests from around > 4m30s to around 3m10s on my machine,

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-06-25 Thread Michael Paquier
On Mon, Jun 26, 2017 at 12:12 PM, Craig Ringer wrote: > On 26 June 2017 at 11:06, Michael Paquier wrote: > >> As long as we are on it, there is this code block in the test: >> my ($xmin, $catalog_xmin) = get_slot_xmins($node_master, $slotname_1); >> is($xmin, &#x

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-06-25 Thread Michael Paquier
On Mon, Jun 26, 2017 at 11:44 AM, Craig Ringer wrote: > On 26 June 2017 at 10:09, Tom Lane wrote: >> Michael Paquier writes: >>> On Mon, Jun 26, 2017 at 10:48 AM, Craig Ringer >>> wrote: >>>> $node_standby_1->poll_query_until('postgres',

Re: [HACKERS] Timing-sensitive case in src/test/recovery TAP tests

2017-06-25 Thread Michael Paquier
On Mon, Jun 26, 2017 at 10:48 AM, Craig Ringer wrote: > $node_standby_1->poll_query_until('postgres', q[SELECT xmin IS NULL > from pg_replication_slots WHERE slot_name = '] . $slotname_2 . q[']); +1 for avoiding a sleep call if it is not necessary. Fast platforms would always pay a cost on that,

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-06-25 Thread Michael Paquier
On Fri, Jun 23, 2017 at 11:17 AM, Amit Langote wrote: > That was it, thanks for the pointer. GinInitMetabuffer() sets up pd_lower and pd_upper anyway using PageInit so the check of PageIsVerified is guaranteed to work in any case. Upgraded pages will still have their pd_lower set to the previous

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-25 Thread Michael Paquier
On Sun, Jun 25, 2017 at 2:11 PM, Alvaro Herrera wrote: > Noah Misch wrote: > >> IMMEDIATE ATTENTION REQUIRED. This PostgreSQL 10 open item is long past due >> for your status update. Please reacquaint yourself with the policy on open >> item ownership[1] and then reply immediately. If I do not

Re: [HACKERS] FIPS mode?

2017-06-23 Thread Michael Paquier
On Sat, Jun 24, 2017 at 12:56 PM, Curtis Ruck wrote: > I've got a requirement for enabling FIPS support in our environment. > Looking at postgresql's be-secure-openssl.c and mucking with it, it seems > fairly straight forward to just add a few ifdefs and enable fips with a new > configure flag and

Re: [HACKERS] pg_terminate_backend can terminate background workers and autovacuum launchers

2017-06-23 Thread Michael Paquier
On Sat, Jun 24, 2017 at 5:07 AM, Alvaro Herrera wrote: > Yugo Nagata wrote: > >> I tried to make it. Patch attached. >> >> It is easy and simple. Although I haven't tried for autovacuum worker, >> I confirmed I can change other process' parameters without affecting others. >> Do you want this in P

[HACKERS] Removing deprecation changing automatically type of opaque functions

2017-06-23 Thread Michael Paquier
Hi all, Per bug #14706, it has been discussed the possibility to remove the transformation of opaque functions happening for example when declaring a trigger: https://www.postgresql.org/message-id/cab7npqtc-7u1k2vz3qlxvsyaxbvncrc9zmuvyf4-wuzj8zf...@mail.gmail.com As far as I can see, the deprecat

Re: [HACKERS] Broken O(n^2) avoidance in wal segment recycling.

2017-06-23 Thread Michael Paquier
On Fri, Jun 23, 2017 at 3:25 PM, Michael Paquier wrote: > On Thu, Jun 22, 2017 at 6:10 AM, Andres Freund wrote: >> We've not heard any complaints about this afaik, but it's not something >> that's easily diagnosable as being a problem. Therefore I suspect we &g

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-06-22 Thread Michael Paquier
On Fri, Jun 23, 2017 at 3:17 PM, Amit Langote wrote: > Initially, I had naively set wal_consistency_check = all before running > make installcheck and then had to wait for a long time to confirm that WAL > generated by the gin test indeed caused consistency check failure on the > standby with the

Re: [HACKERS] Broken O(n^2) avoidance in wal segment recycling.

2017-06-22 Thread Michael Paquier
On Thu, Jun 22, 2017 at 6:10 AM, Andres Freund wrote: > We've not heard any complaints about this afaik, but it's not something > that's easily diagnosable as being a problem. Therefore I suspect we > should fix and backpatch this? Agreed. I have just poked at this problem and have finished with

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-06-22 Thread Michael Paquier
On Fri, Jun 23, 2017 at 2:56 PM, Masahiko Sawada wrote: > Thank you for updating the patch. It looks good to me. > BTW I'm inclined to have a regression test case where doing 'make > check' to the streaming replication environment with > wal_consistency_check on standby server so that we can detec

Re: [HACKERS] Fast promotion not used when doing a recovery_target PITR restore?

2017-06-22 Thread Michael Paquier
On Fri, Jun 23, 2017 at 2:34 AM, Andres Freund wrote: > I don't think it's really a bug - just a missed optimization. I'd > personally not be in favor of backpatching this - it'll have some chance > of screwing things up, even if I hope that chance is fairly small. It would be better to wait unt

Re: [HACKERS] shift_sjis_2004 related autority files are remaining

2017-06-22 Thread Michael Paquier
On Fri, Jun 23, 2017 at 9:39 AM, Tatsuo Ishii wrote: > I think we should keep the original .txt files because: Hm. I am wondering about licensing issues here to keep those files in the tree. I am no lawyer. > - It allows to track the changes in the original file if we decide to > change the ma

Re: [HACKERS] shift_sjis_2004 related autority files are remaining

2017-06-22 Thread Michael Paquier
On Fri, Jun 23, 2017 at 8:12 AM, Tatsuo Ishii wrote: >> Hi, I happned to notice that backend/utils/mb/Unicode directory >> contains two encoding authority files, which I believe are not to >> be there. (Worked on that with Horiguchi-san a couple of weeks back.) >> euc-jis-2004-std.txt >> sjis-02

Re: [HACKERS] Pluggable storage

2017-06-22 Thread Michael Paquier
On Thu, Jun 22, 2017 at 11:27 PM, Robert Haas wrote: > On Thu, Jun 22, 2017 at 8:32 AM, Alexander Korotkov > wrote: >> On Thu, Jun 22, 2017 at 4:01 AM, Michael Paquier >> wrote: >>> Putting that in a couple of words. >>> 1. Table AM with a 6-byte TID. &

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-22 Thread Michael Paquier
On Fri, Jun 23, 2017 at 7:02 AM, Alvaro Herrera wrote: > Thomas Munro wrote: >> I thought about this when designing the DSA API. I couldn't think of >> any good reason to provide an 'am-I-already-attached?' function >> equivalent to dsm_find_mapping. It seemed to me that the client code >> shoul

Re: [HACKERS] SQL MERGE patches for PostgreSQL Versions

2017-06-22 Thread Michael Paquier
On Thu, Jun 22, 2017 at 5:30 PM, Kang Yuzhe wrote: > I wish I could but it's because I don't believe that I have the right > capability to fix code conflicts. My ultimate goal is to be PG hacker > like YOU. Anyway, I will consider your perspective. Nice to see such a goal, though as a first patc

Re: [HACKERS] SQL MERGE patches for PostgreSQL Versions

2017-06-22 Thread Michael Paquier
On Thu, Jun 22, 2017 at 5:05 PM, Kang Yuzhe wrote: > First my apology if I appear to be a jerk or not following the policy. No problem. > I emailed Boxuan Zhai who was in charge of the SQL Merge keyword in > 2010 of GSoC but without reply. > > I want to apply merge_v201.patch to specific PG ver

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-06-22 Thread Michael Paquier
On Wed, Jun 21, 2017 at 9:42 AM, Amit Langote wrote: > On 2017/06/20 20:37, Amit Kapila wrote: >> On Tue, Jun 20, 2017 at 1:50 PM, Amit Langote >> wrote: >>> On 2017/06/19 23:31, Tom Lane wrote: I'd suggest a rule like "if pd_lower is smaller than SizeOfPageHeaderData then don't trust i

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 2:44 PM, Kuntal Ghosh wrote: > On Thu, Jun 22, 2017 at 9:48 AM, Michael Paquier > wrote: >> On Thu, Jun 22, 2017 at 1:29 AM, Kuntal Ghosh >> wrote: >>> But, I've some more doubts. >>> 1. When should we use dsm_find_mapping()?

Re: [HACKERS] archive_timeout ignored directly after promotion

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 2:58 AM, Andres Freund wrote: > One easy way to fix that would be to just wakeup the checkpointer from > the startup process once at the end of recovery, but it'd not be > pretty. I think it'd be better to change the > do_restartpoint = RecoveryInP

Re: [HACKERS] pg_terminate_backend can terminate background workers and autovacuum launchers

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 1:52 PM, Yugo Nagata wrote: > On Thu, 22 Jun 2017 12:05:19 +0900 > Michael Paquier wrote: >> signal-able). Different thought, but I'd love to see a SQL function >> that allows triggering SIGHUP on a specific process, like an >> autovac

Re: [HACKERS] Fast promotion not used when doing a recovery_target PITR restore?

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 3:04 AM, Andres Freund wrote: > When doing a PITR style recovery, with recovery target set, we're > currently not doing a fast promotion, in contrast to the handling when > doing a pg_ctl or trigger file based promotion. That can prolong making > the server available for wr

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 1:29 AM, Kuntal Ghosh wrote: > On Wed, Jun 21, 2017 at 7:52 PM, Dilip Kumar wrote: >> On Wed, Jun 21, 2017 at 7:47 PM, Kuntal Ghosh >> wrote: IMHO, It's not a good idea to use DSM call to verify the DSA handle. >>> Okay. Is there any particular scenario you've i

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Michael Paquier
On Wed, Jun 21, 2017 at 9:15 PM, Yugo Nagata wrote: > This errors continue until this process is terminated or the server is > restarted. > > When SIGINT is issued, the process exits from the main loop and returns > to sigsetjmp, and calls dsa_attach() before entering into the loop again, > this

Re: [HACKERS] pg_terminate_backend can terminate background workers and autovacuum launchers

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 11:52 AM, Andres Freund wrote: > On 2017-06-22 11:49:47 +0900, Yugo Nagata wrote: >> I agree that we can kill theses processes by the OS command. However, >> It seems to me that pg_{cancel,terminate}_backend don't need to be able to >> kill processes except for client backe

Re: [HACKERS] Pluggable storage

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 11:12 AM, Amit Langote wrote: > On 2017/06/22 10:01, Michael Paquier wrote: >> #3 implies that the index AM logic is implemented in the table >> AM. Not saying that it is not useful, but it does not feel natural to >> have the planner request for a seq

Re: [HACKERS] An attempt to reduce WALWriteLock contention

2017-06-21 Thread Michael Paquier
On Wed, Jun 21, 2017 at 4:57 PM, jasrajd wrote: > We are also seeing contention on the walwritelock and repeated writes to the > same offset if we move the flush outside the lock in the Azure environment. > pgbench doesn't scale beyond ~8 cores without saturating the IOPs or > bandwidth. Is there

Re: [HACKERS] Pluggable storage

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 4:47 AM, Robert Haas wrote: > I think that BitmapHeapScan, at least, is applicable to any table AM > that has TIDs. It seems to me that in general we can imagine three > kinds of table AMs: > > 1. Table AMs where a tuple can be efficiently located by a real TID. > By a re

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-21 Thread Michael Paquier
On Wed, Jun 21, 2017 at 11:42 PM, Daniel Gustafsson wrote: > The message is truncated in SetBackendCancelMessage() for safety, but > pg_{cancel|terminate}_backend() could throw an error on too long message, or > warning truncation, to the caller as well. Personally I think a warning is > the > a

Re: [HACKERS] [COMMITTERS] pgsql: Restart logical replication launcher when killed

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 4:16 AM, Peter Eisentraut wrote: > Restart logical replication launcher when killed - /* The logical replication launcher can be stopped at any time. */ - proc_exit(0); + /* The logical replication launcher can be stopped at any time. +

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-20 Thread Michael Paquier
On Tue, Jun 20, 2017 at 3:24 AM, Daniel Gustafsson wrote: > The message is stored in a new shmem area which is checked when the session is > aborted. To keep things simple a small buffer is kept per backend for the > message. If deemed too costly, keeping a central buffer from which slabs are >

Re: [HACKERS] Proposal for CSN based snapshots

2017-06-20 Thread Michael Paquier
On Wed, Aug 24, 2016 at 5:54 PM, Heikki Linnakangas wrote: > And here are the results on the 72 core machine (thanks again, Alexander!). > The test setup was the same as on the 32-core machine, except that I ran it > with more clients since the system has more CPU cores. In summary, in the > best

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-20 Thread Michael Paquier
On Wed, Jun 21, 2017 at 8:43 AM, Tom Lane wrote: > Yeah, I thought it would work fine with Makefile-using Windows toolchains. > But people who use MSVC need something else, no? Are there that many anyway who care? Personally I already fallback to Linux when it comes to indentation of Windows patc

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

2017-06-20 Thread Michael Paquier
On Wed, Jun 21, 2017 at 4:04 AM, Álvaro Hernández Tortosa wrote: > In the coming weeks, and once my PR for pgjdbc has been added, I will > work towards another patch to implement channel binding. Should be > reasonably easy now, thanks to this. So you basically have an equivalent of OpenSSL s

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

2017-06-19 Thread Michael Paquier
On Tue, Jun 6, 2017 at 3:40 PM, Michael Paquier wrote: > As far as I can see, there are a couple of things that I still need to > work on to make people happy: > - Rework the generic APIs for TLS finish and endpoint so as any > implementation can use channel binding without induci

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-19 Thread Michael Paquier
On Tue, Jun 20, 2017 at 10:43 AM, Craig Ringer wrote: > Especially with a 6-week-old baby now Congratulations! -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Directory pg_replslot is not properly cleaned

2017-06-19 Thread Michael Paquier
On Mon, Jun 19, 2017 at 10:58 PM, Fabrízio de Royes Mello wrote: > Do you know when the next minor versions will be released? Because depending > of the schedule I'll patch the current customer version because we need the > "pglogical" running stable. The next round is planned for the 10th of Aug

Re: [HACKERS] Incorrect comment in 001_ssltests.pl

2017-06-17 Thread Michael Paquier
On Sat, Jun 17, 2017 at 5:20 PM, Magnus Hagander wrote: > One could argue that the first sentences should just read "the argument" > once the second one isn't referred, but I can't get too excited about that. > Thus, patch applied as-is - thanks! Thanks. -- Michael -- Sent via pgsql-hackers m

[HACKERS] Incorrect comment in 001_ssltests.pl

2017-06-16 Thread Michael Paquier
Hi all, I have noticed the following thing: --- a/src/test/ssl/t/001_ssltests.pl +++ b/src/test/ssl/t/001_ssltests.pl @@ -34,8 +34,6 @@ sub run_test_psql # The first argument is a (part of a) connection string, and it's also printed # out as the test case name. It is appended to $common_connstr

Re: [HACKERS] PATCH: Don't downcase filepath/filename while loading libraries

2017-06-15 Thread Michael Paquier
On Fri, Jun 16, 2017 at 1:01 PM, Tom Lane wrote: > (2) My inclination would be not to back-patch. This change could break > configurations that worked before, and the lack of prior complaints > says that not many people are having a problem with it. That's fourteen years without complains, still

Re: [HACKERS] PATCH: Don't downcase filepath/filename while loading libraries

2017-06-15 Thread Michael Paquier
On Fri, Jun 16, 2017 at 11:04 AM, QL Zhuo wrote: > I just put this line in my postgresql.conf: > > ``` > shared_preload_libraries = '/Path/Contains/UpCaseWords/an_ext.so' > ``` > > Then the server couldn't start. It tried to load the file > "/path/contains/upcasewords/an_ext.so" and failed. > > Af

Re: [HACKERS] Assorted leaks of PGresults

2017-06-14 Thread Michael Paquier
On Thu, Jun 15, 2017 at 9:36 AM, Tom Lane wrote: > Coverity complained that libpqwalreceiver.c's libpqrcv_PQexec() > potentially leaks a PGresult. It's right: if PQconsumeInput() fails after > we've already collected at least one PGresult, the function just returns > NULL without remembering to f

Re: [HACKERS] pg_receivewal and messages printed in non-verbose mode

2017-06-13 Thread Michael Paquier
On Tue, Jun 13, 2017 at 4:50 PM, Craig Ringer wrote: > On 13 June 2017 at 14:33, Michael Paquier wrote: >> Those come from stop_streaming in pg_receivewal.c. Shouldn't those >> messages only show up to the user if --verbose is used? It seems >> strange to me that

Re: [HACKERS] Detection of IPC::Run presence in SSL TAP tests

2017-06-13 Thread Michael Paquier
On Tue, Jun 13, 2017 at 11:14 PM, Tom Lane wrote: > Certainly, it's pointless to have a defense only here. And I know very > well that make check falls over in an ugly, hard-to-interpret-if-you've- > not-seen-it-before fashion if you do --enable-tap-tests and don't have > IPC::Run installed. > >

Re: [HACKERS] Document bug regarding read only transactions

2017-06-13 Thread Michael Paquier
On Wed, Jun 14, 2017 at 11:08 AM, Tatsuo Ishii wrote: > > In normal operation, read-only transactions are allowed to > -update sequences and to use LISTEN, UNLISTEN, and > +use LISTEN, UNLISTEN, and > NOTIFY, so Hot Standby sessions operate under slightly > tighter > r

[HACKERS] Detection of IPC::Run presence in SSL TAP tests

2017-06-13 Thread Michael Paquier
Hi all, 001_ssltests.pl in src/test/ssl/ includes the following to skip all tests should IPC::Run be not available: # Like TestLib.pm, we use IPC::Run BEGIN { eval { require IPC::Run; import IPC::Run qw(run start); 1; } or do

[HACKERS] pg_receivewal and messages printed in non-verbose mode

2017-06-12 Thread Michael Paquier
Hi all, I have noticed that the following messages can show up from pg_receivewal even if the verbose mode is not used: if (prevtimeline != 0 && prevtimeline != timeline) fprintf(stderr, _("%s: switched to timeline %u at %X/%X\n"), progname,

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-12 Thread Michael Paquier
On Thu, Jun 8, 2017 at 1:15 PM, Michael Paquier wrote: > 3) In walreceiver.c's pg_stat_get_wal_receiver's: > - Launch pg_stat_get_wal_receiver and take a checkpoint on it. > - Pass the lookups of pid and ready_to_display. > - Make the WAL receiver stop. > - The view repo

Re: [HACKERS] pg_subscription_rel entry can be updated concurrently

2017-06-12 Thread Michael Paquier
On Tue, Jun 13, 2017 at 9:52 AM, Masahiko Sawada wrote: > Hi, > > I often get an error "ERROR: tuple concurrently updated" when > changing subscription state(ALTER SUBSCRIPTION or DROP SUBSCRIPTION). > The cause of this error is that table sync worker and apply worker can > try to update the same

Re: [HACKERS] ICU support on Windows

2017-06-12 Thread Michael Paquier
On Tue, Jun 13, 2017 at 6:47 AM, Alvaro Herrera wrote: > Michael Paquier wrote: >> I do some low level packaging on Windows (libxml2, libxslt, etc.), and >> the compilation code usually allows you to usually use the >> installation paths you want. At the end using only lib/

Re: [HACKERS] ICU support on Windows

2017-06-12 Thread Michael Paquier
On Tue, Jun 13, 2017 at 3:13 AM, Alvaro Herrera wrote: > Ashutosh Sharma wrote: >> > I noticed that this only works if you use the "Win32" download of ICU, >> > because the "Win64" download uses "lib64" paths. I'm not sure what the >> > impact of this is in practice. >> >> Yes, that's right, Win6

Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled when creating a logical decoding slot on standby

2017-06-11 Thread Michael Paquier
On Mon, Jun 12, 2017 at 3:16 PM, sanyam jain wrote: > I have created a logical decoding slot on a standby but i haven't enabled > Hot_standby_feedback.What are the test cases where this setup will fail? hot_standby_feedback needs to be enabled at all times in logical decoding so as the node does

Re: [HACKERS] ICU support on Windows

2017-06-11 Thread Michael Paquier
On Sat, Jun 10, 2017 at 6:40 PM, Ashutosh Sharma wrote: > Currently, we cannot perform ICU enabled build for postgres on Windows > platform. However, this can be done on Linux platforms using > '--with-icu' configuration parameter. Attached is the patch that > allows us to perform icu enabled buil

[HACKERS] Re: BUG #14680: startup process on standby encounter a deadlock of TwoPhaseStateLock when redo 2PC xlog

2017-06-08 Thread Michael Paquier
On Fri, Jun 9, 2017 at 3:17 PM, Noah Misch wrote: > This PostgreSQL 10 open item is past due for your status update. Kindly send > a status update within 24 hours, and include a date for your subsequent status > update. Refer to the policy on open item ownership: > https://www.postgresql.org/mes

Re: [HACKERS] Coverage improvements of src/bin/pg_basebackup and pg_receivewal --endpos

2017-06-08 Thread Michael Paquier
On Wed, Jun 7, 2017 at 9:20 AM, Michael Paquier wrote: > While it is possible to tackle some of those issues independently, > like pg_basebackup stuff, it seems to me that it would be helpful to > make the tests more deterministic by having an --endpos option for > pg_receivewal,

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-07 Thread Michael Paquier
v On Thu, Jun 8, 2017 at 3:31 AM, Robert Haas wrote: > I think if you're going to fix it so that we take spinlocks on > MyWalSnd in a bunch of places that we didn't take them before, it > would make sense to fix all the places where we're accessing those > fields without a spinlock instead of onl

Re: [HACKERS] Server ignores contents of SASLInitialResponse

2017-06-07 Thread Michael Paquier
On Wed, Jun 7, 2017 at 8:27 PM, Heikki Linnakangas wrote: > Ok, I committed your patch, with some minor changes. Thanks for the commit. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgs

Re: [HACKERS] tap tests on older branches fail if concurrency is used

2017-06-06 Thread Michael Paquier
On Thu, Jun 1, 2017 at 10:48 PM, Tom Lane wrote: > Andres Freund writes: >> when using >> $ cat ~/.proverc >> -j9 >> some tests fail for me in 9.4 and 9.5. > > Weren't there fixes specifically intended to make that safe, awhile ago? 60f826c has not been back-patched. While this would fix paralle

Re: [HACKERS] tap tests on older branches fail if concurrency is used

2017-06-06 Thread Michael Paquier
On Wed, May 31, 2017 at 8:45 PM, Craig Ringer wrote: > On 1 June 2017 at 08:15, Andres Freund wrote: >> Hi, >> >> when using >> $ cat ~/.proverc >> -j9 >> >> some tests fail for me in 9.4 and 9.5. E.g. src/bin/script's tests >> yields a lot of fun like: >> $ (cd ~/build/postgres/9.5-assert/vpath

Re: [HACKERS] Notes on testing Postgres 10b1

2017-06-06 Thread Michael Paquier
On Wed, Jun 7, 2017 at 2:01 PM, Josh Berkus wrote: > Q1. Why does wal_level default to "replica" and not "logical"? The difference of WAL generated is way higher between archive->hot_standby than hot_standby->logical. And unlike replica, logical decoding is not something that is widely spread in

Re: [HACKERS] [BUGS] BUG #14682: row level security not work with partitioned table

2017-06-06 Thread Michael Paquier
On Wed, Jun 7, 2017 at 9:52 AM, Joe Conway wrote: > On 06/06/2017 02:44 PM, Mike Palmiotto wrote: >> On Tue, Jun 6, 2017 at 4:07 PM, Joe Conway wrote: >>> On 06/06/2017 11:57 AM, Mike Palmiotto wrote: On Mon, Jun 5, 2017 at 10:36 AM, Robert Haas wrote: > On Mon, Jun 5, 2017 at 10:20 AM,

[HACKERS] Coverage improvements of src/bin/pg_basebackup and pg_receivewal --endpos

2017-06-06 Thread Michael Paquier
Hi all, The coverage of pg_basebackup is reaching 50%, which is not bad: https://coverage.postgresql.org/src/bin/pg_basebackup/index.html In this set pg_receivewal.c is the bad student with less than 20%. There are a couple of causes behind that, with no tests like: - option interactions like --c

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

2017-06-05 Thread Michael Paquier
On Fri, Jun 2, 2017 at 10:08 AM, Stephen Frost wrote: > I don't have any issue with asking that Michael, or someone, to go look > at other OpenSSL-using implementations which support channel binding. I don't see the implementation of other TLS/SSL as a requirement for channel binding with OpenSSL

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

2017-06-05 Thread Michael Paquier
On Fri, Jun 2, 2017 at 10:25 AM, Robert Haas wrote: > On Thu, Jun 1, 2017 at 9:13 PM, Michael Paquier > wrote: >> It seems to me that any testing in this area won't fly high as long as >> there is no way to enforce the list of TLS implementations that a >> s

Re: [HACKERS] Server ignores contents of SASLInitialResponse

2017-06-05 Thread Michael Paquier
On Thu, Jun 1, 2017 at 4:58 AM, Heikki Linnakangas wrote: > I bisected that; the culprit was commit 61bf96cab0, where I refactored the > libpq authentication code in preparation for SCRAM. The logic around that > free() was always a bit wonky, but the refactoring made it outright broken. > Attache

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-05 Thread Michael Paquier
On Tue, Jun 6, 2017 at 9:47 AM, Andres Freund wrote: > On 2017-06-05 15:30:38 +0900, Michael Paquier wrote: >> I think that it would be interesting to be able to >> trigger a feedback message using SIGHUP in WAL receivers, refactoring >> at the same time SIGHUP handling for

Re: [HACKERS] Server ignores contents of SASLInitialResponse

2017-06-04 Thread Michael Paquier
On Mon, Jun 5, 2017 at 7:49 AM, Noah Misch wrote: > IMMEDIATE ATTENTION REQUIRED. This PostgreSQL 10 open item is long past due > for your status update. Please reacquaint yourself with the policy on open > item ownership[1] and then reply immediately. If I do not hear from you by > 2017-06-05

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-04 Thread Michael Paquier
On Mon, Jun 5, 2017 at 10:29 AM, Andres Freund wrote: > On 2017-06-02 17:20:23 -0700, Andres Freund wrote: >> Attached is a *preliminary* patch series implementing this. I've first >> reverted the previous patch, as otherwise backpatchable versions of the >> necessary patches would get too compli

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-06-04 Thread Michael Paquier
On Mon, Jun 5, 2017 at 10:29 AM, Andres Freund wrote: > Michael, Peter, Fujii, is either of you planning to review this? I'm > planning to commit this tomorrow morning PST, unless somebody protest > till then. Yes, I am. It would be nice if you could let me 24 hours to look at it in details. --

Re: [HACKERS] Support to COMMENT ON DATABASE CURRENT_DATABASE

2017-06-04 Thread Michael Paquier
On Mon, Jun 5, 2017 at 10:09 AM, Jing Wang wrote: > By using the patch the CURRENT_DATABASE as a keyword can be used in the > following SQL commands: > > 1. COMMENT ON DATABASE CURRENT_DATABASE is ... > 2. ALTER DATABASE CURRENT_DATABASE OWNER to ... > 3. ALTER DATABASE CURRENT_DATABAS

[HACKERS] Re: BUG #14680: startup process on standby encounter a deadlock of TwoPhaseStateLock when redo 2PC xlog

2017-06-04 Thread Michael Paquier
On Mon, Jun 5, 2017 at 7:24 AM, Noah Misch wrote: > [Action required within three days. This is a generic notification.] > > The above-described topic is currently a PostgreSQL 10 open item. Simon, > since you committed the patch believed to have created it, you own this open > item. If some ot

Re: [HACKERS] Should we standardize on a type for signal handler flags?

2017-06-04 Thread Michael Paquier
On Mon, Jun 5, 2017 at 8:00 AM, Andres Freund wrote: > At the moment a number of flag variables set in signal handlers have > 'volatile bool' as type, others have 'volatile sig_atomic_t'. That's > kinda confusing. I think either is safe, but I think we should > standardize one of them. sig_ato

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-06-03 Thread Michael Paquier
On Mon, May 29, 2017 at 10:47 AM, Thomas Munro wrote: >> [Quoting Michael] >>> Actually, with the recent work that has been done with >>> unicode_norm_table.h which has been to transpose UnicodeData.txt into >>> user-friendly tables, shouldn't the python script of unaccent/ be >>> replaced by some

<    1   2   3   4   5   6   7   8   9   10   >