Re: serializable transaction: exclude constraint violation (backed by GIST index) instead of ssi conflict

2019-04-12 Thread Thomas Munro
On Fri, Apr 12, 2019 at 6:01 AM Peter Billen wrote: > On Thu, Apr 11, 2019 at 6:12 PM Peter Billen wrote: >> I believe we are after multiple issues/improvements: >> >> 1. Could we extend >> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=fcff8a575198478023ada8a48e13b50f70054766

Re: pg_dump is broken for partition tablespaces

2019-04-12 Thread Alvaro Herrera
On 2019-Mar-06, David Rowley wrote: > Over on [1] Andres pointed out that the pg_dump support for the new to > PG12 tablespace inheritance feature is broken. This is the feature > added in ca4103025dfe26 to allow a partitioned table to have a > tablespace that acts as the default tablespace for

Re: change password_encryption default to scram-sha-256?

2019-04-12 Thread Bruce Momjian
On Mon, Apr 8, 2019 at 10:08:07AM -0400, Tom Lane wrote: > "Jonathan S. Katz" writes: > > On 4/8/19 8:49 AM, Magnus Hagander wrote: > >> I think the real question is, is it OK to give them basically 5months > >> warning, by right now saying if you don't have a release out in 6 > >> months,

Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3

2019-04-12 Thread Bruce Momjian
On Sat, Apr 6, 2019 at 03:29:13PM +0200, Antonin Houska wrote: > Robert Haas wrote: > > > On Fri, Apr 5, 2019 at 11:22 AM Antonin Houska wrote: > > > > Wouldn't Tom's proposal to use a stream cipher fix all this? > > > > > > Yes it would make the extra alignment unnecessary, but our solution

Re: Useless code in RelationCacheInitializePhase3

2019-04-12 Thread Andres Freund
Hi, On 2019-04-12 14:17:11 -0400, Tom Lane wrote: > While looking at the pending patch to clean up management of > rd_partcheck, I noticed that RelationCacheInitializePhase3 has code that > purports to reload rd_partkey and rd_partdesc, but none for rd_partcheck. > However, that reload code is

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-12 Thread Tom Lane
Robert Haas writes: > As a parenthetical note, I observe that relcache.c seems to know > almost nothing about rd_partcheck. rd_partkey and rd_partdesc both > have handling in RelationClearRelation(), but rd_partcheck does not, > and I suspect that's wrong. So the problems are probably not

Re: PANIC: could not flush dirty data: Operation not permitted power8, Redhat Centos

2019-04-12 Thread Tom Lane
Andres Freund writes: > On 2019-04-12 20:04:00 +0200, reiner peterke wrote: >> We build Postgres on Power and x86 With the latest Postgres 11 release >> (11.2) we get error on >> power8 ppc64le (Redhat and CentOS). No error on SUSE on power8 > Any chance you can strace this? Because I don't

Re: Minimal logical decoding on standbys

2019-04-12 Thread Andres Freund
Hi, On 2019-04-12 23:34:02 +0530, Amit Khandekar wrote: > I tried to see if I can quickly understand what's going on. > > Here, master wal_level is hot_standby, not logical, though slave > wal_level is logical. Oh, that's well diagnosed. Cool. Also nicely tested - this'd be ugly in

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Fred .Flintstone
So there is no regression potential. When and who can send the patch to rename the programs to carry the pg_ prefixes, and create symlinks from the old names? On Fri, Apr 12, 2019 at 5:19 PM Andreas Karlsson wrote: > > On 4/12/19 5:14 PM, Chris Travers wrote: > > 1. naming things is

PANIC: could not flush dirty data: Operation not permitted power8, Redhat Centos

2019-04-12 Thread reiner peterke
Hi All, We build Postgres on Power and x86 With the latest Postgres 11 release (11.2) we get error on power8 ppc64le (Redhat and CentOS). No error on SUSE on power8 No error on x86_64 (RH, Centos and SUSE) from the log file 2019-04-09 12:30:10 UTC pid:203 xid:0 ip: LOG: listening on IPv4

Useless code in RelationCacheInitializePhase3

2019-04-12 Thread Tom Lane
While looking at the pending patch to clean up management of rd_partcheck, I noticed that RelationCacheInitializePhase3 has code that purports to reload rd_partkey and rd_partdesc, but none for rd_partcheck. However, that reload code is dead code, as is easily confirmed by checking the code

Re: Minimal logical decoding on standbys

2019-04-12 Thread Amit Khandekar
On Wed, 10 Apr 2019 at 21:39, Andres Freund wrote: > > Hi, > > On 2019-04-10 12:11:21 +0530, tushar wrote: > > > > On 03/13/2019 08:40 PM, tushar wrote: > > > Hi , > > > > > > I am getting a server crash on standby while executing > > > pg_logical_slot_get_changes function , please refer this

Re: Reducing the runtime of the core regression tests

2019-04-12 Thread Peter Geoghegan
On Fri, Apr 12, 2019 at 10:49 AM Peter Geoghegan wrote: > It's definitely generally recommended that "-O0" be used, so I think > that we can agree that that was an improvement, even if it doesn't fix > the remaining problem that I noticed when I rechecked nbtutils.c. I'm not sure that we can

Re: Fix doc bug in logical replication.

2019-04-12 Thread Robert Treat
On Mon, Apr 8, 2019 at 7:19 PM Euler Taveira wrote: > > Em seg, 8 de abr de 2019 às 19:38, Robert Treat escreveu: > > > > I noticed that the docs currently state "A different order of columns > > in the target table is allowed, but the column types have to match." > > This is untrue, as you can

Re: Reducing the runtime of the core regression tests

2019-04-12 Thread Peter Geoghegan
On Fri, Apr 12, 2019 at 10:24 AM Alvaro Herrera wrote: > I wonder if it would be useful to add --enable-debug. I think I > purposefully removed that, but I don't remember any details about it. As usual, this stuff is horribly under-documented. I think it's possible that --enable-debug would

Re: Reducing the runtime of the core regression tests

2019-04-12 Thread Alvaro Herrera
On 2019-Apr-12, Peter Geoghegan wrote: > On Fri, Apr 12, 2019 at 9:31 AM Alvaro Herrera > wrote: > > Done. Do you have a preferred spot where the counts were wrong? > > Not really, but I can give you an example. > > Line counts for each of the two "break" statements within >

Re: PostgreSQL pollutes the file system

2019-04-12 Thread John W Higgins
Could I please ask a couple of questions? Why does the first answer to everything seem to be "we need to destroy something to make it better for others"? Why does createdb need to be removed? Why do we use the "newbie that can't understand whether or not createdb is for PostgreSQL or MySQL or

Re: Reducing the runtime of the core regression tests

2019-04-12 Thread Peter Geoghegan
On Fri, Apr 12, 2019 at 9:31 AM Alvaro Herrera wrote: > Done. Do you have a preferred spot where the counts were wrong? Not really, but I can give you an example. Line counts for each of the two "break" statements within _bt_keep_natts_fast() are exactly the same. I don't think that this

Re: Attempt to consolidate reading of XLOG page

2019-04-12 Thread Alvaro Herrera
On 2019-Apr-11, Antonin Houska wrote: > While working on the instance encryption I found it annoying to apply > decyption of XLOG page to three different functions. Attached is a patch that > tries to merge them all into one function, XLogRead(). The existing > implementations differ in the way

TM format can mix encodings in to_char()

2019-04-12 Thread Juan José Santamaría Flecha
Hackers, I will use as an example the code in the regression test 'collate.linux.utf8'. There you can find: SET lc_time TO 'tr_TR'; SELECT to_char(date '2010-04-01', 'DD TMMON '); to_char - 01 NIS 2010 (1 row) The problem is that the locale 'tr_TR' uses the encoding

Re: Reducing the runtime of the core regression tests

2019-04-12 Thread Alvaro Herrera
On 2019-Apr-11, Peter Geoghegan wrote: > On Thu, Apr 11, 2019 at 11:00 AM Alvaro Herrera > wrote: > > ./configure --enable-depend --enable-coverage --enable-tap-tests > > --enable-nls --with-python --with-perl --with-tcl --with-openssl > > --with-libxml --with-ldap --with-pam >> $LOG 2>&1 > >

Re: jsonpath

2019-04-12 Thread Tom Lane
Andrew Dunstan writes: > OK, I have tried this with an earlier compiler (gcc 7.3.0 vs gcc 8.1.0) > and the problem disappears. So I think we can put this down as a > compiler bug. I will downgrade jacana to 7.3.0. OK. There is still the matter of those two failures on skate and snapper, which

Re: jsonpath

2019-04-12 Thread Andrew Dunstan
On 3/28/19 12:43 PM, Andrew Dunstan wrote: > On 3/28/19 9:50 AM, Tom Lane wrote: >> Andres Freund writes: >>> On March 28, 2019 9:31:14 AM EDT, Tom Lane wrote: Has anybody gotten through a valgrind run on this code yet? >>> Skink has successfully passed since - but that's x86... >> Yeah,

Re: Adding Unix domain socket path and port to pg_stat_get_wal_senders()

2019-04-12 Thread Tom Lane
Euler Taveira writes: > The question is: what is the problem we want to solve? Ishii-san asked > for a socket path. If we have already figured out the replica (via > application_name), use the replica PID to find the socket path. A new > column as suggested by Tom could show the desired info. Is

Re: improve PQexec documentation

2019-04-12 Thread Fabien COELHO
Hello Alvaro, I'm looking at psql's use of PQexec for implementing some feature. When running with multiple SQL commands, the doc is not very helpful. From the source code I gathered that PQexec returns the first COPY results if any, and if not the last non-empty results, unless all is

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-12 Thread Tom Lane
Michael Paquier writes: > On Wed, Apr 10, 2019 at 05:03:21PM +0900, Amit Langote wrote: >> The problem lies in all branches that have partitioning, so it should be >> listed under Older Bugs, right? You may have noticed that I posted >> patches for all branches down to 10. > I have noticed.

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Andreas Karlsson
On 4/12/19 5:14 PM, Chris Travers wrote: 1. naming things is surprisingly hard.  How sure are we that we can do this right?  Can we come up with a correct name for initdb?  Maybe pg_createcluster? The Debian packagers already use pg_createcluster for their script which wraps initdb, and

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Chris Travers
On Fri, Apr 12, 2019 at 3:20 PM Alvaro Herrera wrote: > On 2019-Apr-12, Daniel Gustafsson wrote: > > > There are many good reasons for the changes proposed in this thread, but > I'm > > not sure if discoverability is one. Relying on autocompleting a > filename to > > figure out existing tooling

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Fred .Flintstone
I think of discoverability as as how easy it is to discover and rediscover things. Like rediscover commands you forgot the name of. Like "what was the command to create a database?", just type pg_ and press tab and see whats there. The LWN article is now unlocked to all readers, not just paying

Re: Adding Unix domain socket path and port to pg_stat_get_wal_senders()

2019-04-12 Thread Euler Taveira
Em sex, 12 de abr de 2019 às 01:39, Michael Paquier escreveu: > > On Thu, Apr 11, 2019 at 10:19:01PM -0300, Euler Taveira wrote: > > application_name. I'm not sure if it solves your complain but Peter > > committed a patch [1] for v12 that distinguishes replicas in the same > > host via

Re: Adding Unix domain socket path and port to pg_stat_get_wal_senders()

2019-04-12 Thread Tatsuo Ishii
>> client_addr does not seem the right place to store this information, >> and it is already documented for years that NULL is used when using a >> Unix socket. But I think that we could change *client_hostname* so as >> the path name is reported instead of NULL when connecting through a >> Unix

Re: Calling pgstat_report_wait_end() before ereport(ERROR)

2019-04-12 Thread Tom Lane
Masahiko Sawada writes: > There are something like the following code in many places in PostgreSQL code. > ... > Since we eventually call > pgstat_report_wait_end() in AbortTransaction(). I think that we don't > need to call pgstat_report_wait_end() if we're going to raise an error > just after

Re: Checksum errors in pg_stat_database

2019-04-12 Thread Julien Rouhaud
On Fri, Apr 12, 2019 at 2:18 PM Magnus Hagander wrote: > > On Sun, Apr 7, 2019 at 6:28 PM Julien Rouhaud wrote: >> >> v5 attached. > > Thanks. Pushed! Thanks!

Re: Adding Unix domain socket path and port to pg_stat_get_wal_senders()

2019-04-12 Thread Tom Lane
Michael Paquier writes: > On Thu, Apr 11, 2019 at 10:19:01PM -0300, Euler Taveira wrote: >> Socket has different semantic from TCP/UDP. We can't add socket >> information into client_addr unless we are prepared to break this view >> (client_addr has type inet and it would be necessary to change

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Daniel Gustafsson
On Friday, April 12, 2019 3:20 PM, Alvaro Herrera wrote: > On 2019-Apr-12, Daniel Gustafsson wrote: > > > There are many good reasons for the changes proposed in this thread, but I'm > > not sure if discoverability is one. Relying on autocompleting a filename to > > figure out existing tooling

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Alvaro Herrera
On 2019-Apr-12, Daniel Gustafsson wrote: > There are many good reasons for the changes proposed in this thread, but I'm > not sure if discoverability is one. Relying on autocompleting a filename to > figure out existing tooling for database maintenance and DBA type operations > seems like a

Re: improve PQexec documentation

2019-04-12 Thread Alvaro Herrera
On 2019-Apr-12, Fabien COELHO wrote: > I'm looking at psql's use of PQexec for implementing some feature. > > When running with multiple SQL commands, the doc is not very helpful. > > From the source code I gathered that PQexec returns the first COPY results > if any, and if not the last

Re: Calling pgstat_report_wait_end() before ereport(ERROR)

2019-04-12 Thread Masahiko Sawada
On Fri, Apr 12, 2019 at 9:07 PM Michael Paquier wrote: > > On Fri, Apr 12, 2019 at 07:27:44PM +0900, Masahiko Sawada wrote: > > As far as I know there are three places where call > > pgstat_report_wait_end before ereport(ERROR): two in twophase.c > > andanother in copydir.c(at L199). Since we

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Fred .Flintstone
I would disagree. Discoverability is important, and having a user space that is intuitive and predictable. With the discoverability exposed by pg_ then you immediately find out what is available. One shouldn't have to delve down into manuals and books. Then forget what that darn command was next

Re: [PATCH v20] GSSAPI encryption support

2019-04-12 Thread Michael Paquier
On Fri, Apr 12, 2019 at 10:22:03AM +0200, Peter Eisentraut wrote: > Another problem is that the two test files cannot be run in parallel > because they use the same hardcoded data directories. That would have > to be replaced by temporary directories. Please note that I have added an open item

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Daniel Gustafsson
On Friday, April 12, 2019 2:25 PM, Fred .Flintstone wrote: > It would make the old commands more easily discoverable. Just type pg_ > and press the tab key for auto-completion. There are many good reasons for the changes proposed in this thread, but I'm not sure if discoverability is one.

Re: PostgreSQL pollutes the file system

2019-04-12 Thread Fred .Flintstone
It would make the old commands more easily discoverable. Just type pg_ and press the tab key for auto-completion. On Wed, Apr 10, 2019 at 9:46 PM Peter Eisentraut wrote: > > On 2019-04-10 15:01, Tatsuo Ishii wrote: > >> On 2019-03-29 20:32, Joe Conway wrote: > >>> pg_util > >> > >> How is

Re: Checksum errors in pg_stat_database

2019-04-12 Thread Magnus Hagander
On Sun, Apr 7, 2019 at 6:28 PM Julien Rouhaud wrote: > Thanks for looking it it! > > On Sun, Apr 7, 2019 at 4:36 PM Magnus Hagander > wrote: > > > > I'm not sure I like the idea of using "" as the database > name. It's not very likely that somebody would be using that as a name for > their

Re: Calling pgstat_report_wait_end() before ereport(ERROR)

2019-04-12 Thread Michael Paquier
On Fri, Apr 12, 2019 at 07:27:44PM +0900, Masahiko Sawada wrote: > As far as I know there are three places where call > pgstat_report_wait_end before ereport(ERROR): two in twophase.c > andanother in copydir.c(at L199). Since we eventually call > pgstat_report_wait_end() in AbortTransaction(). I

Re: REINDEX CONCURRENTLY 2.0

2019-04-12 Thread Dagfinn Ilmari Mannsåker
Michael Paquier writes: > So... I am coming up with the patch attached. I have introduced some > tests using a trick with CIC to have an invalid index to work on. I don't have any comments on the code (but the test looks sensible, it's the same trick I used to discover the issue in the first

Calling pgstat_report_wait_end() before ereport(ERROR)

2019-04-12 Thread Masahiko Sawada
Hi, There are something like the following code in many places in PostgreSQL code. pgstat_report_wait_start(WAIT_EVENT_xxx); if (write(...) != len) { ereport(ERROR, ...); } pgstat_report_wait_end(); Almost of these places don't call pgstat_report_wait_end() before ereport(ERROR) but some

Re: Transparent data encryption support as an extension

2019-04-12 Thread Masahiko Sawada
On Fri, Apr 12, 2019 at 6:34 PM Haribabu Kommi wrote: > > Hi Hackers, > > I read many mail discussions in supporting data at rest encryption support in > PostgreSQL. > > I checked the discussions around full instance encryption or tablespace or > table level encryption. In my observation, all the

Transparent data encryption support as an extension

2019-04-12 Thread Haribabu Kommi
Hi Hackers, I read many mail discussions in supporting data at rest encryption support in PostgreSQL. I checked the discussions around full instance encryption or tablespace or table level encryption. In my observation, all the proposals are trying to modify the core code to support encryption.

Re: Attempt to consolidate reading of XLOG page

2019-04-12 Thread Haribabu Kommi
On Fri, Apr 12, 2019 at 2:06 AM Antonin Houska wrote: > While working on the instance encryption I found it annoying to apply > decyption of XLOG page to three different functions. Attached is a patch > that > tries to merge them all into one function, XLogRead(). The existing > implementations

improve PQexec documentation

2019-04-12 Thread Fabien COELHO
Hello devs, I'm looking at psql's use of PQexec for implementing some feature. When running with multiple SQL commands, the doc is not very helpful. From the source code I gathered that PQexec returns the first COPY results if any, and if not the last non-empty results, unless all is empty

Re: [PATCH v20] GSSAPI encryption support

2019-04-12 Thread Peter Eisentraut
On 2019-04-09 09:32, Peter Eisentraut wrote: > On 2019-04-09 04:51, Stephen Frost wrote: >>> Running just 002_enc.pl by itself passes the tests! >> Great! I think what I'll do is work to incorporate the two tests back >> into one script, to avoid whatever the race condition or other confusion >>

Re: cache lookup failed for collation 0

2019-04-12 Thread Peter Eisentraut
On 2019-04-11 17:04, Jeevan Chalke wrote: > The error is coming from get_collation_isdeterministic() when colloid > passed is 0. I think like we do in get_collation_name(), we should > return false here when such collation oid does not exist. I'm not in favor of doing that. It would risk

Re: REINDEX CONCURRENTLY 2.0

2019-04-12 Thread Michael Paquier
On Fri, Apr 12, 2019 at 08:44:18AM +0200, Peter Eisentraut wrote: > Looks good, committed. Thanks for committing! -- Michael signature.asc Description: PGP signature

Re: REINDEX CONCURRENTLY 2.0

2019-04-12 Thread Peter Eisentraut
On 2019-04-11 05:59, Michael Paquier wrote: > On Tue, Apr 09, 2019 at 03:50:27PM +0900, Michael Paquier wrote: >> And here is the patch to address this issue. It happens that a bit >> more than the dependency switch was lacking here: >> - At swap time, we need to have the new index definition

Re: cache lookup failed for collation 0

2019-04-12 Thread Jeevan Chalke
On Thu, Apr 11, 2019 at 10:50 PM Tom Lane wrote: > Jeevan Chalke writes: > > Do you mean, the code in get_collation_isdeterministic() should look like > > something like below? > > > If colloid = InvalidOid then > > return TRUE > > ELSE IF tuple is valid then > > return collisdeterministic