Re: Internal key management system

2020-06-01 Thread Masahiko Sawada
On Sun, 31 May 2020 at 17:13, Fabien COELHO wrote: > > > Hello Masahiko-san, > > >> I am sharing here a document patch based on top of kms_v10 that was > >> shared awhile back. This document patch aims to cover more design > >> details of the current KMS design and to help people understand KMS >

Re: Inlining of couple of functions in pl_exec.c improves performance

2020-06-01 Thread Pavel Stehule
po 1. 6. 2020 v 8:15 odesílatel Amit Khandekar napsal: > On Sat, 30 May 2020 at 11:11, Pavel Stehule > wrote: > > I think so the effect of these patches strongly depends on CPU and > compile > > I quickly tried pi() with gcc 10 as well, and saw more or less the > same benefit. I think, we are

[POC] Fast COPY FROM command for the table with foreign partitions

2020-06-01 Thread Andrey Lepikhov
Hi, hackers! Currently i see, COPY FROM insertion into the partitioned table with foreign partitions is not optimal: even if table constraints allows can do multi insert copy, we will flush the buffers and prepare new INSERT query for each tuple, routed into the foreign partition. To solve

Re: OpenSSL 3.0.0 compatibility

2020-06-01 Thread Peter Eisentraut
On 2020-05-30 14:34, Andrew Dunstan wrote: On 5/28/20 6:16 PM, Daniel Gustafsson wrote: OpenSSL also deprecates DES keys in 3.0.0, which cause our password callback tests to fail with the cryptic error "fetch failed", as the test suite keys are encrypted with DES. 0002 fixes this by changing

Re: Inlining of couple of functions in pl_exec.c improves performance

2020-06-01 Thread Amit Khandekar
On Sat, 30 May 2020 at 11:11, Pavel Stehule wrote: > I think so the effect of these patches strongly depends on CPU and compile I quickly tried pi() with gcc 10 as well, and saw more or less the same benefit. I think, we are bound to see some differences in the benefits across architectures,

Re: Strange decreasing value of pg_last_wal_receive_lsn()

2020-06-01 Thread godjan •
Hi, sorry for 2 weeks latency in answer :) >> It fixed out trouble, but there is one another. Now we should wait when all >> ha alive hosts finish replaying WAL to failover. It might take a while(for >> example WAL contains wal_record about splitting b-tree). > > Indeed, this is the concern I

Re: Incorrect comment in be-secure-openssl.c

2020-06-01 Thread Daniel Gustafsson
> On 1 Jun 2020, at 08:06, Michael Paquier wrote: > The problem I have with first and second flavors is that "DH > parameters files" does not sound right. First, the grammar sounds > incorrect to me as in this case "parameters" should not be plural. I think "parameters" is the right term here,

Re: OpenSSL 3.0.0 compatibility

2020-06-01 Thread Daniel Gustafsson
> On 30 May 2020, at 11:29, Peter Eisentraut > wrote: > > On 2020-05-29 14:45, Daniel Gustafsson wrote: >>> I think we should set OPENSSL_API_COMPAT=10001, and move that along with >>> whatever our oldest supported release is going forward. That declares our >>> intention, it will silence

Re: Internal key management system

2020-06-01 Thread Masahiko Sawada
On Sat, 30 May 2020 at 04:20, Robert Haas wrote: > > On Fri, May 29, 2020 at 1:50 AM Masahiko Sawada > wrote: > > However, this usage has a downside that user secret can be logged to > > server logs when log_statement = 'all' or an error happens. To deal > > with this issue I've created a PoC

Re: Inlining of couple of functions in pl_exec.c improves performance

2020-06-01 Thread Amit Khandekar
On Sun, 31 May 2020 at 08:04, Michael Paquier wrote: > This stuff is interesting. Do you have some perf profiles to share? > I am wondering what's the effect of the inlining with your test > cases. Below are the perf numbers for asignmany.sql : HEAD : + 16.88% postgres postgres

Re: Incorrect comment in be-secure-openssl.c

2020-06-01 Thread Michael Paquier
On Sun, May 31, 2020 at 05:47:01PM -0400, Robert Haas wrote: > On Sun, May 31, 2020 at 2:54 AM Michael Paquier wrote: > I don't understand why that change is an improvement. Oops. I have managed to copy-paste an incorrect diff. The existing comment is that: * To prevent problems if the DH

Re: OpenSSL 3.0.0 compatibility

2020-06-01 Thread Peter Eisentraut
On 2020-05-31 04:52, Michael Paquier wrote: 593d4e4 claims that we only support OpenSSL >= 0.9.8, meaning that down to PG 10 we have this requirement, and that PG 9.6 and 9.5 should be able to work with OpenSSL 0.9.7 and 0.9.6, but little effort has been put in testing these. Then we can stick

Re: feature idea: use index when checking for NULLs before SET NOT NULL

2020-06-01 Thread John Bachir
> Maybe something else had a nontrivial lock on the table, and those commands > were waiting on lock. If you "SET deadlock_timeout='1'; SET > log_lock_waits=on;", then you could see that. Just checking - I think you mean lock_timeout? (although setting deadlock_timeout is also not a bad idea

Small doc improvement about spilled txn tracking

2020-06-01 Thread Masahiko Sawada
Hi all, When reading pg_stat_replication doc of PG13, I thought it's better to mention that tracking of spilled transactions works only for logical replication like we already mentioned about replication lag tracking: Lag times work automatically for physical replication. Logical decoding

Re: feature idea: use index when checking for NULLs before SET NOT NULL

2020-06-01 Thread Justin Pryzby
On Mon, Jun 01, 2020 at 10:49:25AM -0400, John Bachir wrote: > On Fri, May 29, 2020, at 10:10 PM, Justin Pryzby wrote: > > > If you do it right, you can see a DEBUG: > > postgres=# SET client_min_messages=debug; > > postgres=# ALTER TABLE tn ALTER i SET NOT NULL ; > > DEBUG: existing constraints

Re: feature idea: use index when checking for NULLs before SET NOT NULL

2020-06-01 Thread John Bachir
Thank you Justin for all that useful info! A couple nitpicky questions, so I can get my recipe right. On Mon, Jun 1, 2020, at 10:04 PM, Justin Pryzby wrote: > On Mon, Jun 01, 2020 at 10:49:25AM -0400, John Bachir wrote: > > Thanks! I'll add that to my recipe for the future. Although by that time

Re: SIGSEGV from START_REPLICATION 0/XXXXXXX in XLogSendPhysical () at walsender.c:2762

2020-06-01 Thread Michael Paquier
On Fri, May 29, 2020 at 06:09:06PM +0900, Masahiko Sawada wrote: > Yes. Conversely, if we start logical replication in a physical > replication connection (i.g. replication=true) we got an error before > staring replication: > > ERROR: logical decoding requires a database connection > > I think

Re: Read access for pg_monitor to pg_replication_origin_status view

2020-06-01 Thread Martín Marqués
Hi, Took me a bit longer than expected, but here is a new version, now with the idea of just removing the superuser() check and REVOKEing execution of the functions from public. At the end I grant permission to functions and the pg_replication_origin_status view. I wonder now if I needed to

Re: Just for fun: Postgres 20?

2020-06-01 Thread Robert Haas
On Wed, Feb 12, 2020 at 11:25 AM Juan José Santamaría Flecha wrote: > On Wed, Feb 12, 2020 at 3:47 PM Tom Lane wrote: >> Yeah; I don't think it's *that* unlikely for it to happen again. But >> my own principal concern about this mirrors what somebody else already >> pointed out: the

Re: Just for fun: Postgres 20?

2020-06-01 Thread Tom Lane
Robert Haas writes: > As has already been pointed out, it could definitely happen, but we > could solve that by just using a longer version number, say, including > the month and, in case we ever do multiple major releases in the same > month, also the day. In fact, we might as well take it one

Re: feature idea: use index when checking for NULLs before SET NOT NULL

2020-06-01 Thread John Bachir
On Fri, May 29, 2020, at 10:10 PM, Justin Pryzby wrote: > If you do it right, you can see a DEBUG: > postgres=# SET client_min_messages=debug; > postgres=# ALTER TABLE tn ALTER i SET NOT NULL ; > DEBUG: existing constraints on column "tn"."i" are sufficient to prove > that it does not

Re: Wrong width of UNION statement

2020-06-01 Thread Tom Lane
Kenichiro Tanaka writes: > I think table column width of UNION statement should be equal one of UNION > ALL. I don't buy that argument, because there could be type coercions involved, so that the result width isn't necessarily equal to any one of the inputs. Having said that, the example you

Re: Compatible defaults for LEAD/LAG

2020-06-01 Thread Tom Lane
Pavel Stehule writes: > po 1. 6. 2020 v 4:07 odesílatel Tom Lane napsal: >> That's just the tip of the iceberg, though. If you consider all the >> old-style polymorphic types, we have [for example] >> array_append(anyarray,anyelement) > I am not sure, if using anycompatible for buildin's array

Re: Small code cleanup

2020-06-01 Thread Tom Lane
Mark Dilger writes: > One line change to remove a duplicate check. The comment just above this mentions a connection to the "Finish printing the footer information about a table" stanza below. I think some work is needed to clarify what's going on there --- it doesn't seem actually buggy, but

Re: Compatible defaults for LEAD/LAG

2020-06-01 Thread Pavel Stehule
po 1. 6. 2020 v 17:36 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > po 1. 6. 2020 v 4:07 odesílatel Tom Lane napsal: > >> That's just the tip of the iceberg, though. If you consider all the > >> old-style polymorphic types, we have [for example] > >>

Re: OpenSSL 3.0.0 compatibility

2020-06-01 Thread Tom Lane
Andrew Dunstan writes: > On 6/1/20 8:03 AM, Daniel Gustafsson wrote: >> +1 for adding it to the Makefile. > OK, here's a patch. Likewise +1 for having it in the makefile. But now you have two copies, the other being in comments in the test script. The latter should go away, as we surely won't

Small code cleanup

2020-06-01 Thread Mark Dilger
One line change to remove a duplicate check. v1-0001-Code-cleanup.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Small code cleanup

2020-06-01 Thread Mark Dilger
> On Jun 1, 2020, at 9:59 AM, Tom Lane wrote: > > Mark Dilger writes: >> Yeah, I noticed the `git blame` last night when writing the patch that you >> had originally wrote the code around 2017, and that the duplication was >> introduced in a patch committed by others around 2018. I was

Re: Small code cleanup

2020-06-01 Thread Mark Dilger
> On Jun 1, 2020, at 8:53 AM, Tom Lane wrote: > > Mark Dilger writes: >> One line change to remove a duplicate check. > > The comment just above this mentions a connection to the "Finish printing > the footer information about a table" stanza below. I think some work is > needed to clarify

Re: Small code cleanup

2020-06-01 Thread Tom Lane
Mark Dilger writes: > Yeah, I noticed the `git blame` last night when writing the patch that you > had originally wrote the code around 2017, and that the duplication was > introduced in a patch committed by others around 2018. I was hoping that > you, as the original author, or somebody

Re: Inlining of couple of functions in pl_exec.c improves performance

2020-06-01 Thread Amit Khandekar
On Mon, 1 Jun 2020 at 12:27, Pavel Stehule wrote: > po 1. 6. 2020 v 8:15 odesílatel Amit Khandekar > napsal: >> >> On Sat, 30 May 2020 at 11:11, Pavel Stehule wrote: >> > I think so the effect of these patches strongly depends on CPU and compile >> >> I quickly tried pi() with gcc 10 as well,

Re: OpenSSL 3.0.0 compatibility

2020-06-01 Thread Andrew Dunstan
On 6/1/20 8:03 AM, Daniel Gustafsson wrote: >> On 1 Jun 2020, at 13:58, Andrew Dunstan >> wrote: >> If you want I can add a rule for it to the Makefile, although who knows >> what commands will actually apply when the certificate runs out? > Being able to easily regenerate the testdata,

Re: pg_dump dumps row level policies on extension tables

2020-06-01 Thread Masahiko Sawada
On Tue, 19 May 2020 at 15:31, Pavan Deolasee wrote: > > Hi, > > I noticed that if a row level policy is defined on an extension > object, even in the extension creation script, pg_dump dumps a > separate CREATE POLICY statement for such policies. That makes the > dump unrestorable because the

Re: OpenSSL 3.0.0 compatibility

2020-06-01 Thread Andrew Dunstan
On 6/1/20 4:33 AM, Peter Eisentraut wrote: > On 2020-05-30 14:34, Andrew Dunstan wrote: >> >> On 5/28/20 6:16 PM, Daniel Gustafsson wrote: >>> >>> OpenSSL also deprecates DES keys in 3.0.0, which cause our password >>> callback >>> tests to fail with the cryptic error "fetch failed", as the test

Re: OpenSSL 3.0.0 compatibility

2020-06-01 Thread Daniel Gustafsson
> On 1 Jun 2020, at 13:58, Andrew Dunstan > wrote: > If you want I can add a rule for it to the Makefile, although who knows > what commands will actually apply when the certificate runs out? Being able to easily regenerate the testdata, regardless of expiration status, has proven very helpful

Wrong width of UNION statement

2020-06-01 Thread Kenichiro Tanaka
Hello hackers I think I found a bug about estimating width of table column when I perform SQL with UNION statement. I think table column width of UNION statement should be equal one of UNION ALL. But they don't match.This can be reproduce it on HEAD. See following example. --CREATE TEST TABLE

Re: Another modest proposal for docs formatting: catalog descriptions

2020-06-01 Thread Jonathan S. Katz
On 6/1/20 6:57 PM, Tom Lane wrote: > =?UTF-8?B?Sm9zZWYgxaBpbcOhbmVr?= writes: >> I have spotted this change recently at progress monitoring devel docs ( >> https://www.postgresql.org/docs/devel/progress-reporting.html#CREATE-INDEX-PROGRESS-REPORTING). >> Current version seems a little chaotic

Re: Another modest proposal for docs formatting: catalog descriptions

2020-06-01 Thread Josef Šimánek
út 2. 6. 2020 v 0:30 odesílatel Tom Lane napsal: > As of HEAD, building the PDF docs for A4 paper draws 538 "contents > ... exceed the available area" warnings. While this is a nice step > forward from where we were (v12 has more than 1500 such warnings), > we're far from done fixing that

Re: Postgres Windows build system doesn't work with python installed in Program Files

2020-06-01 Thread David Zhang
Hi Michael, I performed a quick test for the path "msvc-build-init-v2.patch" using below cases: 1. perl build.pl 2. perl build.pl debug psql 3. perl build.pl RELEASE psql 4. perl build.pl deBUG psql 5. perl build.pl psql The above cases (case-insensitive) are all working great without any

Re: Speeding up parts of the planner using a binary search tree structure for nodes

2020-06-01 Thread David Rowley
On Sat, 30 May 2020 at 01:52, Ashutosh Bapat wrote: > > On Fri, May 29, 2020 at 10:47 AM David Rowley wrote: > > In [1] I mentioned that I'd like to look into coding a data structure > > to allow Node types to be looked up more efficiently than what List > > allows. There are many places in the

Re: Read access for pg_monitor to pg_replication_origin_status view

2020-06-01 Thread Martín Marqués
Hi, > Took me a bit longer than expected, but here is a new version, now > with the idea of just removing the superuser() check and REVOKEing > execution of the functions from public. At the end I grant permission > to functions and the pg_replication_origin_status view. > > I wonder now if I

Re: Another modest proposal for docs formatting: catalog descriptions

2020-06-01 Thread Tom Lane
=?UTF-8?B?Sm9zZWYgxaBpbcOhbmVr?= writes: > I have spotted this change recently at progress monitoring devel docs ( > https://www.postgresql.org/docs/devel/progress-reporting.html#CREATE-INDEX-PROGRESS-REPORTING). > Current version seems a little chaotic since there are multiple tables on > the

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-06-01 Thread Etsuro Fujita
Hi Andrey, On Mon, Jun 1, 2020 at 6:29 PM Andrey Lepikhov wrote: > Currently i see, COPY FROM insertion into the partitioned table with > foreign partitions is not optimal: even if table constraints allows can > do multi insert copy, we will flush the buffers and prepare new INSERT > query for

Re: Small doc improvement about spilled txn tracking

2020-06-01 Thread Amit Kapila
On Tue, Jun 2, 2020 at 9:10 AM Masahiko Sawada wrote: > > Hi all, > > When reading pg_stat_replication doc of PG13, I thought it's better to > mention that tracking of spilled transactions works only for logical > replication like we already mentioned about replication lag tracking: > > >

Re: Read access for pg_monitor to pg_replication_origin_status view

2020-06-01 Thread Kyotaro Horiguchi
Hi. At Mon, 1 Jun 2020 21:41:13 -0300, Martín Marqués wrote in > Hi, > > > Took me a bit longer than expected, but here is a new version, now > > with the idea of just removing the superuser() check and REVOKEing > > execution of the functions from public. At the end I grant permission > > to

Re: SIGSEGV from START_REPLICATION 0/XXXXXXX in XLogSendPhysical () at walsender.c:2762

2020-06-01 Thread Fujii Masao
On 2020/06/02 13:24, Michael Paquier wrote: On Fri, May 29, 2020 at 06:09:06PM +0900, Masahiko Sawada wrote: Yes. Conversely, if we start logical replication in a physical replication connection (i.g. replication=true) we got an error before staring replication: ERROR: logical decoding

Re: race condition when writing pg_control

2020-06-01 Thread Michael Paquier
On Sun, May 31, 2020 at 09:11:35PM +, Bossart, Nathan wrote: > Thanks for the feedback. I've attached a new set of patches. Thanks for splitting the set. 0001 and 0002 are the minimum set for back-patching, and it would be better to merge them together. 0003 is debatable and not an actual

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-06-01 Thread Andrey Lepikhov
Thank you for the answer, 02.06.2020 05:02, Etsuro Fujita пишет: I think I also thought something similar to this before [1]. Will take a look. [1] https://www.postgresql.org/message-id/23990375-45a6-5823-b0aa-a6a7a6a957f0%40lab.ntt.co.jp I have looked into the thread. My first version

Re: Small doc improvement about spilled txn tracking

2020-06-01 Thread Amit Kapila
On Tue, Jun 2, 2020 at 10:22 AM Amit Kapila wrote: > > On Tue, Jun 2, 2020 at 9:10 AM Masahiko Sawada > wrote: > > > > > Please find attached patch. > > > > On a quick look, it seems fine but I will look in more detail and let > you know if I have any feedback. > I am not sure if we need to add

Re: Read access for pg_monitor to pg_replication_origin_status view

2020-06-01 Thread Michael Paquier
On Mon, Jun 01, 2020 at 03:38:07PM -0300, Martín Marqués wrote: > Took me a bit longer than expected, but here is a new version, now > with the idea of just removing the superuser() check and REVOKEing > execution of the functions from public. At the end I grant permission > to functions and the