Re: rand48 replacement

2021-09-25 Thread Fabien COELHO
Just FTR, I strongly object to your removal of process-startup srandom() calls. Ok. The point of the patch is to replace and unify the postgres underlying PRNG, so there was some logic behind this removal. FTR, this was triggered by your comment on Jul 1: [...] I see that you probably

Re: Gather performance analysis

2021-09-25 Thread Dilip Kumar
On Sat, Sep 25, 2021 at 2:18 AM Tomas Vondra wrote: > > On 9/24/21 7:08 PM, Robert Haas wrote: > > On Fri, Sep 24, 2021 at 3:50 AM Dilip Kumar wrote: > >> Tomas, can you share your test script, I would like to repeat the same > >> test in my environment and with different batching sizes. For

Re: extended stats on partitioned tables

2021-09-25 Thread Justin Pryzby
(Resending with -hackers) It seems like your patch should also check "inh" in examine_variable and statext_expressions_load. Which leads to another issue in stable branches: ANALYZE builds only non-inherited stats, but they're incorrectly used for inherited queries - the rowcount estimate is

Re: extended stats on partitioned tables

2021-09-25 Thread Justin Pryzby
On Sat, Sep 25, 2021 at 11:01:21PM +0200, Tomas Vondra wrote: > > Do you think it's possible to backpatch a fix to handle partitioned tables > > specifically ? > > > > The "tuple already updated" error which I reported and which was fixed by > > 859b3003 involved inheritence children. Since

Re: extended stats on partitioned tables

2021-09-25 Thread Tomas Vondra
On 9/25/21 9:53 PM, Justin Pryzby wrote: On Sat, Sep 25, 2021 at 09:27:10PM +0200, Tomas Vondra wrote: On 9/23/21 11:26 PM, Justin Pryzby wrote: extended stats objects are allowed on partitioned tables since v10.

Re: rand48 replacement

2021-09-25 Thread Fabien COELHO
Hello Tom, Just FTR, I strongly object to your removal of process-startup srandom() calls. Ok. The point of the patch is to replace and unify the postgres underlying PRNG, so there was some logic behind this removal. Those are not only setting the seed for our own use, but also ensuring

Re: logical decoding and replication of sequences

2021-09-25 Thread Hannu Krosing
Just a note for some design decisions > 1) By default, sequences are treated non-transactionally, i.e. sent to the > output plugin right away. If our aim is just to make sure that all user-visible data in *transactional* tables is consistent with sequence state then one very much simplified

Re: extended stats on partitioned tables

2021-09-25 Thread Justin Pryzby
On Sat, Sep 25, 2021 at 09:27:10PM +0200, Tomas Vondra wrote: > On 9/23/21 11:26 PM, Justin Pryzby wrote: > > extended stats objects are allowed on partitioned tables since v10. > > https://www.postgresql.org/message-id/flat/CAKJS1f-BmGo410bh5RSPZUvOO0LhmHL2NYmdrC_Jm8pk_FfyCA%40mail.gmail.com > >

Re: logical replication restrictions

2021-09-25 Thread Hannu Krosing
On Wed, Sep 22, 2021 at 6:18 AM Amit Kapila wrote: > > On Tue, Sep 21, 2021 at 4:21 PM Marcos Pegoraro wrote: >> >> No, I´m talking about that configuration you can have on standby servers >> recovery_min_apply_delay = '8h' >> > > oh okay, I think this can be useful in some cases where we want

Re: extended stats on partitioned tables

2021-09-25 Thread Tomas Vondra
On 9/23/21 11:26 PM, Justin Pryzby wrote: extended stats objects are allowed on partitioned tables since v10. https://www.postgresql.org/message-id/flat/CAKJS1f-BmGo410bh5RSPZUvOO0LhmHL2NYmdrC_Jm8pk_FfyCA%40mail.gmail.com 8c5cdb7f4f6e1d6a6104cb58ce4f23453891651b But since 859b3003de they're not

Re: OpenSSL 3.0.0 compatibility

2021-09-25 Thread Daniel Gustafsson
> On 25 Sep 2021, at 15:45, Tom Lane wrote: > > Daniel Gustafsson writes: >>> On 25 Sep 2021, at 12:03, Michael Paquier wrote: >>> As 9.6 will be EOL'd in a couple of weeks, is that really >>> worth the effort though? It sounds risky to me to introduce an >>> invasive change as that would

Re: Fixing WAL instability in various TAP tests

2021-09-25 Thread Mark Dilger
> On Sep 25, 2021, at 9:00 AM, Noah Misch wrote: > >> You may be right, but the conversation about "all possible settings" was >> started by Noah. > > You wrote, "I would expect tests which fail under legal alternate GUC settings > to be hardened to explicitly set the GUCs as they need,

Re: rand48 replacement

2021-09-25 Thread Tom Lane
Fabien COELHO writes: > Not enough. Here is a v14 which might improve things further more again. > Sorry for this noise due to blind windows tests. Just FTR, I strongly object to your removal of process-startup srandom() calls. Those are not only setting the seed for our own use, but also

Re: rand48 replacement

2021-09-25 Thread Fabien COELHO
[1]: http://cfbot.cputube.org/ Indeed. I wish that these results would be available from the cf interface. Attached a v11 which might improve things. Not enough. Here is a v12 which might improve things further. Not enough. Here is a v13 which might improve things further more. Not

Re: rand48 replacement

2021-09-25 Thread Fabien COELHO
[1]: http://cfbot.cputube.org/ Indeed. I wish that these results would be available from the cf interface. Attached a v11 which might improve things. Not enough. Here is a v12 which might improve things further. Not enough. Here is a v13 which might improve things further more. --

Re: Fixing WAL instability in various TAP tests

2021-09-25 Thread Tom Lane
Noah Misch writes: > On Sat, Sep 25, 2021 at 08:20:06AM -0700, Mark Dilger wrote: >> You may be right, but the conversation about "all possible settings" was >> started by Noah. > You wrote, "I would expect tests which fail under legal alternate GUC settings > to be hardened to explicitly set

Re: Fixing WAL instability in various TAP tests

2021-09-25 Thread Noah Misch
On Sat, Sep 25, 2021 at 08:20:06AM -0700, Mark Dilger wrote: > > On Sep 25, 2021, at 7:17 AM, Tom Lane wrote: > >> Leaving the tests brittle wastes developer time. > > > > Trying to make them proof against all possible settings would waste > > a lot more time, though. > > You may be right, but

Re: Fixing WAL instability in various TAP tests

2021-09-25 Thread Tom Lane
Mark Dilger writes: > You may be right, but the conversation about "all possible settings" was > started by Noah. I was really just talking about tests that depend on wal > files not being removed, but taking no action to guarantee that, merely > trusting that under default settings they

Re: Release 14 Schedule

2021-09-25 Thread Tom Lane
Robert Haas writes: > Still, this kind of seems like a laundry list to me. I'd argue for > cutting range types, extended statistics, SEARCH / CYCLE, TOAST, and > emergency mode vacuum. They're all nice, and I'm glad we have them, > but they're also things that only people who are deeply steeped

Re: Fixing WAL instability in various TAP tests

2021-09-25 Thread Mark Dilger
> On Sep 25, 2021, at 7:17 AM, Tom Lane wrote: > >> Leaving the tests brittle wastes developer time. > > Trying to make them proof against all possible settings would waste > a lot more time, though. You may be right, but the conversation about "all possible settings" was started by Noah.

Re: BUG #16583: merge join on tables with different DB collation behind postgres_fdw fails

2021-09-25 Thread Tom Lane
Etsuro Fujita writes: > On Sat, Sep 25, 2021 at 4:11 AM Tom Lane wrote: >> As a short-term answer, I propose that we apply (and back-patch) the >> attached documentation changes. > The attached patch looks good to me. I've pushed that, and marked the current CF entry as returned with feedback.

Re: rand48 replacement

2021-09-25 Thread Fabien COELHO
Hello again, Just wanted to let you know that cfbot [1] doesn't seem to be happy with the patch. Apparently, some tests are falling. To be honest, I didn't invest too much time into investigating this. Hopefully, it's not a big deal. [1]: http://cfbot.cputube.org/ Indeed. I wish that these

Re: Fixing WAL instability in various TAP tests

2021-09-25 Thread Tom Lane
Mark Dilger writes: >> On Sep 24, 2021, at 10:21 PM, Noah Misch wrote: >>> I would >>> expect tests which fail under legal alternate GUC settings to be hardened to >>> explicitly set the GUCs as they need, rather than implicitly relying on the >>> defaults. >> That is not the general practice

Re: Fixing WAL instability in various TAP tests

2021-09-25 Thread Mark Dilger
> On Sep 24, 2021, at 10:21 PM, Noah Misch wrote: > >> I would >> expect tests which fail under legal alternate GUC settings to be hardened to >> explicitly set the GUCs as they need, rather than implicitly relying on the >> defaults. > > That is not the general practice in PostgreSQL tests

Re: BUG #16583: merge join on tables with different DB collation behind postgres_fdw fails

2021-09-25 Thread Tom Lane
Etsuro Fujita writes: > On Sat, Sep 25, 2021 at 4:11 AM Tom Lane wrote: >> Longer-term, it seems like we really have to be able to represent >> the notion of a remote column that has an "unknown" collation (that >> is, one that doesn't match any local collation, or at least is not >> known to do

Re: BUG #16583: merge join on tables with different DB collation behind postgres_fdw fails

2021-09-25 Thread Etsuro Fujita
On Sat, Sep 25, 2021 at 4:11 AM Tom Lane wrote: > Etsuro Fujita writes: > > One thing I noticed is that collatable operators/functions sent to the > > remote might also cause an unexpected result when the default > > collations are not compatible. Consider this example (even with your > >

Re: OpenSSL 3.0.0 compatibility

2021-09-25 Thread Tom Lane
Daniel Gustafsson writes: >> On 25 Sep 2021, at 12:03, Michael Paquier wrote: >> As 9.6 will be EOL'd in a couple of weeks, is that really >> worth the effort though? It sounds risky to me to introduce an >> invasive change as that would increase the risk of bugs for existing >> users. So my

Re: pgcrypto support for bcrypt $2b$ hashes

2021-09-25 Thread Daniel Gustafsson
> On 24 Sep 2021, at 04:12, Daniel Fone wrote: > At the moment, pgcrypto’s `crypt` doesn’t recognise this prefix. However, > simply `replace`ing the prefix with $2a$ allows crypt to validate the hashes. > This patch simply adds recognition for the prefix and treats the hash > identically to

Re: prevent immature WAL streaming

2021-09-25 Thread Alvaro Herrera
On 2021-Sep-24, Alvaro Herrera wrote: > Here's the set for all branches, which I think are really final, in case > somebody wants to play and reproduce their respective problem scenarios. I forgot to mention that I'll wait until 14.0 is tagged before getting anything pushed. -- Álvaro Herrera

Re: Timeout failure in 019_replslot_limit.pl

2021-09-25 Thread Amit Kapila
On Wed, Sep 22, 2021 at 12:57 PM Michael Paquier wrote: > > On Mon, Sep 20, 2021 at 09:38:29AM -0300, Alvaro Herrera wrote: > > On 2021-Sep-20, Michael Paquier wrote: > >> The test gets the right PIDs, as the logs showed: > >> ok 17 - have walsender pid 12663 > >> ok 18 - have walreceiver pid

Re: OpenSSL 3.0.0 compatibility

2021-09-25 Thread Daniel Gustafsson
> On 25 Sep 2021, at 12:03, Michael Paquier wrote: > As 9.6 will be EOL'd in a couple of weeks, is that really > worth the effort though? It sounds risky to me to introduce an > invasive change as that would increase the risk of bugs for existing > users. So my vote would be to just let this

Re: row filtering for logical replication

2021-09-25 Thread Tomas Vondra
On 9/25/21 6:23 AM, Amit Kapila wrote: On Sat, Sep 25, 2021 at 3:30 AM Tomas Vondra wrote: On 9/24/21 7:20 AM, Amit Kapila wrote: I think the right way to support functions is by the explicit marking of functions and in one of the emails above Jeff Davis also agreed with the same. I

Re: OpenSSL 3.0.0 compatibility

2021-09-25 Thread Michael Paquier
On Sat, Sep 25, 2021 at 11:55:03AM +0200, Daniel Gustafsson wrote: > These have now been pushed to 14 through to 10 ahead of next week releases, I > will keep an eye on caiman as it builds these branches. The OpenSSL support > in > 9.6 pgcrypto isn't using the EVP API (committed in 5ff4a67f63)

Re: OpenSSL 3.0.0 compatibility

2021-09-25 Thread Daniel Gustafsson
> On 23 Sep 2021, at 23:41, Daniel Gustafsson wrote: > Thanks for confirming, I will go ahead and do that. These have now been pushed to 14 through to 10 ahead of next week releases, I will keep an eye on caiman as it builds these branches. The OpenSSL support in 9.6 pgcrypto isn't using the

Re: psql - add SHOW_ALL_RESULTS option

2021-09-25 Thread Fabien COELHO
Hallo Peter, It turns out that your v8 patch still has the issue complained about in [0]. The issue is that after COMMIT AND CHAIN, the internal savepoint is gone, but the patched psql still thinks it should be there and tries to release it, which leads to errors. Indeed. Thanks for the

Re: Column Filtering in Logical Replication

2021-09-25 Thread vignesh C
On Fri, Sep 24, 2021 at 6:55 PM Alvaro Herrera wrote: > > On 2021-Sep-23, Amit Kapila wrote: > > > Alvaro, do you have any thoughts on these proposed grammar changes? > > Yeah, I think pubobj_name remains a problem in that you don't know its > return type -- could be a String or a RangeVar, and

Re: rand48 replacement

2021-09-25 Thread Fabien COELHO
Hello Aleksander, Attached a v10 which is some kind of compromise where the interface uses inclusive min and max bounds, so that all values can be reached. Just wanted to let you know that cfbot [1] doesn't seem to be happy with the patch. Apparently, some tests are falling. To be honest, I

Re: Skipping logical replication transactions on subscriber side

2021-09-25 Thread Amit Kapila
On Fri, Sep 24, 2021 at 6:44 PM Masahiko Sawada wrote: > > On Fri, Sep 24, 2021 at 8:01 PM Amit Kapila wrote: > > > > > 6. > > +typedef struct PgStat_StatSubEntry > > +{ > > + Oid subid; /* hash table key */ > > + > > + /* > > + * Statistics of errors that occurred during logical replication.

Re: POC: Cleaning up orphaned files using undo logs

2021-09-25 Thread Amit Kapila
On Fri, Sep 24, 2021 at 4:44 PM Antonin Houska wrote: > > Amit Kapila wrote: > > > On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote: > > > > > > Amit Kapila wrote: > > > > > > > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> > > > > wrote: > > > > > > > > > > > On