Re: Refactor "mutually exclusive options" error reporting code in parse_subscription_options

2021-06-14 Thread Kyotaro Horiguchi
At Fri, 11 Jun 2021 16:29:10 -0400, Robert Haas wrote in > On Tue, May 25, 2021 at 9:38 AM Alvaro Herrera > wrote: > > This should be okay, right? Well, almost. The problem here is if you > > want to have a variable where you set more than one option, you have to > > use bit-and of the enum v

Re: Question about StartLogicalReplication() error path

2021-06-14 Thread Kyotaro Horiguchi
At Mon, 14 Jun 2021 14:51:35 -0700, Jeff Davis wrote in > On Mon, 2021-06-14 at 13:13 -0400, Robert Haas wrote: > > I'm happy to hear other opinions, but I think I would be inclined to > > vote in favor of #1 and/or #2 but against #3. > > What about upgrading it to, say, LOG? It seems like it wo

Re: Race condition in recovery?

2021-06-14 Thread Kyotaro Horiguchi
At Fri, 11 Jun 2021 10:46:45 -0400, Tom Lane wrote in > I think jacana uses msys[2?], so this likely indicates a problem > in path sanitization for the archive command. Andrew, any advice? Thanks for fixing it. # I haven't still succeed to run TAP tests on MSYS2 environment. I # cannot install

Re: Different compression methods for FPI

2021-06-14 Thread Peter Eisentraut
On 15.06.21 07:37, Michael Paquier wrote: Actually, I was just thinking that default yes/no/on/off stuff maybe should be defined to mean "lz4" rather than meaning pglz for "backwards compatibility". +1 I am not sure that we have any reasons to be that aggressive about this one either, and this

Re: Different compression methods for FPI

2021-06-14 Thread Michael Paquier
On Tue, Jun 15, 2021 at 08:08:54AM +0300, Heikki Linnakangas wrote: > On 15/06/2021 06:42, Justin Pryzby wrote: >> Why ? This is WAL, not table data. WAL depends on the major version, so >> I think wal_compression could provide a different set of compression methods >> at >> every major release?

Re: Use singular number when appropriate

2021-06-14 Thread Julien Rouhaud
On Tue, Jun 15, 2021 at 04:59:24AM +, David Fetter wrote: > Hi, > > I thought about using the dual, but wasn't sure how many languages > support it. I don't think that you can assume that appending something will work in all languages. Note that it also doesn't always work in english (e.g. t

Re: Error on pgbench logs

2021-06-14 Thread Michael Paquier
On Sun, Jun 13, 2021 at 03:07:51AM +0900, Yugo NAGATA wrote: > It will not work if the transaction is skipped, in which case latency is 0.0. > It would work if we check also "skipped" as bellow. > > + if (!logged && !skipped && latency == 0.0) > > However, it still might not work if t

Re: Use singular number when appropriate

2021-06-14 Thread Heikki Linnakangas
On 15/06/2021 07:59, David Fetter wrote: Hi, I thought about using the dual, but wasn't sure how many languages support it. if (fail_count == 0 && fail_ignore_count == 0) snprintf(buf, sizeof(buf), _(" %s %d test%s passed. "),

Re: Different compression methods for FPI

2021-06-14 Thread Heikki Linnakangas
On 15/06/2021 06:42, Justin Pryzby wrote: On Tue, Jun 15, 2021 at 11:39:24AM +0900, Michael Paquier wrote: On Mon, Jun 14, 2021 at 08:42:08PM -0500, Justin Pryzby wrote: It's USERSET following your own suggestion (which is a good suggestion): + {"wal_compression_method", PGC_SIGHUP, WAL_

Use singular number when appropriate

2021-06-14 Thread David Fetter
Hi, I thought about using the dual, but wasn't sure how many languages support it. Best, David. -- David Fetter http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate >From 1fd595576c5972d8604adf77d1959008daebacb0 Mon S

Re: locking [user] catalog tables vs 2pc vs logical rep

2021-06-14 Thread vignesh C
On Mon, Jun 14, 2021 at 5:33 PM osumi.takami...@fujitsu.com wrote: > > On Friday, June 11, 2021 2:13 PM vignesh C wrote: > > Thanks for the updated patch: > > Few comments: > > 1) We have used Reordering and Clustering for the same command, we could > > rephrase similarly in both places. > > +

Re: RFC: Logging plan of the running query

2021-06-14 Thread Bharath Rupireddy
On Mon, Jun 14, 2021 at 5:48 PM torikoshia wrote: > > 1) We could just say "Requests to log query plan of the presently > > running query of a given backend along with an untruncated query > > string in the server logs." > > Instead of > > +They will be logged at LOG message level > > and

Re: Different compression methods for FPI

2021-06-14 Thread Justin Pryzby
On Tue, Jun 15, 2021 at 11:39:24AM +0900, Michael Paquier wrote: > On Mon, Jun 14, 2021 at 08:42:08PM -0500, Justin Pryzby wrote: It's USERSET following your own suggestion (which is a good suggestion): > >> + {"wal_compression_method", PGC_SIGHUP, WAL_SETTINGS, > >> + gettext_

Re: PG 14 release notes, first draft

2021-06-14 Thread Michael Paquier
On Tue, Jun 15, 2021 at 11:49:21AM +0900, Masahiko Sawada wrote: > On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian wrote: >> OK, but I need more information on how users will see a difference based >> on this commit: +1. That would be good to have in the release notes. > I think that since with

Improving the isolationtester: fewer failures, less delay

2021-06-14 Thread Tom Lane
This is a followup to the conversation at [1], in which we speculated about constraining the isolationtester's behavior by annotating the specfiles, in order to eliminate common buildfarm failures such as [2] and reduce the need to use long delays to stabilize the test results. I've spent a couple

Re: Isolation tests vs. SERIALIZABLE isolation level

2021-06-14 Thread Thomas Munro
On Tue, Jun 15, 2021 at 2:09 PM Tom Lane wrote: > * Do we still care about that policy? > * If so, who's going to fix the above-listed problems? > * Should we try to get some routine testing of this case > in place? I wondered the same in commit 37929599 (the same problem for src/test/regress,

Re: PG 14 release notes, first draft

2021-06-14 Thread Masahiko Sawada
On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian wrote: > > On Tue, Jun 15, 2021 at 10:06:49AM +0900, Masahiko Sawada wrote: > > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian wrote: > > > > > > I have committed the first draft of the PG 14 release notes. You can > > > see the most current build o

Re: Duplicate history file?

2021-06-14 Thread Julien Rouhaud
On Tue, Jun 15, 2021 at 10:20:37AM +0900, Kyotaro Horiguchi wrote: > > Actually there's large room for losing data with cp. Yes, we would > need additional redundancy of storage and periodical integrity > inspection of the storage and archives on maybe need copies at the > other sites on the other

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-06-14 Thread Peter Geoghegan
On Mon, Jun 14, 2021 at 5:23 PM Michael Paquier wrote: > > *Why* does it have to work at the system level? I don't understand > > what you mean about the system level. > > I mean that you lack a GUC that allows to enforce to *not* use this > optimization for all relations, for all processes. You'

Re: Different compression methods for FPI

2021-06-14 Thread Michael Paquier
On Mon, Jun 14, 2021 at 08:42:08PM -0500, Justin Pryzby wrote: > On Tue, Jun 15, 2021 at 09:50:41AM +0900, Michael Paquier wrote: >> + {"wal_compression_method", PGC_SIGHUP, WAL_SETTINGS, >> + gettext_noop("Set the method used to compress full page images >> in the WAL."), >> +

Isolation tests vs. SERIALIZABLE isolation level

2021-06-14 Thread Tom Lane
In the past people have tried to ensure that the isolation tests would pass regardless of the prevailing default_transaction_isolation setting. (That was sort of the point, in fact, for the earliest tests using that infrastructure.) This seems to have been forgotten about lately, as all of these

Re: Different compression methods for FPI

2021-06-14 Thread Justin Pryzby
On Tue, Jun 15, 2021 at 09:50:41AM +0900, Michael Paquier wrote: > On Sun, Jun 13, 2021 at 08:24:12PM -0500, Justin Pryzby wrote: > > I think it's more nuanced than just finding the algorithm with the least CPU > > use. The GUC is PGC_USERSET, and it's possible that a data-loading process > > migh

Re: Delegating superuser tasks to new security roles

2021-06-14 Thread torikoshia
On 2021-06-14 23:53, Mark Dilger wrote: On Jun 14, 2021, at 5:51 AM, torikoshia wrote: Thanks for working on this topic, I appreciate it! Thank you for taking a look! BTW, do these patches enable non-superusers to create user with bypassrls? No, I did not break out the ability to create

Re: Duplicate history file?

2021-06-14 Thread Kyotaro Horiguchi
At Fri, 11 Jun 2021 16:08:33 +0900, Michael Paquier wrote in > On Fri, Jun 11, 2021 at 03:32:28PM +0900, Kyotaro Horiguchi wrote: > > I think cp can be an example as far as we explain the limitations. (On > > the other hand "test !-f" cannot since it actually prevents server > > from working cor

Re: PG 14 release notes, first draft

2021-06-14 Thread Bruce Momjian
On Tue, Jun 15, 2021 at 10:06:49AM +0900, Masahiko Sawada wrote: > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian wrote: > > > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-14 Thread Andres Freund
Hi, > @@ -4032,6 +4039,24 @@ GlobalVisTestShouldUpdate(GlobalVisState *state) > static void > GlobalVisUpdateApply(ComputeXidHorizonsResult *horizons) > { > + /* assert non-decreasing nature of horizons */ > + Assert(FullTransactionIdFollowsOrEquals( > +FullXidRe

Re: Duplicate history file?

2021-06-14 Thread Kyotaro Horiguchi
At Fri, 11 Jun 2021 15:18:03 +0800, Julien Rouhaud wrote in > On Fri, Jun 11, 2021 at 03:32:28PM +0900, Kyotaro Horiguchi wrote: > I disagree, cp is probably the worst command that can be used for this > purpose. > On top on the previous problems already mentioned, you also have the fact that >

Re: PG 14 release notes, first draft

2021-06-14 Thread Masahiko Sawada
On Mon, May 10, 2021 at 3:03 PM Bruce Momjian wrote: > > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > It might have already been discussed but I think we should mention

Re: Different compression methods for FPI

2021-06-14 Thread Andres Freund
Hi, On 2021-06-15 09:50:41 +0900, Michael Paquier wrote: > On Sun, Jun 13, 2021 at 08:24:12PM -0500, Justin Pryzby wrote: > > I think it's more nuanced than just finding the algorithm with the least CPU > > use. The GUC is PGC_USERSET, and it's possible that a data-loading process > > might want

Re: Different compression methods for FPI

2021-06-14 Thread Michael Paquier
On Sun, Jun 13, 2021 at 08:24:12PM -0500, Justin Pryzby wrote: > I think it's more nuanced than just finding the algorithm with the least CPU > use. The GUC is PGC_USERSET, and it's possible that a data-loading process > might want to use zlib for better compress ratio, but an interactive OLTP > p

Re: Skipping logical replication transactions on subscriber side

2021-06-14 Thread Masahiko Sawada
On Wed, Jun 2, 2021 at 3:07 PM Amit Kapila wrote: > > On Tue, Jun 1, 2021 at 9:05 PM Peter Eisentraut > wrote: > > > > On 01.06.21 06:01, Amit Kapila wrote: > > > But, won't that be costly in cases where we have errors in the > > > processing of very large transactions? Subscription has to proces

Re: A new function to wait for the backend exit after termination

2021-06-14 Thread Noah Misch
On Mon, Jun 14, 2021 at 07:34:59PM +0530, Bharath Rupireddy wrote: > Thanks. +1 to remove the pg_wait_for_backend_termination function. The > patch basically looks good to me. I'm attaching an updated patch. I > corrected a minor typo in the commit message, took docs and code > comment changes sugg

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-06-14 Thread Michael Paquier
On Fri, Jun 11, 2021 at 02:46:20PM -0700, Peter Geoghegan wrote: > On Thu, Jun 3, 2021 at 11:15 PM Michael Paquier wrote: >> I have read through the patch, and I am surprised to see that this >> only makes possible to control the optimization at relation level. >> The origin of the complaints is t

Re: Signed vs Unsigned (take 2) (src/backend/storage/ipc/procarray.c)

2021-06-14 Thread Ranier Vilela
I took it a step further. Transactions HEAD patched 1000220710586781 1014616710388685 100489191059 10065764,333 10436275 3,55021946687555 TPS HEAD patched 33469,016009 35399

Re: [PATCH] Fix select from wrong table array_op_test

2021-06-14 Thread Jason Kim
On 2021-06-11T19:00:41+0300, Heikki Linnakangas wrote: > We already have these same queries in the 'arrays' test against the > 'array_op_test' table, though. It sure looks like a copy-paste error to me > as well. That's reason enough, but another reason is that I don't think GIN_CAT_NULL_KEY is co

Re: Support for NSS as a libpq TLS backend

2021-06-14 Thread Jacob Champion
On Fri, 2021-05-28 at 11:04 +0200, Daniel Gustafsson wrote: > Attached is a rebase to keep bitrot at bay. I get a failure during one of the CRL directory tests due to a missing database -- it looks like the Makefile is missing an entry. (I'm dusting off my build after a few months away, so I don't

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-14 Thread Andres Freund
Hi, On 2021-06-14 11:53:47 +0200, Matthias van de Meent wrote: > On Thu, 10 Jun 2021 at 19:43, Peter Geoghegan wrote: > > > > On Thu, Jun 10, 2021 at 10:29 AM Matthias van de Meent > > wrote: > > > I see one exit for HEAPTUPLE_DEAD on a potentially recently committed > > > xvac (?), and we might

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Andres Freund
Hi, On 2021-06-14 23:20:47 +0200, Jehan-Guillaume de Rorthais wrote: > > On 2021-06-14 16:10:32 +0200, Jehan-Guillaume de Rorthais wrote: > > > In the patch in attachment, I tried to fix this by using kind of an > > > internal > > > hook for pgstat_report_wait_start and pgstat_report_wait_end. Th

Re: Question about StartLogicalReplication() error path

2021-06-14 Thread Jeff Davis
On Mon, 2021-06-14 at 13:13 -0400, Robert Haas wrote: > I'm happy to hear other opinions, but I think I would be inclined to > vote in favor of #1 and/or #2 but against #3. What about upgrading it to, say, LOG? It seems like it would happen pretty infrequently, and in the event something strange h

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Jehan-Guillaume de Rorthais
Hi, On Mon, 14 Jun 2021 11:27:21 -0700 Andres Freund wrote: > On 2021-06-14 16:10:32 +0200, Jehan-Guillaume de Rorthais wrote: > > In the patch in attachment, I tried to fix this by using kind of an internal > > hook for pgstat_report_wait_start and pgstat_report_wait_end. This allows to > > "in

Re: a path towards replacing GEQO with something better

2021-06-14 Thread Bruce Momjian
On Mon, Jun 14, 2021 at 06:10:28PM +0200, Tomas Vondra wrote: > On 6/14/21 1:16 PM, John Naylor wrote: > > On Sun, Jun 13, 2021 at 9:50 AM Tomas Vondra > > mailto:tomas.von...@enterprisedb.com>> > > wrote: > > > >> > 2) We can still keep GEQO around (with some huge limit by default) for a > >> > f

Re: Failure in subscription test 004_sync.pl

2021-06-14 Thread Alvaro Herrera
On 2021-Jun-14, Masahiko Sawada wrote: > The condition should be the opposite; we should raise the error when > 'nowait' is true. I think this is the cause of the test failure. Even > if DROP SUBSCRIPTION tries to drop the slot with the WAIT option, we > don't wait but raise the error. Hi, thanks

Re: PG 14 release notes, first draft

2021-06-14 Thread Bruce Momjian
On Mon, Jun 14, 2021 at 01:29:42PM -0400, John Naylor wrote: > Hi Bruce, > > For this item: > > > >         >         Allow BRIN indexes to use bloom filters >         (Tomas Vondra) >         > >         >         This allows bloom indexes to be used effectively with data that >         is no

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-06-14 Thread David Christensen
> I don't see the need to extend the unit to YB. > What use case do you have in mind? Practical or no, I saw no reason not to support all defined units. I assume we’ll get to a need sooner or later. :) David

Re: PG 14 release notes, first draft

2021-06-14 Thread Bruce Momjian
On Mon, Jun 14, 2021 at 06:57:41PM +0200, Tomas Vondra wrote: > Sorry it took me a while to look at the release notes. I have one > suggestion regarding this item: > > > Allow logical replication to stream long in-progress transactions to > subscribers (Tomas Vondra, Dilip Kumar, Amit Kapila,

Re: PG 14 release notes, first draft

2021-06-14 Thread Bruce Momjian
On Mon, Jun 14, 2021 at 11:37:58AM -0500, Justin Pryzby wrote: > Some more: > > | VACUUM now has a PROCESS_TOAST which can be set to false to disable TOAST > processing, and vacuumdb has a --no-process-toast option. > has a process_toast *option Agreed. > | Previously, if the object already ex

Re: Race condition in recovery?

2021-06-14 Thread Andrew Dunstan
On 6/14/21 3:32 PM, Andrew Dunstan wrote: > On 6/14/21 1:50 PM, Andrew Dunstan wrote: >> On 6/14/21 1:11 PM, Robert Haas wrote: >>> On Mon, Jun 14, 2021 at 12:56 PM Andrew Dunstan wrote: $^X is not at all broken. The explanation here is pretty simple - the argument to perl2host is

Re: Race condition in recovery?

2021-06-14 Thread Andrew Dunstan
On 6/14/21 1:50 PM, Andrew Dunstan wrote: > On 6/14/21 1:11 PM, Robert Haas wrote: >> On Mon, Jun 14, 2021 at 12:56 PM Andrew Dunstan wrote: >>> $^X is not at all broken. >>> >>> The explanation here is pretty simple - the argument to perl2host is >>> meant to be a directory. If we're going to a

Re: Race condition in recovery?

2021-06-14 Thread Robert Haas
On Mon, Jun 14, 2021 at 1:50 PM Andrew Dunstan wrote: > Heres a snippet: > > sub perl2host > { > my ($subject) = @_; > ... > if (chdir $subject) > > Last time I looked you can't chdir to anything except a directory. OK, but like I said, you can't tell that from the

Re: Position of ClientAuthentication hook

2021-06-14 Thread Robert Haas
On Mon, Jun 14, 2021 at 8:51 AM Rafia Sabih wrote: > I have a doubt regarding the positioning of clientAuthentication hook > in function ClientAuthentication. Particularly, in case of hba errors, > i.e. cases uaReject or uaImplicitReject it errors out, leading to no > calls to any functions attach

Re: MultiXact\SLRU buffers configuration

2021-06-14 Thread Васильев Дмитрий
пн, 14 июн. 2021 г. в 15:07, Andrey Borodin : > PFA patch implementing this idea. > I'm benchmarked v17 patches. Testing was done on a 96-core machine, with PGDATA completely placed in tmpfs. PostgreSQL was built with CFLAGS -O2. for-update PgBench script: \set aid random_zipfian(1, 100, 2) begi

Re: PG 14 release notes, first draft

2021-06-14 Thread Bruce Momjian
On Fri, Jun 11, 2021 at 10:45:51PM -0500, Justin Pryzby wrote: > | Add Set Server Name Indication (SNI) for SSL connection packets (Peter > Eisentraut) > Remove "Set" Fixed. > | Reduce the default value of vacuum_cost_page_miss from 10 milliseconds to 2 > (Peter Geoghegan) > Peter mentioned t

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Andres Freund
Hi, On 2021-06-14 11:27:21 -0700, Andres Freund wrote: > On 2021-06-14 16:10:32 +0200, Jehan-Guillaume de Rorthais wrote: > > In the patch in attachment, I tried to fix this by using kind of an internal > > hook for pgstat_report_wait_start and pgstat_report_wait_end. This allows to > > "instrumen

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Andres Freund
Hi, On 2021-06-14 16:10:32 +0200, Jehan-Guillaume de Rorthais wrote: > In the patch in attachment, I tried to fix this by using kind of an internal > hook for pgstat_report_wait_start and pgstat_report_wait_end. This allows to > "instrument" wait events only when required, on the fly, dynamically.

Re: Race condition in recovery?

2021-06-14 Thread Andrew Dunstan
On 6/14/21 1:11 PM, Robert Haas wrote: > On Mon, Jun 14, 2021 at 12:56 PM Andrew Dunstan wrote: >> $^X is not at all broken. >> >> The explanation here is pretty simple - the argument to perl2host is >> meant to be a directory. If we're going to accomodate plain files then >> we have some more w

Re: using extended statistics to improve join estimates

2021-06-14 Thread Tomas Vondra
Hi, Here's a slightly improved / cleaned up version of the PoC patch, removing a bunch of XXX and FIXMEs, adding comments, etc. The approach is sound in principle, I think, although there's still a bunch of things to address: 1) statext_compare_mcvs only really deals with equijoins / inner

Re: PG 14 release notes, first draft

2021-06-14 Thread John Naylor
Hi Bruce, For this item: Allow BRIN indexes to use bloom filters (Tomas Vondra) This allows bloom indexes to be used effectively with data that is not physically localized in the heap. The text implies that this affects co

Re: recent failures on lorikeet

2021-06-14 Thread Tom Lane
I wrote: > What that'd have to imply is that get_ps_display() messed up, > returning a bad pointer or a bad length. > A platform-specific problem in get_ps_display() seems plausible > enough. The apparent connection to a concurrent VACUUM FULL seems > pretty hard to explain that way ... but maybe

Re: recent failures on lorikeet

2021-06-14 Thread Tom Lane
Andrew Dunstan writes: > The line in lmgr.c is where the process title gets changed to "waiting". > I recently stopped setting process title on this animal on REL_13_STABLE > and its similar errors have largely gone away. Oooh, that certainly seems like a smoking gun. > I can do the same on > HE

Re: Question about StartLogicalReplication() error path

2021-06-14 Thread Robert Haas
On Mon, Jun 14, 2021 at 12:50 PM Jeff Davis wrote: > It seems that there's not much agreement in a behavior change here. I > suggest one or more of the following: > > 1. change the logical rep protocol docs to match the current behavior > a. also briefly explain in the docs why it's different

Re: Race condition in recovery?

2021-06-14 Thread Robert Haas
On Mon, Jun 14, 2021 at 12:56 PM Andrew Dunstan wrote: > $^X is not at all broken. > > The explanation here is pretty simple - the argument to perl2host is > meant to be a directory. If we're going to accomodate plain files then > we have some more work to do in TestLib. This explanation seems to

Re: PG 14 release notes, first draft

2021-06-14 Thread Tomas Vondra
On 5/10/21 8:03 AM, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items pro

Re: Race condition in recovery?

2021-06-14 Thread Andrew Dunstan
On 6/14/21 11:52 AM, Robert Haas wrote: > On Sat, Jun 12, 2021 at 10:20 AM Tom Lane wrote: >> Andrew Dunstan writes: >>> I have pushed a fix, tested on a replica of fairywren/drongo, >> This bit seems a bit random: >> >> # WAL segment, this is enough to guarantee that the history file was >>

Re: Question about StartLogicalReplication() error path

2021-06-14 Thread Jeff Davis
On Sat, 2021-06-12 at 16:17 +0530, Amit Kapila wrote: > AFAIU, currently, in such a case, the subscriber (client) won't > advance the flush location (confirmed_flush_lsn). So, we won't lose > any data. I think you are talking about the official Logical Replication specifically, rather than an arbi

Re: array_cat anycompatible change is breaking xversion upgrade tests (v14 release notes)

2021-06-14 Thread Bruce Momjian
On Fri, Jun 11, 2021 at 09:40:07PM -0400, Bruce Momjian wrote: > On Fri, Jun 11, 2021 at 09:17:46PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > OK, I used some of your ideas and tried for something more general; > > > patch attached. > > > > I think it's a good idea to mention custom

Re: A new function to wait for the backend exit after termination

2021-06-14 Thread Justin Pryzby
On Mon, Jun 14, 2021 at 09:40:27AM -0700, Noah Misch wrote: > > > Agreed, these strings generally give less detail. I can revert that to > > > the > > > v13 wording, 'terminate a server process'. ... > > Maybe you'd also update the release notes. > > What part of the notes did you expect to ch

Re: recent failures on lorikeet

2021-06-14 Thread Andrew Dunstan
On 6/14/21 12:33 PM, Andrew Dunstan wrote: > > The line in lmgr.c is where the process title gets changed to "waiting". > I recently stopped setting process title on this animal on REL_13_STABLE > and its similar errors have largely gone away. I can do the same on > HEAD. But it does make me wond

Re: A new function to wait for the backend exit after termination

2021-06-14 Thread Noah Misch
On Sat, Jun 12, 2021 at 01:27:43PM -0500, Justin Pryzby wrote: > On Sat, Jun 12, 2021 at 08:21:39AM -0700, Noah Misch wrote: > > On Sat, Jun 12, 2021 at 12:12:12AM -0500, Justin Pryzby wrote: > > > Even if it's not removed, the descriptions should be cleaned up. > > > > > > | src/include/catalog/p

Re: PG 14 release notes, first draft

2021-06-14 Thread Justin Pryzby
On Fri, Jun 11, 2021 at 10:45:51PM -0500, Justin Pryzby wrote: > | Add Set Server Name Indication (SNI) for SSL connection packets (Peter > Eisentraut) > Remove "Set" > > | Reduce the default value of vacuum_cost_page_miss from 10 milliseconds to 2 > (Peter Geoghegan) > Peter mentioned that th

Re: recent failures on lorikeet

2021-06-14 Thread Andrew Dunstan
On 6/14/21 9:39 AM, Tom Lane wrote: > Andrew Dunstan writes: >> I've been looking at the recent spate of intermittent failures on my >> Cygwin animal lorikeet. Most of them look something like this, where >> there's 'VACUUM FULL pg_class' and an almost simultaneous "CREATE TABLE' >> which fails.

Re: Use extended statistics to estimate (Var op Var) clauses

2021-06-14 Thread Tomas Vondra
On 6/14/21 5:36 PM, Mark Dilger wrote: > > >> On Jun 13, 2021, at 1:28 PM, Tomas Vondra >> wrote: >> >> Here is a slightly updated version of the patch > > Thanks for taking this up again! > > Applying the new test cases from your patch, multiple estimates have > gotten better. That looks go

Re: a path towards replacing GEQO with something better

2021-06-14 Thread Tomas Vondra
On 6/14/21 1:16 PM, John Naylor wrote: > On Sun, Jun 13, 2021 at 9:50 AM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: > >> > 2) We can still keep GEQO around (with some huge limit by default) for a >> > few years as an escape hatch, while we refine the replacement. If there >>

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-14 Thread Robert Haas
On Sun, Jun 13, 2021 at 10:04 PM tsunakawa.ta...@fujitsu.com wrote: > I know sending a commit request may get an error from various underlying > functions, but we're talking about the client side, not the Postgres's server > side that could unexpectedly ereport(ERROR) somewhere. So, the new FDW

Re: Race condition in recovery?

2021-06-14 Thread Robert Haas
On Sat, Jun 12, 2021 at 10:20 AM Tom Lane wrote: > Andrew Dunstan writes: > > I have pushed a fix, tested on a replica of fairywren/drongo, > > This bit seems a bit random: > > # WAL segment, this is enough to guarantee that the history file was > # archived. > my $archive_wait_query = > - "S

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-06-14 Thread Robert Haas
On Mon, Jun 14, 2021 at 2:32 AM Amit Kapila wrote: > Why do you think we don't need to check index AM functions? Say we > have an index expression that uses function and if its parallel safety > is changed then probably that also impacts whether we can do insert in > parallel. Because otherwise, w

Re: Use extended statistics to estimate (Var op Var) clauses

2021-06-14 Thread Mark Dilger
> On Jun 13, 2021, at 1:28 PM, Tomas Vondra > wrote: > > Here is a slightly updated version of the patch Thanks for taking this up again! Applying the new test cases from your patch, multiple estimates have gotten better. That looks good. I wrote a few extra test cases and saw no change,

Re: Delegating superuser tasks to new security roles

2021-06-14 Thread Mark Dilger
> On Jun 14, 2021, at 5:51 AM, torikoshia wrote: > > Thanks for working on this topic, I appreciate it! Thank you for taking a look! > BTW, do these patches enable non-superusers to create user with > bypassrls? No, I did not break out the ability to create such users. > Since I failed to

Re: Replication protocol doc fix

2021-06-14 Thread Robert Haas
On Fri, Jun 11, 2021 at 6:12 PM Jeff Davis wrote: > On Fri, 2021-06-11 at 16:57 -0400, Robert Haas wrote: > > My impression was that CopyBoth can be initiated either way, > > The docs say: "In either physical replication or logical replication > walsender mode, only the simple query protocol can b

Re: Failure in subscription test 004_sync.pl

2021-06-14 Thread Alvaro Herrera
On 2021-Jun-14, Amit Kapila wrote: > On Mon, Jun 14, 2021 at 10:41 AM Masahiko Sawada > wrote: > > The condition should be the opposite; we should raise the error when > > 'nowait' is true. I think this is the cause of the test failure. Even > > if DROP SUBSCRIPTION tries to drop the slot with

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Jehan-Guillaume de Rorthais
Hi Andres, Hi all, First, thank you for your feedback! Please find in attachment a patch implementing accumulated wait event stats only from the backend point of view. As I wrote when I reviewed and rebased the existing patch, I was uncomfortable with the global approach. I still volunteer to wor

Re: A new function to wait for the backend exit after termination

2021-06-14 Thread Bharath Rupireddy
On Sat, Jun 12, 2021 at 10:07 AM Noah Misch wrote: > > On Fri, Jun 11, 2021 at 08:54:08PM -0500, Justin Pryzby wrote: > > On Sat, Jun 05, 2021 at 12:08:01PM -0700, Noah Misch wrote: > > > > > My preference is to remove pg_wait_for_backend_termination(). The > > > > > use case > > > > > that prom

pg_dumpall --exclude-database case folding

2021-06-14 Thread Andrew Dunstan
[discussion transferred from psql-performance] Summary: pg_dumpall and pg_dump fold non-quoted commandline patterns to lower case Tom lane writes: Andrew Dunstan writes: > On 6/10/21 2:23 PM, Andrew Dunstan wrote: >> Ouch. That looks like a plain old bug. Let's fix it. IIRC I just used >> the

Re: recent failures on lorikeet

2021-06-14 Thread Tom Lane
Andrew Dunstan writes: > I've been looking at the recent spate of intermittent failures on my > Cygwin animal lorikeet. Most of them look something like this, where > there's 'VACUUM FULL pg_class' and an almost simultaneous "CREATE TABLE' > which fails. Do you have any idea what "exit code 127"

Re: unnesting multirange data types

2021-06-14 Thread Alexander Korotkov
On Mon, Jun 14, 2021 at 3:50 PM Jonathan S. Katz wrote: > On 6/13/21 5:18 PM, Alexander Korotkov wrote: > > >> "Expands an array into a set of rows. The array's elements are read out > >> in storage order." > >> > >> If we tweaked the multirange "unnest" function, we could change it to: > >> > >>

Re: contrib/pg_visibility fails regression under CLOBBER_CACHE_ALWAYS

2021-06-14 Thread Masahiko Sawada
On Mon, Jun 7, 2021 at 10:01 PM Tomas Vondra wrote: > > > > On 6/7/21 2:11 PM, Masahiko Sawada wrote: > > On Mon, Jun 7, 2021 at 4:30 PM Masahiko Sawada > > wrote: > >> > >> On Mon, Jun 7, 2021 at 11:15 AM Tom Lane wrote: > >>> > >>> husky just reported $SUBJECT: > >>> > >>> https://buildfarm.p

Position of ClientAuthentication hook

2021-06-14 Thread Rafia Sabih
Hello hackers, I have a doubt regarding the positioning of clientAuthentication hook in function ClientAuthentication. Particularly, in case of hba errors, i.e. cases uaReject or uaImplicitReject it errors out, leading to no calls to any functions attached to the authentication hook. Can't we proc

Re: Delegating superuser tasks to new security roles

2021-06-14 Thread torikoshia
On 2021-05-26 05:33, Mark Dilger wrote: On May 13, 2021, at 12:30 PM, Mark Dilger wrote: On May 13, 2021, at 12:18 PM, Jacob Champion wrote: On Thu, 2021-05-13 at 11:42 -0700, Mark Dilger wrote: The distinction that Theme+Security would make is that capabilities can be categorized by th

Re: unnesting multirange data types

2021-06-14 Thread Jonathan S. Katz
On 6/13/21 5:18 PM, Alexander Korotkov wrote: >> "Expands an array into a set of rows. The array's elements are read out >> in storage order." >> >> If we tweaked the multirange "unnest" function, we could change it to: >> >> + >> +Expands a multirange into a set of rows. >> +

Re: Fdw batch insert error out when set batch_size > 65535

2021-06-14 Thread Bharath Rupireddy
On Mon, Jun 14, 2021, 5:33 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > Okay. Here are the readings on my dev system: > 1) on master with the existing test case with inserting 70K rows: > 4263200 ms (71.05 min) > 2) with Tomas's patch with the test case modified with 150

recent failures on lorikeet

2021-06-14 Thread Andrew Dunstan
I've been looking at the recent spate of intermittent failures on my Cygwin animal lorikeet. Most of them look something like this, where there's 'VACUUM FULL pg_class' and an almost simultaneous "CREATE TABLE' which fails. 2021-06-14 05:04:00.220 EDT [60c71b7f.e8bf:60] pg_regress/vacuum LOG:

Re: RFC: Logging plan of the running query

2021-06-14 Thread torikoshia
On 2021-06-11 01:20, Bharath Rupireddy wrote: Thanks for your review! On Wed, Jun 9, 2021 at 1:14 PM torikoshia wrote: Updated the patch. Thanks for the patch. Here are some comments on v3 patch: 1) We could just say "Requests to log query plan of the presently running query of a given bac

Re: BUG #15293: Stored Procedure Triggered by Logical Replication is Unable to use Notification Events

2021-06-14 Thread Sergey Fedchenko
Hi all! I still can reproduce it on 14beta1 version. I adapted a patch I found in this thread https://github.com/seregayoga/postgres/commit/338bc33f2cf77edde7c45bfdfb9f39a92ec57eb8 . It solved this bug for me (tested with simple Go program using https://github.com/lib/pq ). It would be nice

Re: Fdw batch insert error out when set batch_size > 65535

2021-06-14 Thread Bharath Rupireddy
On Sun, Jun 13, 2021 at 9:28 PM Tomas Vondra wrote: > On 6/13/21 5:25 PM, Bharath Rupireddy wrote: > > On Sun, Jun 13, 2021 at 6:10 AM Alvaro Herrera > > wrote: > >> > >> On 2021-Jun-12, Tomas Vondra wrote: > >> > >>> There's one caveat, though - for regular builds the slowdown is pretty > >>> m

RE: locking [user] catalog tables vs 2pc vs logical rep

2021-06-14 Thread osumi.takami...@fujitsu.com
On Friday, June 11, 2021 2:13 PM vignesh C wrote: > Thanks for the updated patch: > Few comments: > 1) We have used Reordering and Clustering for the same command, we could > rephrase similarly in both places. > + > +Reordering pg_class by > CLUSTER > +command in a transac

Fwd: BUG #17056: Segmentation fault on altering the type of the foreign table column with a default

2021-06-14 Thread Andrew Dunstan
reposting to -hackers to get more eyeballs. Summary: only RELKIND_RELATION type relations should have attributes with atthasmissing/attmissingval Forwarded Message Subject:Re: BUG #17056: Segmentation fault on altering the type of the foreign table column with a defa

Re: a path towards replacing GEQO with something better

2021-06-14 Thread John Naylor
On Sun, Jun 13, 2021 at 9:50 AM Tomas Vondra wrote: > > 2) We can still keep GEQO around (with some huge limit by default) for a > > few years as an escape hatch, while we refine the replacement. If there > > is some bug that prevents finding a plan, we can emit a WARNING and fall > > back to GEQ

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-06-14 Thread Matthias van de Meent
On Thu, 10 Jun 2021 at 19:43, Peter Geoghegan wrote: > > On Thu, Jun 10, 2021 at 10:29 AM Matthias van de Meent > wrote: > > I see one exit for HEAPTUPLE_DEAD on a potentially recently committed > > xvac (?), and we might also check against recently committed > > transactions if xmin == xmax, alt

Re: Different compression methods for FPI

2021-06-14 Thread Andrey Borodin
Thanks for benchmarks, Justin! > 14 июня 2021 г., в 06:24, Justin Pryzby написал(а): > > The GUC is PGC_USERSET Oh, wow, that's neat. I did not realize that we can tune this for each individual client connection. Cool! > pglz writes ~half as much, but takes twice as long as uncompressed: > |T

Re: GiST operator class for bool

2021-06-14 Thread Andrey Borodin
> 8 июня 2021 г., в 19:53, Emre Hasegeli написал(а): > >> But patch that you propose does not support sorting build added in PG14. > > It looks like the change to btree_gist is not committed yet. I'll > rebase my patch once it's committed. Changes to GiST are committed. There will be no need

Re: pgbench bug candidate: negative "initial connection time"

2021-06-14 Thread Fabien COELHO
Hmmm. Possibly. Another option could be not to report anything after some errors. I'm not sure, because it would depend on the use case. I guess the command returned an error status as well. I did not know any use cases and decisions , but I vote to report nothing when error occurs. I woul

  1   2   >