Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Amit Kapila
On Mon, Jun 21, 2021 at 10:55 AM Mark Dilger wrote: > > > On Jun 20, 2021, at 10:11 PM, Amit Kapila wrote: > > > > However, I see a different case where there could be > > multiple failing xids and that can happen during initial table sync > > where multiple workers failed due to some error. I

Re: PG 14 release notes, first draft

2021-06-20 Thread Tatsuo Ishii
> On Mon, Jun 21, 2021 at 12:50 PM Bruce Momjian wrote: >> >> On Tue, Jun 15, 2021 at 12:01:00PM +0900, Michael Paquier wrote: >> > On Tue, Jun 15, 2021 at 11:49:21AM +0900, Masahiko Sawada wrote: >> > > On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian wrote: >> > >> OK, but I need more

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Amit Kapila
On Mon, Jun 21, 2021 at 9:21 AM Masahiko Sawada wrote: > > On Mon, Jun 21, 2021 at 12:09 PM Amit Kapila wrote: > > > > On Mon, Jun 21, 2021 at 7:56 AM Mark Dilger > > wrote: > > > > > > > On Jun 20, 2021, at 7:17 PM, Masahiko Sawada > > > > wrote: > > > > > > > > I will submit the patch. > >

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Mark Dilger
> On Jun 20, 2021, at 10:11 PM, Amit Kapila wrote: > > However, I see a different case where there could be > multiple failing xids and that can happen during initial table sync > where multiple workers failed due to some error. I am not sure your > patch would be able to capture all such

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-06-20 Thread Thomas Munro
On Mon, Jun 21, 2021 at 4:32 PM Sandeep Thakkar wrote: > Do we see any solution to this issue? or using the older SDK is the way to go? > On Thu, May 20, 2021 at 2:04 PM Dave Page wrote: >> The ability to target older releases with a newer SDK is essential for >> packages such as the EDB

Re: Added schema level support for publication.

2021-06-20 Thread vignesh C
Thanks for the comments. On Fri, Jun 18, 2021 at 5:25 PM Ajin Cherian wrote: > > On Thu, Jun 17, 2021 at 12:41 AM vignesh C wrote: > > > Thanks for reporting it, the attached patch is a rebased version of > > the patch with few review comment fixes, I will reply with the comment > > fixes to

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Mark Dilger
> On Jun 20, 2021, at 10:11 PM, Amit Kapila wrote: > > Then also it will fail on the first such conflict, so even without > your patch, the apply worker corresponding to the subscription won't > be able to proceed after the first error, it won't lead to multiple > failing xids. I'm not sure

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Mark Dilger
> On Jun 20, 2021, at 8:09 PM, Amit Kapila wrote: > > (a) to define exactly what all > information is required to be logged on error, (b) where do we want to > store the information based on requirements. I'm not sure it has to be stored anywhere durable. I have a patch in the works to do

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Amit Kapila
On Mon, Jun 21, 2021 at 10:24 AM Mark Dilger wrote: > > > On Jun 20, 2021, at 8:09 PM, Amit Kapila wrote: > > > > Because currently, we don't proceed after an error unless it is > > resolved. Why do you think there could be multiple such transactions? > > Just as one example, if the subscriber

Re: PG 14 release notes, first draft

2021-06-20 Thread Masahiko Sawada
On Mon, Jun 21, 2021 at 2:07 PM Bruce Momjian wrote: > > On Mon, Jun 21, 2021 at 01:57:32PM +0900, Masahiko Sawada wrote: > > > OK, I went with this text and put it in the Source Code section since it > > > applies to several layers of Postgres. > > > > Thanks! > > > > I got the parse error after

Re: PG 14 release notes, first draft

2021-06-20 Thread Bruce Momjian
On Fri, Jun 18, 2021 at 08:47:21PM -0700, Peter Geoghegan wrote: > On Mon, Jun 14, 2021 at 1:11 PM Bruce Momjian wrote: > > FYI, the most recent PG 14 relnote doc build is at: > > > > https://momjian.us/pgsql_docs/release-14.html > > I just pushed a commit that makes the existing

Re: PG 14 release notes, first draft

2021-06-20 Thread Bruce Momjian
On Mon, Jun 21, 2021 at 01:57:32PM +0900, Masahiko Sawada wrote: > > OK, I went with this text and put it in the Source Code section since it > > applies to several layers of Postgres. > > Thanks! > > I got the parse error after applying the patch: > > release-14.sgml:3562: parser error : Input

Re: PG 14 release notes, first draft

2021-06-20 Thread Masahiko Sawada
On Mon, Jun 21, 2021 at 12:50 PM Bruce Momjian wrote: > > On Tue, Jun 15, 2021 at 12:01:00PM +0900, Michael Paquier wrote: > > On Tue, Jun 15, 2021 at 11:49:21AM +0900, Masahiko Sawada wrote: > > > On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian wrote: > > >> OK, but I need more information on

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-06-20 Thread Amit Kapila
On Thu, Jun 17, 2021 at 8:29 PM Robert Haas wrote: > > On Thu, Jun 17, 2021 at 4:54 AM Amit Kapila wrote: > > I have responded about heavy-weight locking stuff in my next email [1] > > and why I think the approach I mentioned will work. I don't deny that > > I might be missing something here. >

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Mark Dilger
> On Jun 20, 2021, at 8:09 PM, Amit Kapila wrote: > > Because currently, we don't proceed after an error unless it is > resolved. Why do you think there could be multiple such transactions? Just as one example, if the subscriber has a unique index that the publisher lacks, any number of

Re: [PATCH v3 1/1] Fix detection of preadv/pwritev support for OSX.

2021-06-20 Thread Sandeep Thakkar
Hi, Do we see any solution to this issue? or using the older SDK is the way to go? On Thu, May 20, 2021 at 2:04 PM Dave Page wrote: > > > On Tue, Mar 30, 2021 at 6:58 AM Tom Lane wrote: > >> Thomas Munro writes: >> > I'll move it when committing. I'll let this patch sit for another day >> >

Re: Maintaining a list of pgindent commits for "git blame" to ignore

2021-06-20 Thread Peter Geoghegan
On Thu, Mar 18, 2021 at 4:32 PM Peter Geoghegan wrote: > On Thu, Mar 18, 2021 at 4:05 PM Tom Lane wrote: > > b5d69b7c22ee4c44b8bb99cfa0466ffaf3b5fab9 # Sun Jun 7 16:57:08 2020 -0400 > > # pgindent run prior to branching v13. > > > > which is easy to make from "git log" or "git show" output.

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Masahiko Sawada
On Mon, Jun 21, 2021 at 12:09 PM Amit Kapila wrote: > > On Mon, Jun 21, 2021 at 7:56 AM Mark Dilger > wrote: > > > > > On Jun 20, 2021, at 7:17 PM, Masahiko Sawada > > > wrote: > > > > > > I will submit the patch. > > > > Great, thanks! > > > > > There was a discussion that the skipping

Re: PG 14 release notes, first draft

2021-06-20 Thread Bruce Momjian
On Tue, Jun 15, 2021 at 12:01:00PM +0900, Michael Paquier wrote: > On Tue, Jun 15, 2021 at 11:49:21AM +0900, Masahiko Sawada wrote: > > On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian wrote: > >> OK, but I need more information on how users will see a difference based > >> on this commit: > > +1.

RE: locking [user] catalog tables vs 2pc vs logical rep

2021-06-20 Thread osumi.takami...@fujitsu.com
On Sunday, June 20, 2021 9:50 PM I wrote: > On Sunday, June 20, 2021 3:23 PM Amit Kapila > wrote: > > On Sun, Jun 20, 2021 at 9:28 AM osumi.takami...@fujitsu.com > > wrote: > > > * doc/src/sgml/logicaldecoding.sgml > ... > > > > > > Now we have the four paren supplementary descriptions, not to

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Amit Kapila
On Mon, Jun 21, 2021 at 7:56 AM Mark Dilger wrote: > > > On Jun 20, 2021, at 7:17 PM, Masahiko Sawada wrote: > > > > I will submit the patch. > > Great, thanks! > > > There was a discussion that the skipping transaction patch would also > > need to have a feature that tells users the details of

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Amit Kapila
On Sat, Jun 19, 2021 at 8:14 PM Mark Dilger wrote: > > > On Jun 19, 2021, at 3:17 AM, Amit Kapila wrote: > > Also, why not also log the xid of the failed > > transaction? > > We could also do that. Reading [1], it seems you are overly focused on > user-facing xids. The errdetail in the

Windows' copy of ICU

2021-06-20 Thread Thomas Munro
Hi, I noticed a striking similarity between the collation versions reported by Windows and ICU, and found my way to this new system copy of ICU (C APIs only) that you can use on recent enough Windows[1]. Not planning to do anything with that observation myself but it seemed interesting enough to

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Mark Dilger
> On Jun 20, 2021, at 7:17 PM, Masahiko Sawada wrote: > > I will submit the patch. Great, thanks! > There was a discussion that the skipping transaction patch would also > need to have a feature that tells users the details of the last > failure transaction such as its XID, timestamp,

Re: Optionally automatically disable logical replication subscriptions on error

2021-06-20 Thread Masahiko Sawada
On Sat, Jun 19, 2021 at 11:44 PM Mark Dilger wrote: > > > > > On Jun 19, 2021, at 3:17 AM, Amit Kapila wrote: > > > > Right, but there are things that could be common from the design > > perspective. > > I went to reconcile my patch with that from [1] only to discover there is no > patch on

Re: seawasp failing, maybe in glibc allocator

2021-06-20 Thread Thomas Munro
On Sun, Jun 20, 2021 at 11:01 PM Andres Freund wrote: > On 2021-06-19 10:12:03 -0400, Tom Lane wrote: > > Is a compile-time conditional really going to be reliable? See nearby > > arguments about compile-time vs run-time checks for libpq features. > > It's not clear to me how tightly LLVM binds

Re: seawasp failing, maybe in glibc allocator

2021-06-20 Thread Tom Lane
Thomas Munro writes: > Looking at their release schedule on https://llvm.org/, I see we have > a gamble to make. They currently plan to cut RC1 at the end of July, > and to release in late September (every second LLVM major release > coincides approximately with a PG major release). Option 1:

Re: seawasp failing, maybe in glibc allocator

2021-06-20 Thread Thomas Munro
On Sun, Jun 20, 2021 at 10:59 PM Andres Freund wrote: > I think this should be part of the earlier loop? Once > LLVMOrcAbsoluteSymbols() is called that owns the reference, so there > doesn't seem to be a reason to increase the refcount only later? Right, that makes sense. Here's a patch like

Re: Reducing the cycle time for CLOBBER_CACHE_ALWAYS buildfarm members

2021-06-20 Thread Andrew Dunstan
On 6/20/21 6:10 PM, Tom Lane wrote: > (3) Since this only works in v14 and up, older branches would > have to fall back to -DCLOBBER_CACHE_ALWAYS. Perhaps we could > improve the buildfarm client script so that buildfarm owners > just configure "clobber_cache_testing => 1" and then the script >

Re: unnesting multirange data types

2021-06-20 Thread Alexander Korotkov
On Sun, Jun 20, 2021 at 11:09 AM Noah Misch wrote: > On Sat, Jun 19, 2021 at 10:05:09PM -0400, Tom Lane wrote: > > Alexander Korotkov writes: > > > I also don't feel comfortable hurrying with unnest part to beta2. > > > According to the open items wiki page, there should be beta3. Does > > >

Reducing the cycle time for CLOBBER_CACHE_ALWAYS buildfarm members

2021-06-20 Thread Tom Lane
I had an idea for another way to attack $SUBJECT, now that we have the ability to adjust debug_invalidate_system_caches_always at runtime. Namely, that a lot of time goes into repeated initdb runs (especially if the TAP tests are enabled); but surely we learn little from CCA initdb runs after the

Re: Race condition in InvalidateObsoleteReplicationSlots()

2021-06-20 Thread Tom Lane
I wrote: > I studied this failure a bit more, and I think the test itself has > a race condition. It's doing > > # freeze walsender and walreceiver. Slot will still be active, but walreceiver > # won't get anything anymore. > kill 'STOP', $senderpid, $receiverpid; > $logstart =

Re: Different compression methods for FPI

2021-06-20 Thread Andrey Borodin
> 17 июня 2021 г., в 11:44, Michael Paquier написал(а): > > I have worked more on that today and finished with two patches I have some small questions. 1. + report_invalid_record(record, "image at %X/%X compressed with %s not supported, block %d", +

Re: a path towards replacing GEQO with something better

2021-06-20 Thread AJG
Hi, I stumbled across this which may be of interest to this topic and GEQO alternative. The main creator/author of Neo and Bao (ML for Query Optimizer) Ryan Marcus (finishing Postdoc and looking for job) recently posted [1] about Bao for distributed systems. But what was interesting

Re: PXGS vs TAP tests

2021-06-20 Thread Andrew Dunstan
On 6/20/21 10:56 AM, Andrew Dunstan wrote: > On 6/20/21 10:45 AM, Tom Lane wrote: > >>> I removed the REGRESS_SHLIB setting altogether for the PGXS case - it's >>> not clear to me why we need it in a TAP test recipe at all. >> After some digging in the git history, it looks like it's there

Re: Race condition in InvalidateObsoleteReplicationSlots()

2021-06-20 Thread Tom Lane
I wrote: > Hmm ... desmoxytes has failed this test once, out of four runs since > it went in: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=desmoxytes=2021-06-19%2003%3A06%3A04 I studied this failure a bit more, and I think the test itself has a race condition. It's doing # freeze

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2021-06-20 Thread Zhihong Yu
On Sun, Jun 20, 2021 at 6:56 AM David Rowley wrote: > On Wed, 14 Aug 2019 at 19:25, David Rowley > wrote: > > For now, I'm out of ideas. If anyone else feels like suggesting > > something of picking this up, feel free. > > This is a pretty old thread, so we might need a recap: > > # Recap > >

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-06-20 Thread Peter Geoghegan
On Sun, Jun 20, 2021 at 9:22 AM Mark Dilger wrote: > I'd want to see some evidence that the GUC is necessary. (For that matter, > why is a per relation setting necessary?) Is there a reproducible > pathological case, perhaps with a pgbench script, to demonstrate the need? > I'm not asking

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-06-20 Thread Mark Dilger
> On Jun 14, 2021, at 7:46 PM, Peter Geoghegan wrote: > > Does anyone else have an opinion on this? Of course I can easily add a > GUC. But I won't do so in the absence of any real argument in favor of > it. I'd want to see some evidence that the GUC is necessary. (For that matter, why is

Re: Race condition in InvalidateObsoleteReplicationSlots()

2021-06-20 Thread Álvaro Herrera
Hah, desmoxytes failed once: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=desmoxytes=2021-06-19%2003%3A06%3A04 I'll revert it and investigate later. There have been no other failures. -- Álvaro Herrera39°49'30"S 73°17'W "Hay que recordar que la existencia

Re: PXGS vs TAP tests

2021-06-20 Thread Andrew Dunstan
On 6/20/21 10:45 AM, Tom Lane wrote: > Andrew Dunstan writes: >> The recipe for running TAP tests in src/Makefile.global doesn't work for >> the PGXS case. If you try it you get something like this: >> ... >> Notice those bogus settings for top_builddir, PG_REGRESS and >> REGRESS_SHLIB. The

Re: PXGS vs TAP tests

2021-06-20 Thread Tom Lane
Andrew Dunstan writes: > The recipe for running TAP tests in src/Makefile.global doesn't work for > the PGXS case. If you try it you get something like this: > ... > Notice those bogus settings for top_builddir, PG_REGRESS and > REGRESS_SHLIB. The attached patch fixes this bug. OK, but does the

Re: pgbench logging broken by time logic changes

2021-06-20 Thread Andrew Dunstan
On 6/20/21 5:02 AM, Fabien COELHO wrote: > >> Upon reflection, that amounts to the same thing really, so yeah, >> scratch that plan.  I'll defer until after that (and then I'll be >> leaning more towards the revert option). > > Sigh. I do not understand anything about the decision processus.

Re: [PATCH] Finally split StdRdOptions into HeapOptions and ToastOptions

2021-06-20 Thread Nikolay Shaplov
В письме от пятница, 4 июня 2021 г. 3:19:09 MSK пользователь Jeff Davis написал: > > Moving reloptions to AM code is the goal I am slowly moving to. I've > > started > > some time ago with big patch > > https://commitfest.postgresql.org/14/992/ and > > have been told to split it into smaller

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2021-06-20 Thread David Rowley
On Wed, 14 Aug 2019 at 19:25, David Rowley wrote: > For now, I'm out of ideas. If anyone else feels like suggesting > something of picking this up, feel free. This is a pretty old thread, so we might need a recap: # Recap Basically LockReleaseAll() becomes slow after a large number of locks

PXGS vs TAP tests

2021-06-20 Thread Andrew Dunstan
The recipe for running TAP tests in src/Makefile.global doesn't work for the PGXS case. If you try it you get something like this: andrew@emma:tests $ make PG_CONFIG=../inst.head.5701/bin/pg_config installcheck rm -rf '/home/andrew/pgl/tests'/tmp_check /usr/bin/mkdir -p

RE: locking [user] catalog tables vs 2pc vs logical rep

2021-06-20 Thread osumi.takami...@fujitsu.com
On Sunday, June 20, 2021 3:23 PM Amit Kapila wrote: > On Sun, Jun 20, 2021 at 9:28 AM osumi.takami...@fujitsu.com > wrote: > > * doc/src/sgml/logicaldecoding.sgml ... > > > > Now we have the four paren supplementary descriptions, not to make > > users miss any other [user] catalog tables. > >

Re: seawasp failing, maybe in glibc allocator

2021-06-20 Thread Andres Freund
Hi, On 2021-06-19 10:12:03 -0400, Tom Lane wrote: > Is a compile-time conditional really going to be reliable? See nearby > arguments about compile-time vs run-time checks for libpq features. > It's not clear to me how tightly LLVM binds its headers and running > code. It should be fine (and if

Re: seawasp failing, maybe in glibc allocator

2021-06-20 Thread Andres Freund
Hi, On 2021-06-19 17:07:51 +1200, Thomas Munro wrote: > On Sat, May 22, 2021 at 12:25 PM Andres Freund wrote: > > On 2021-05-21 15:57:01 -0700, Andres Freund wrote: > > > I found the LLVM commit to blame > > > (c8fc5e3ba942057d6c4cdcd1faeae69a28e7b671). > > > Contacting the author and reading

Re: pgbench logging broken by time logic changes

2021-06-20 Thread Fabien COELHO
Upon reflection, that amounts to the same thing really, so yeah, scratch that plan. I'll defer until after that (and then I'll be leaning more towards the revert option). Sigh. I do not understand anything about the decision processus. If you do revert, please consider NOT reverting the

Re: unnesting multirange data types

2021-06-20 Thread Noah Misch
On Sat, Jun 19, 2021 at 10:05:09PM -0400, Tom Lane wrote: > Alexander Korotkov writes: > > I also don't feel comfortable hurrying with unnest part to beta2. > > According to the open items wiki page, there should be beta3. Does > > unnest part have a chance for beta3? > > Hm. I'd prefer to

Re: pgbench logging broken by time logic changes

2021-06-20 Thread Thomas Munro
On Sun, Jun 20, 2021 at 4:52 PM Thomas Munro wrote: > On Sun, Jun 20, 2021 at 3:18 PM Alvaro Herrera > wrote: > > On 2021-Jun-19, Thomas Munro wrote: > > > Thanks for looking so far. It's the weekend here and I need to > > > unplug, but I'll test these changes and if all looks good push on > >

Re: locking [user] catalog tables vs 2pc vs logical rep

2021-06-20 Thread Amit Kapila
On Sun, Jun 20, 2021 at 9:28 AM osumi.takami...@fujitsu.com wrote: > > On Saturday, June 19, 2021 6:51 PM Amit Kapila > wrote: > > On Fri, Jun 18, 2021 at 2:25 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > On Friday, June 18, 2021 11:41 AM osumi.takami...@fujitsu.com > > wrote: > > >