Re: Changes to recovery_min_apply_delay are ignored while waiting for delay

2021-08-03 Thread Kyotaro Horiguchi
At Mon, 2 Aug 2021 22:21:56 -0700, Soumyadeep Chakraborty wrote in > Hello, > > I came across this issue while I was tweaking a TAP test with Ashwin for > this thread [1]. > > We noticed that while the startup process waits for a recovery delay, it does > not respect changes to the

Re: straightening out backend process startup

2021-08-03 Thread Kyotaro Horiguchi
At Mon, 2 Aug 2021 09:41:24 -0700, Andres Freund wrote in > Hi, > > I've previously complained ([1]) that process initialization has gotten > very complicated. I hit this once more last week when trying to commit > one of the shared memory stats pieces... > > I think there's quite a few

Re: Remove unused 'len' from pg_stat_recv_* functions

2021-08-03 Thread Masahiko Sawada
On Tue, Aug 3, 2021 at 3:17 PM Kyotaro Horiguchi wrote: > > At Tue, 3 Aug 2021 12:40:23 +0900, Masahiko Sawada > wrote in > > Hi all, > > > > While working on a patch adding new stats, houzj pointed out that > > 'len' function arguments of all pgstat_recv_* functions are not used > > at all.

RE: Added schema level support for publication.

2021-08-03 Thread houzj.f...@fujitsu.com
On Tuesday, August 3, 2021 4:10 PM houzj.f...@fujitsu.com wrote: > On August 2, 2021 11:40 PM vignesh C wrote: > > > > Thanks for the comments, attached v17 patches has the fixes for the same. > > Thanks for the new patch, it looks good to me except one minor thing: > It might be better to add

Re: Remove unused 'len' from pg_stat_recv_* functions

2021-08-03 Thread Kyotaro Horiguchi
At Tue, 3 Aug 2021 12:40:23 +0900, Masahiko Sawada wrote in > Hi all, > > While working on a patch adding new stats, houzj pointed out that > 'len' function arguments of all pgstat_recv_* functions are not used > at all. Looking at git history, pgstat_recv_* functions have been > having ‘len’

Re: when the startup process doesn't (logging startup delays)

2021-08-03 Thread Nitin Jadhav
> Thanks. Can you please have a look at what I suggested down toward the > bottom of > http://postgr.es/m/ca+tgmoap2wefsktmcgwt9lxuz7y99hnduyshpk7qnfuqb98...@mail.gmail.com ? Implemented the above approach and verified the patch. Kindly have a look and share your thoughts. Thanks & Regards,

Re: Failed transaction statistics to measure the logical replication progress

2021-08-03 Thread Amit Kapila
On Tue, Aug 3, 2021 at 10:59 AM Masahiko Sawada wrote: > > On Tue, Aug 3, 2021 at 11:47 AM Amit Kapila wrote: > > > > On Mon, Aug 2, 2021 at 1:13 PM Masahiko Sawada > > wrote: > > > > > > On Mon, Aug 2, 2021 at 2:52 PM osumi.takami...@fujitsu.com > > > wrote: > > > > > > > > > > > >

RE: Added schema level support for publication.

2021-08-03 Thread tanghy.f...@fujitsu.com
On Monday, August 2, 2021 11:40 PM vignesh C wrote: > > Thanks for the comments, attached v17 patches has the fixes for the same. Thanks for your new patch. I saw the following warning when compiling. It seems we don't need this variable any more. publicationcmds.c: In function

RE: Added schema level support for publication.

2021-08-03 Thread houzj.f...@fujitsu.com
On August 2, 2021 11:40 PM vignesh C wrote: > > Thanks for the comments, attached v17 patches has the fixes for the same. Thanks for the new patch, it looks good to me except one minor thing: It might be better to add the [CREATE PUBLICATION xxx FOR SCHEMA ] in tab-complete.c Best regards,

RE: SI messages sent when excuting ROLLBACK PREPARED command

2021-08-03 Thread liuhuail...@fujitsu.com
Hi, tom > >Hmmm, yeah, I think you're right. It probably doesn't make a big difference > >in > the real world --- anyone who's dependent on the performance of 2PC rollbaxks > is Doing It Wrong. > > But we'd have already done LocalExecuteInvalidationMessage when getting > out of the prepared

Re: Lowering the ever-growing heap->pd_lower

2021-08-03 Thread Simon Riggs
On Tue, 18 May 2021 at 20:33, Peter Geoghegan wrote: > > On Tue, May 18, 2021 at 12:29 PM Matthias van de Meent > wrote: > > PFA the updated version of this patch. Apart from adding line pointer > > truncation in PageRepairFragmentation (as in the earlier patches), I > > also altered

Re: Changes to recovery_min_apply_delay are ignored while waiting for delay

2021-08-03 Thread Soumyadeep Chakraborty
Hi Kyotaro, Thanks for the review! On Mon, Aug 2, 2021 at 11:42 PM Kyotaro Horiguchi wrote: > One comment from me. > > +$node_standby->safe_psql('postgres', "ALTER SYSTEM SET > recovery_min_apply_delay TO 0;"); > > It might be better do "SET reco.. TO DEFAULT" instead. > Sure. > And how

Re: [HACKERS] logical decoding of two-phase transactions

2021-08-03 Thread Peter Smith
On Tue, Aug 3, 2021 at 5:02 PM Amit Kapila wrote: > > On Tue, Aug 3, 2021 at 6:17 AM Peter Smith wrote: > > > > Please find attached the latest patch set v102* > > > > I have made minor modifications in the comments and docs, please see > attached. Can you please check whether the names of

RE: Failed transaction statistics to measure the logical replication progress

2021-08-03 Thread osumi.takami...@fujitsu.com
On Tuesday, August 3, 2021 2:29 PM Masahiko Sawada wrote: > On Tue, Aug 3, 2021 at 11:47 AM Amit Kapila > wrote: > > > > On Mon, Aug 2, 2021 at 1:13 PM Masahiko Sawada > wrote: > > > > > > On Mon, Aug 2, 2021 at 2:52 PM osumi.takami...@fujitsu.com > > > wrote: > > > > > > > > > > > >

Use POPCNT on MSVC

2021-08-03 Thread David Rowley
Going by [1], it looks like we can use the __popcnt and __popcnt64 intrinsic functions on MSVC if the CPU supports POPCNT. We already have code to check for that, we just need to enable it on MSVC. The attached patch seems to be all that's needed. David [1]

Re: [HACKERS] logical decoding of two-phase transactions

2021-08-03 Thread Amit Kapila
On Tue, Aug 3, 2021 at 6:17 AM Peter Smith wrote: > > Please find attached the latest patch set v102* > I have made minor modifications in the comments and docs, please see attached. Can you please check whether the names of contributors in the commit message are correct or do we need to change

Re: Extra code in commit_ts.h

2021-08-03 Thread David Rowley
On Tue, 3 Aug 2021 at 23:12, Andrey Lepikhov wrote: > > On 3/8/21 15:16, David Rowley wrote: > > it diff 73c986adde5~1.. | grep check_track_commit_timestamp > I think this is waste code. May be delete it? Yeah, I think it can be safely removed. I can do so unless someone thinks otherwise.

Re: Use POPCNT on MSVC

2021-08-03 Thread John Naylor
On Tue, Aug 3, 2021 at 5:03 AM David Rowley wrote: > > Going by [1], it looks like we can use the __popcnt and __popcnt64 > intrinsic functions on MSVC if the CPU supports POPCNT. We already > have code to check for that, we just need to enable it on MSVC. > > The attached patch seems to be all

[PATCH] Tab completion for ALTER TABLE … ADD …

2021-08-03 Thread Dagfinn Ilmari Mannsåker
Hi Hackers, The other day I noticed that there's no tab completion after ALTER TABLE … ADD, so here's a patch. In addition to COLUMN and all the table constraint types, it also completes with the list of unique indexes on the table after ALTER TABLE … ADD … USING INDEX. - ilmari >From

Re: [HACKERS] logical decoding of two-phase transactions

2021-08-03 Thread vignesh C
On Tue, Aug 3, 2021 at 12:32 PM Amit Kapila wrote: > > On Tue, Aug 3, 2021 at 6:17 AM Peter Smith wrote: > > > > Please find attached the latest patch set v102* > > > > I have made minor modifications in the comments and docs, please see > attached. Can you please check whether the names of

Re: row filtering for logical replication

2021-08-03 Thread Amit Kapila
On Tue, Jul 27, 2021 at 9:56 AM Dilip Kumar wrote: > > On Tue, Jul 27, 2021 at 6:21 AM houzj.f...@fujitsu.com > wrote: > > > 1) UPDATE a nonkey column in publisher. > > 2) Use debugger to block the walsender process in function > >pgoutput_row_filter_exec_expr(). > > 3) Open another psql to

Re: Extra code in commit_ts.h

2021-08-03 Thread Andrey Lepikhov
On 3/8/21 15:16, David Rowley wrote: it diff 73c986adde5~1.. | grep check_track_commit_timestamp I think this is waste code. May be delete it? -- regards, Andrey Lepikhov Postgres Professional

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-03 Thread Gilles Darold
Le 03/08/2021 à 11:45, Gilles Darold a écrit : Actually I just found that the regexp_like() function doesn't support the start parameter which is something we should support. I saw that Oracle do not support it but DB2 does and I think we should also support it. I will post a new version of

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-03 Thread Erik Rijkers
On 8/3/21 1:26 PM, Gilles Darold wrote: Le 03/08/2021 à 11:45, Gilles Darold a écrit : Actually I just found that the regexp_like() function doesn't support the start parameter which is something we should support. I saw that Oracle do not support it but DB2 does and I think we should also

Re: when the startup process doesn't (logging startup delays)

2021-08-03 Thread Justin Pryzby
Two issues that I saw: The first syncfs message is not output, because it's before InitStartupProgress() is called: 2021-08-03 07:53:02.176 CDT startup[9717] LOG: database system was interrupted; last known up at 2021-08-03 07:52:15 CDT 2021-08-03 07:53:02.733 CDT startup[9717] LOG: data

Extra code in commit_ts.h

2021-08-03 Thread Andrey V. Lepikhov
Hi, I found two extra code lines in commit_ts.h (see attachment). They confused me during exploring of the code. If they still needed, may be add some comments? -- regards, Andrey Lepikhov Postgres Professional diff --git a/src/include/access/commit_ts.h b/src/include/access/commit_ts.h index

Re: Skipping logical replication transactions on subscriber side

2021-08-03 Thread vignesh C
On Tue, Aug 3, 2021 at 12:20 PM Masahiko Sawada wrote: > > On Mon, Aug 2, 2021 at 12:21 PM Amit Kapila wrote: > > > > On Mon, Aug 2, 2021 at 7:45 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Jul 30, 2021 at 12:52 PM Amit Kapila > > > wrote: > > > > > > > > On Thu, Jul 29, 2021 at 11:18

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-03 Thread Gilles Darold
Le 02/08/2021 à 23:22, Gilles Darold a écrit : Le 02/08/2021 à 01:21, Tom Lane a écrit : Gilles Darold writes: [ v5-0001-regexp-foo-functions.patch ] I've gone through this whole patch now, and found quite a lot that I did not like. In no particular order: * Wrapping parentheses around the

RE: Implementing Incremental View Maintenance

2021-08-03 Thread r.takahash...@fujitsu.com
Hi Nagata-san, I am interested in this patch since it is good feature. I run some simple tests. I found the following problems. (1) Failed to "make world". I think there are extra "" in doc/src/sgml/ref/create_materialized_view.sgml (line 110 and 117) (2) In the case of partition, it

Re: Extra code in commit_ts.h

2021-08-03 Thread David Rowley
On Tue, 3 Aug 2021 at 21:36, Andrey V. Lepikhov wrote: > I found two extra code lines in commit_ts.h (see attachment). > They confused me during exploring of the code. If they still needed, may > be add some comments? Going by: $ git config --global diff.renamelimit 1000 $ git diff

CLOBBER_CACHE_ALWAYS testing (was Re: Release 13 of the PostgreSQL BuildFarm client)

2021-08-03 Thread Tom Lane
Andrew Dunstan writes: > I have pushed Release 13 of the PostgreSQL BuildFarm client. > ... > the `use_discard_caches` setting reflects a change in the way postgres > handles this - it's now a runtime setting rather than a compile time > setting. On older branches it sets

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-03 Thread Gilles Darold
Le 03/08/2021 à 15:39, Tom Lane a écrit : Erik Rijkers writes: On 8/3/21 1:26 PM, Gilles Darold wrote: Le 03/08/2021 à 11:45, Gilles Darold a écrit : Actually I just found that the regexp_like() function doesn't support the start parameter which is something we should support. I saw that

Re: [PATCH]Comment improvement in publication.sql

2021-08-03 Thread vignesh C
On Tue, Aug 3, 2021 at 8:36 AM tanghy.f...@fujitsu.com wrote: > > On Monday, August 2, 2021 11:56 PM vignesh C wrote: > > > > Few minor suggestions: > > 1) Should we change below to "fail - tables can't be added, dropped or > > set to "FOR ALL TABLES" publications" > > --- fail - can't add to

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-08-03 Thread Pavel Borisov
пт, 23 июл. 2021 г. в 10:00, Greg Nancarrow : > On Thu, Jul 22, 2021 at 2:15 AM Robert Haas wrote: > > > > Thanks to Thomas Munro for drawing my attention to this thread. I > > wasn't intentionally ignoring it, but there's a lot of email in the > > world and only so much time. > > > > When I

Re: when the startup process doesn't (logging startup delays)

2021-08-03 Thread Robert Haas
On Tue, Aug 3, 2021 at 2:48 AM Nitin Jadhav wrote: > Implemented the above approach and verified the patch. Kindly have a > look and share your thoughts. +LogStartupProgressTimerExpired(bool force, long *secs, int *usecs) The name of this function begins with "log" but it does not log anything,

Re: Make relfile tombstone files conditional on WAL level

2021-08-03 Thread Robert Haas
On Mon, Aug 2, 2021 at 6:38 PM Andres Freund wrote: > What I proposed in the past was to have a new shared table that tracks > relfilenodes. I still think that's a decent solution for just the problem at > hand. It's not really clear to me what problem is at hand. The problems that the tombstone

Re: make MaxBackends available in _PG_init

2021-08-03 Thread Robert Haas
On Mon, Aug 2, 2021 at 4:37 PM Andres Freund wrote: > I'm not so sure it's a good idea. I've seen several shared_preload_library > using extensions that adjust some GUCs (e.g. max_prepared_transactions) > because they need some more resources internally - that's perhaps not a great > idea, but

Re: Commitfest overflow

2021-08-03 Thread Bruce Momjian
On Tue, Aug 3, 2021 at 04:53:40PM +0100, Simon Riggs wrote: > There are 273 patches in the queue for the Sept Commitfest already, so > it seems clear the queue is not being cleared down each CF as it was > before. We've been trying hard, but it's overflowing. > > Of those, about 50 items have

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-03 Thread Tom Lane
Erik Rijkers writes: > On 8/3/21 1:26 PM, Gilles Darold wrote: >> Le 03/08/2021 à 11:45, Gilles Darold a écrit : >>> Actually I just found that the regexp_like() function doesn't support >>> the start parameter which is something we should support. I saw that >>> Oracle do not support it but

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Bruce Momjian
On Thu, Jul 29, 2021 at 03:06:45PM -0400, Bruce Momjian wrote: > > I haven't seen it mentioned in the thread, but I think the final phrase > > of this should be a separate step, > > > > > > Copy custom full-text search files > > > > Copy any custom full text search file (dictionary,

Commitfest overflow

2021-08-03 Thread Simon Riggs
There are 273 patches in the queue for the Sept Commitfest already, so it seems clear the queue is not being cleared down each CF as it was before. We've been trying hard, but it's overflowing. Of those, about 50 items have been waiting more than one year, and about 25 entries waiting for more

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Bruce Momjian
On Fri, Jul 30, 2021 at 12:40:06PM -0400, Bruce Momjian wrote: > On Thu, Jul 29, 2021 at 06:19:56PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > I think we need to first give clear instructions on how to find out if > > > an extension update is available, and then how to update it. I

Re: Commitfest overflow

2021-08-03 Thread Tom Lane
Bruce Momjian writes: > On Tue, Aug 3, 2021 at 04:53:40PM +0100, Simon Riggs wrote: >> There are 273 patches in the queue for the Sept Commitfest already, so >> it seems clear the queue is not being cleared down each CF as it was >> before. We've been trying hard, but it's overflowing. > I

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Justin Pryzby
On Tue, Aug 03, 2021 at 12:00:47PM -0400, Bruce Momjian wrote: > Patch applied through 9.6. The comment seems to be a leftover from a copy pasto. + /* find hash indexes */ + res = executeQueryOrDie(conn, +

Re: Have I found an interval arithmetic bug?

2021-08-03 Thread Bruce Momjian
On Fri, Jul 30, 2021 at 03:03:13PM -0400, Bruce Momjian wrote: > > On output, the months field is shown as an appropriate number of > > years and months; the days field is shown as-is; the microseconds > > field is converted to hours, minutes, and possibly-fractional > >

Re: .ready and .done files considered harmful

2021-08-03 Thread Robert Haas
On Mon, Aug 2, 2021 at 9:06 AM Dipesh Pandit wrote: > We can maintain the current timeline ID in archiver specific shared memory. > If we switch to a new timeline then the backend process can update the new > timeline ID in shared memory. Archiver can keep a track of current timeline ID > and if

Re: [PATCH] proposal for regexp_count, regexp_instr, regexp_substr and regexp_replace

2021-08-03 Thread Tom Lane
Gilles Darold writes: > Sorry I have missed that, but I'm fine with this implemenation so let's > keep the v6 version of the patch and drop this one. Pushed, then. There's still lots of time to tweak the behavior of course. regards, tom lane

Re: Slow standby snapshot

2021-08-03 Thread Andres Freund
Hi, On 2021-08-03 10:33:50 +0500, Andrey Borodin wrote: > > 3 авг. 2021 г., в 03:01, Andres Freund написал(а): > > On 2021-08-03 00:07:23 +0300, Michail Nikolaev wrote: > >> The main idea is simple optimistic optimization - store offset to next > >> valid entry. So, in most cases, we could just

Re: Lowering the ever-growing heap->pd_lower

2021-08-03 Thread Matthias van de Meent
On Tue, 3 Aug 2021 at 08:57, Simon Riggs wrote: > > On Tue, 18 May 2021 at 20:33, Peter Geoghegan wrote: > > > > On Tue, May 18, 2021 at 12:29 PM Matthias van de Meent > > wrote: > > > PFA the updated version of this patch. Apart from adding line pointer > > > truncation in

Re: pg_upgrade does not upgrade pg_stat_statements properly

2021-08-03 Thread Bruce Momjian
On Tue, Aug 3, 2021 at 11:13:45AM -0500, Justin Pryzby wrote: > On Tue, Aug 03, 2021 at 12:00:47PM -0400, Bruce Momjian wrote: > > Patch applied through 9.6. > > The comment seems to be a leftover from a copy pasto. > > + /* find hash indexes */ > + res =

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-08-03 Thread Robert Haas
On Tue, Aug 3, 2021 at 9:59 AM Pavel Borisov wrote: > I've looked at v5 patch. It is completely similar to v2 patch, which I've > already tested using the workflow, I've described in the comments above. > Before the patch I get the errors quite soon, the patch corrects them. > Furthermore

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2021-08-03 Thread Andres Freund
Hi, On 2021-08-02 18:25:56 -0400, Melanie Plageman wrote: > Thanks for the feedback! > > I agree it makes sense to count strategy writes separately. > > I thought about this some more, and I don't know if it makes sense to > only count "avoidable" strategy writes. > > This would mean that a

Re: Commitfest overflow

2021-08-03 Thread Greg Stark
On Tue, 3 Aug 2021 at 11:56, Bruce Momjian wrote: > I wonder if our lack of in-person developer meetings is causing some of > our issues to not get closed. That's an interesting thought. Every year there are some especially contentious patches that don't get dealt with until the in-person

Re: .ready and .done files considered harmful

2021-08-03 Thread Bossart, Nathan
+ /* +* Perform a full directory scan to identify the next log segment. There +* may be one of the following scenarios which may require us to perform a +* full directory scan. +* +* 1. This is the first cycle since archiver has started and there is

Re: Reduce lock level for ALTER TABLE ... ADD CHECK .. NOT VALID

2021-08-03 Thread Simon Riggs
On Thu, 15 Jul 2021 at 07:47, Simon Riggs wrote: > > On Sat, Jul 10, 2021 at 2:50 PM John Naylor > wrote: > > On Thu, Apr 22, 2021 at 8:01 AM Simon Riggs > > wrote: > > > > > > 897795240cfaaed724af2f53ed2c50c9862f951f forgot to reduce the lock > > > level for CHECK constraints when allowing

Re: Failed transaction statistics to measure the logical replication progress

2021-08-03 Thread Amit Kapila
On Wed, Aug 4, 2021 at 6:19 AM Masahiko Sawada wrote: > > On Tue, Aug 3, 2021 at 6:11 PM Amit Kapila wrote: > > > > I was trying to think based on similar counters in pg_stat_database > > but if you think there is a value in showing errored and actual > > rollbacked transactions separately then

Re: Commitfest overflow

2021-08-03 Thread Thomas Munro
On Wed, Aug 4, 2021 at 8:23 AM Greg Stark wrote: > On Tue, 3 Aug 2021 at 11:56, Bruce Momjian wrote: > > > I wonder if our lack of in-person developer meetings is causing some of > > our issues to not get closed. > > That's an interesting thought. Every year there are some especially >

RE: Skipping logical replication transactions on subscriber side

2021-08-03 Thread houzj.f...@fujitsu.com
On Tuesday, August 3, 2021 2:49 PM Masahiko Sawada wrote: > > I've attached new patches that incorporate all comments I got so far. > Please review them. Hi, I had a few comments for the 0003 patch. 1). - This clause alters parameters originally set by - . See there for more -

Re: Use POPCNT on MSVC

2021-08-03 Thread Thomas Munro
On Tue, Aug 3, 2021 at 10:43 PM John Naylor wrote: > (Side note, but sort of related to #1 above: non-x86 platforms have to > indirect through a function pointer even though they have no fast > implementation to make it worth their while. It would be better for them if > the "slow"

RE: [HACKERS] logical decoding of two-phase transactions

2021-08-03 Thread tanghy.f...@fujitsu.com
On Tuesday, August 3, 2021 6:03 PM vignesh C wrote: > > On Tue, Aug 3, 2021 at 12:32 PM Amit Kapila wrote: > > > > On Tue, Aug 3, 2021 at 6:17 AM Peter Smith wrote: > > > > > > Please find attached the latest patch set v102* > > > > > > > I have made minor modifications in the comments and

Re: Autovacuum on partitioned table (autoanalyze)

2021-08-03 Thread Kyotaro Horiguchi
At Thu, 29 Jul 2021 18:03:55 -0700, Andres Freund wrote in > And if one instead inverts the order of pgstat_report_analyze() and > pgstat_report_anl_ancestors() one gets a slightly different problem: A manual > ANALYZE of the partition root results in the partition root having a non-zero >

Re: Use POPCNT on MSVC

2021-08-03 Thread David Rowley
On Tue, 3 Aug 2021 at 22:43, John Naylor wrote: > 1. the __popcnt64() intrinsic is put inside pg_popcount64_asm(), which is a > bit of a misnomer since it's not assembly. Renaming s/_asm/_fast/ would help > it look better. But then looking around, other platforms have intrinsics > coded, but

Re: Failed transaction statistics to measure the logical replication progress

2021-08-03 Thread Masahiko Sawada
On Tue, Aug 3, 2021 at 6:11 PM Amit Kapila wrote: > > On Tue, Aug 3, 2021 at 10:59 AM Masahiko Sawada wrote: > > > > On Tue, Aug 3, 2021 at 11:47 AM Amit Kapila wrote: > > > > > > On Mon, Aug 2, 2021 at 1:13 PM Masahiko Sawada > > > wrote: > > > > > > > > On Mon, Aug 2, 2021 at 2:52 PM

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-08-03 Thread Greg Nancarrow
On Wed, Aug 4, 2021 at 3:21 AM Robert Haas wrote: > >The idea I sort of had floating around in my mind is a little >different than what Greg has implemented. I was thinking that we could >just skip SerializeSnapshot and the corresponding shm_toc_allocate() >if !IsolationUsesXactSnapshot(). Then

Re: Lowering the ever-growing heap->pd_lower

2021-08-03 Thread Peter Geoghegan
On Mon, Aug 2, 2021 at 11:57 PM Simon Riggs wrote: > 1. Allow same thing as PageTruncateLinePointerArray() during HOT cleanup > That is going to have a clear benefit for HOT workloads, which by > their nature will use a lot of line pointers. Why do you say that? > Many applications are updated

Re: Lowering the ever-growing heap->pd_lower

2021-08-03 Thread Peter Geoghegan
On Tue, Aug 3, 2021 at 12:27 PM Matthias van de Meent wrote: > This change makes it easier and more worthwile to implement a further > optimization for the checkpointer and/or buffer manager to determine > that 1.) this page is now empty, and that 2.) we can therefore write a > specialized WAL

Re: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2021-08-03 Thread Masahiko Sawada
Hi, On Mon, Aug 2, 2021 at 10:52 PM houzj.f...@fujitsu.com wrote: > > > Hi hackers, > > When testing some other logical replication related patches, I found two > unexpected behaviours about ALTER SUBSCRIPTION ADD/DROP PUBLICATION. > > (1) > when I execute the following sqls[1], the data of

Re: Slow standby snapshot

2021-08-03 Thread Andres Freund
Hi, On 2021-08-03 22:23:58 +0300, Michail Nikolaev wrote: > > I'm not sure that we need to care as much about the cost of > > KnownAssignedXidsGetAndSetXmin() - for one, the caching we now have makes > > that > > less frequent. > > It is still about 5-7% of CPU for a typical workload, a

Re: Use generation context to speed up tuplesorts

2021-08-03 Thread Andres Freund
Hi, On 2021-08-03 10:59:25 +1200, David Rowley wrote: > On Sat, 31 Jul 2021 at 08:38, Andres Freund wrote: > > There is at least one path using tuplecontext that reaches code that > > could end up freeing memory to a significant enough degree to care about > > generation.c effectively not using

Re: archive status ".ready" files may be created too early

2021-08-03 Thread Bossart, Nathan
On 8/2/21, 7:37 PM, "Kyotaro Horiguchi" wrote: > I'm afraid that using hash to store boundary info is too much. Isn't a > ring buffer enough for this use? In that case it is enough to > remember only the end LSN of the segment spanning records. It is easy > to expand the buffer if needed. I

Re: A varint implementation for PG?

2021-08-03 Thread Robert Haas
[ resurrecting this 2-year-old thread ] On Fri, Dec 13, 2019 at 12:45 AM Andres Freund wrote: > > If baking a new variant integer format now, I think limiting it to 64 bits > > is probably a mistake given how long-lived PostgreSQL is, and how hard it > > can be to change things in the protocol,

Re: Commitfest overflow

2021-08-03 Thread Simon Riggs
On Tue, 3 Aug 2021 at 17:13, Tom Lane wrote: > > Bruce Momjian writes: > > On Tue, Aug 3, 2021 at 04:53:40PM +0100, Simon Riggs wrote: > >> There are 273 patches in the queue for the Sept Commitfest already, so > >> it seems clear the queue is not being cleared down each CF as it was > >>

Re: Commitfest overflow

2021-08-03 Thread Ibrar Ahmed
On Tue, Aug 3, 2021 at 9:13 PM Tom Lane wrote: > Bruce Momjian writes: > > On Tue, Aug 3, 2021 at 04:53:40PM +0100, Simon Riggs wrote: > >> There are 273 patches in the queue for the Sept Commitfest already, so > >> it seems clear the queue is not being cleared down each CF as it was > >>

Re: Commitfest overflow

2021-08-03 Thread Bruce Momjian
On Tue, Aug 3, 2021 at 07:30:38PM +0100, Simon Riggs wrote: > I would still ask for someone to spend a little time triaging things, > so as to direct people who volunteer to be so directed. Many will not > want to be directed, but I'm sure there must be 5-10 people who would > do that?

Re: Slow standby snapshot

2021-08-03 Thread Michail Nikolaev
Hello, Andres. Thanks for your feedback. >> Maybe use a hashtable of running transactions? It will be slightly faster >> when adding\removing single transactions. But much worse when doing >> KnownAssignedXidsRemove(). > Why would it be worse for KnownAssignedXidsRemove()? Were you intending to

Re: Commitfest overflow

2021-08-03 Thread Tomas Vondra
On 8/3/21 8:57 PM, Bruce Momjian wrote: On Tue, Aug 3, 2021 at 08:51:57PM +0200, Tomas Vondra wrote: How would this be different from the CFM just rejecting patches? It does not matter if there's an explicit number of patches that we allow to be moved to the next CF - someone still needs to

Re: A varint implementation for PG?

2021-08-03 Thread Tom Lane
Robert Haas writes: > However, I suspect that the whole approach should be completely > revised for a user-visible data type. On the one hand, there's no > telling how large a value some user will want to represent, so > limiting ourselves to 64 bits does seem shortsighted. And on the othe >

Re: Commitfest overflow

2021-08-03 Thread Tom Lane
Tomas Vondra writes: > But it's not clear to me whether you're arguing for CFM to assess this, > or whether someone else should make this decision? > IMHO asking the CFM to do this would be a tremendous burden - properly > assessing 50+ patches is a lot of work, and probably requires a fairly

Re: Commitfest overflow

2021-08-03 Thread Bruce Momjian
On Tue, Aug 3, 2021 at 03:42:57PM -0400, Tom Lane wrote: > Tomas Vondra writes: > > But it's not clear to me whether you're arguing for CFM to assess this, > > or whether someone else should make this decision? > > > IMHO asking the CFM to do this would be a tremendous burden - properly > >

Re: Autovacuum on partitioned table (autoanalyze)

2021-08-03 Thread Andrew Dunstan
On 7/29/21 9:03 PM, Andres Freund wrote: > Hi, > > CCing RMT because I think we need to do something about this for v14. Thanks. We are now aware of it. [...] > I don't think the code as is is fit for v14. It looks like it was rewritten > with a new approach just before the freeze ([1]),

Re: Remove unused 'len' from pg_stat_recv_* functions

2021-08-03 Thread Andres Freund
On 2021-08-03 16:56:12 +0900, Masahiko Sawada wrote: > On Tue, Aug 3, 2021 at 3:17 PM Kyotaro Horiguchi > wrote: > > > > At Tue, 3 Aug 2021 12:40:23 +0900, Masahiko Sawada > > wrote in > > > Hi all, > > > > > > While working on a patch adding new stats, houzj pointed out that > > > 'len'

Re: Commitfest overflow

2021-08-03 Thread Ibrar Ahmed
On Tue, Aug 3, 2021 at 11:31 PM Simon Riggs wrote: > On Tue, 3 Aug 2021 at 17:13, Tom Lane wrote: > > > > Bruce Momjian writes: > > > On Tue, Aug 3, 2021 at 04:53:40PM +0100, Simon Riggs wrote: > > >> There are 273 patches in the queue for the Sept Commitfest already, so > > >> it seems clear

Re: Lowering the ever-growing heap->pd_lower

2021-08-03 Thread Simon Riggs
On Tue, 3 Aug 2021 at 17:15, Matthias van de Meent wrote: > and further future optimizations might include > > - Full-page WAL logging of empty pages produced in the checkpointer > could potentially be optimized to only log 'it's an empty page' > instead of writing out the full 8kb page, which

Re: Commitfest overflow

2021-08-03 Thread Tomas Vondra
On 8/3/21 8:30 PM, Simon Riggs wrote: On Tue, 3 Aug 2021 at 17:13, Tom Lane wrote: Bruce Momjian writes: On Tue, Aug 3, 2021 at 04:53:40PM +0100, Simon Riggs wrote: There are 273 patches in the queue for the Sept Commitfest already, so it seems clear the queue is not being cleared down

Re: Commitfest overflow

2021-08-03 Thread Bruce Momjian
On Tue, Aug 3, 2021 at 08:51:57PM +0200, Tomas Vondra wrote: > How would this be different from the CFM just rejecting patches? It does not > matter if there's an explicit number of patches that we allow to be moved to > the next CF - someone still needs to make the decision, and I agree with Tom

Re: Lowering the ever-growing heap->pd_lower

2021-08-03 Thread Matthias van de Meent
On Tue, 3 Aug 2021 at 20:37, Simon Riggs wrote: > > On Tue, 3 Aug 2021 at 17:15, Matthias van de Meent > wrote: > > > and further future optimizations might include > > > > - Full-page WAL logging of empty pages produced in the checkpointer > > could potentially be optimized to only log 'it's an

Re: A varint implementation for PG?

2021-08-03 Thread Andres Freund
Hi, On 2021-08-03 14:26:16 -0400, Robert Haas wrote: > [ resurrecting this 2-year-old thread ] > On Fri, Dec 13, 2019 at 12:45 AM Andres Freund wrote: > > I don't think it's ever going to be sensible to transport 64bit quanta > > of data. Also, uh, it'd be larger than the data a postgres

Re: Commitfest overflow

2021-08-03 Thread Bruce Momjian
On Tue, Aug 3, 2021 at 09:36:41PM +0200, Tomas Vondra wrote: > On 8/3/21 8:57 PM, Bruce Momjian wrote: > > On Tue, Aug 3, 2021 at 08:51:57PM +0200, Tomas Vondra wrote: > > > How would this be different from the CFM just rejecting patches? It does > > > not > > > matter if there's an explicit