Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread David G. Johnston
On Sun, Jan 23, 2022 at 11:55 PM Masahiko Sawada wrote: > On Mon, Jan 24, 2022 at 1:49 PM David G. Johnston > wrote: > > > > On Sun, Jan 23, 2022 at 8:35 PM Amit Kapila > wrote: > >> > >> > I really dislike the user experience this provides, and given it is > new in v15 (and right now this

Re: PSA: Autoconf has risen from the dead

2022-01-24 Thread Andres Freund
Hi, On 2022-01-23 11:29:17 -0500, Tom Lane wrote: > Right now, I'm not sure we care; there seems to be more > enthusiasm for switching to meson. But if that idea falls > through, we should update to a newer autoconf release. Depending on the number of portability fixes in those releases the

Re: row filtering for logical replication

2022-01-24 Thread Peter Smith
Thanks for all the patches! Here are my review comments for v69-0001 ~~~ 1. src/backend/executor/execReplication.c CheckCmdReplicaIdentity call to RelationBuildPublicationDesc + /* + * It is only safe to execute UPDATE/DELETE when all columns, referenced in + * the row filters from

Re: PSA: Autoconf has risen from the dead

2022-01-24 Thread Peter Eisentraut
On 23.01.22 17:29, Tom Lane wrote: While chasing something else, I was surprised to learn that the Autoconf project has started to make releases again. There are 2.70 (2020-12-08) and 2.71 (2021-01-28) versions available at https://ftp.gnu.org/gnu/autoconf/ I have patches ready for this at

Re: XLogReadRecord() error in XlogReadTwoPhaseData()

2022-01-24 Thread Noah Misch
On Sun, Jan 23, 2022 at 06:34:32PM -0800, Andres Freund wrote: > On 2022-01-23 18:10:07 -0800, Noah Misch wrote: > > On Sun, Jan 23, 2022 at 05:40:54PM -0800, Andres Freund wrote: > > > Test::more's description: "If it's something the programmer hasn't done > > > yet, > > > use TODO. This is for

Re: BufferAlloc: don't take two simultaneous locks

2022-01-24 Thread Kyotaro Horiguchi
At Sat, 22 Jan 2022 12:56:14 +0500, Andrey Borodin wrote in > I've took a look into the patch. The idea seems reasonable to me: > clearing\evicting old buffer and placing new one seem to be > different units of work, there is no need to couple both partition > locks together. And the claimed

Re: Error running configure on Mac

2022-01-24 Thread Andres Freund
Hi, On 2022-01-23 23:44:11 -0800, samay sharma wrote: > I also see many more error messages in config.log when I grep for error. > So, I've attached the entire file in case any other output is useful. The important lines seem to be: configure:4591: gcc -c -g -O2 conftest.c >&5 In file

Re: Non-decimal integer literals

2022-01-24 Thread Peter Eisentraut
Rebased patch set On 13.01.22 14:42, Peter Eisentraut wrote: Another modest update, because of the copyright year update preventing the previous patches from applying cleanly. I also did a bit of work on the ecpg scanner so that it also handles some errors on par with the main scanner.

Re: Support tab completion for upper character inputs in psql

2022-01-24 Thread Peter Eisentraut
On 20.01.22 08:37, tanghy.f...@fujitsu.com wrote: 1. The downcasing logic in the patch bears very little resemblance to the backend's actual downcasing logic, which can be found in src/backend/parser/scansup.c's downcase_identifier(). Notably, the patch's restriction to only convert all-ASCII

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread Amit Kapila
On Mon, Jan 24, 2022 at 1:30 PM David G. Johnston wrote: > > That said, at present my two dislikes: > > 1) ALTER SYSTEM SKIP accepts any xid value (I need to consider further the > timing of when this resets to zero) > I think this is required for future extension of this feature wherein I

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread Masahiko Sawada
On Mon, Jan 24, 2022 at 5:00 PM David G. Johnston wrote: > > On Sun, Jan 23, 2022 at 11:55 PM Masahiko Sawada > wrote: >> > >Similarly, the same is true > >for clearing subskipxid. > > I'm confused - pg_subscription is a catalog, not a stat view. Why is it > affected? Sorry, I mistook

Re: row filtering for logical replication

2022-01-24 Thread Amit Kapila
On Mon, Jan 24, 2022 at 1:19 PM Greg Nancarrow wrote: > > On Mon, Jan 24, 2022 at 5:09 PM Amit Kapila wrote: > > But that is not what I am seeing in Logs with a test case where the > > row filter column has NULL values. Could you please try that see what > > is printed in LOG? > > > > You can

DSA failed to allocate memory

2022-01-24 Thread Dongming Liu
Hi, I am using Dynamic shared memory areas(DSA) to manage some variable length shared memory, I've found that in some cases allocation fails even though there are enough contiguous pages. The steps to reproduce are as follows: 1. create a dsa area with a 1MB DSM segment 2. set its size limit to

Re: Make mesage at end-of-recovery less scary.

2022-01-24 Thread Pavel Borisov
> > d2ddfa681db27a138acb63c8defa8cc6fa588922 removed global variables > ReadRecPtr and EndRecPtr. This is rebased version that reads the LSNs > directly from xlogreader instead of the removed global variables. > Hi, hackers! I've checked the latest version of a patch. It applies cleanly,

Re: drop tablespace failed when location contains .. on win32

2022-01-24 Thread Michael Paquier
On Mon, Jan 24, 2022 at 11:21:12AM +, wangsh.f...@fujitsu.com wrote: > Tom Lane wrote: >>> I concur with the upthread comments that there's little chance >>> we'll commit 0003 as-is; the code-to-benefit ratio is too high. >>> Instead, you might consider adding test_canonicalize_path in >>>

RE: drop tablespace failed when location contains .. on win32

2022-01-24 Thread wangsh.f...@fujitsu.com
Hi, The new version is attached. Tom Lane wrote: > I tried to read 0001 but really couldn't make sense of the logic > at all, because it's seriously underdocumented. At minimum you > need an API spec comment for canonicalize_path_sub, explaining > what it's supposed to do and why. I have

Re: [BUG]Update Toast data failure in logical replication

2022-01-24 Thread Amit Kapila
On Mon, Jan 24, 2022 at 9:28 AM Michael Paquier wrote: > > On Wed, Aug 11, 2021 at 06:14:55PM +0530, Dilip Kumar wrote: > > Right > > Amit, are you planning to look more at this patch? It has been a > couple of months since the last update, and this is still a bug as far > as I understand. > >

Re: Fix BUG #17335: Duplicate result rows in Gather node

2022-01-24 Thread Yura Sokolov
В Вс, 23/01/2022 в 14:56 +0300, Yura Sokolov пишет: > В Чт, 20/01/2022 в 09:32 +1300, David Rowley пишет: > > On Fri, 31 Dec 2021 at 00:14, Yura Sokolov wrote: > > > Suggested quick (and valid) fix in the patch attached: > > > - If Append has single child, then copy its parallel awareness. > > >

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread Peter Eisentraut
On 22.01.22 03:54, Amit Kapila wrote: Won't we already do that for Alter Subscription command which means nothing special needs to be done for this? However, it seems to me that the idea we are trying to follow here is that as this option can lead to data inconsistency, it is good to allow only

Re: missing indexes in indexlist with partitioned tables

2022-01-24 Thread Arne Roland
Hi! > From: Zhihong Yu > Subject: Re: missing indexes in indexlist with partitioned tables > > Hi, > > - if (indexRelation->rd_rel->relkind == RELKIND_PARTITIONED_INDEX) > + if (inhparent && (!index->indisunique || > indexRelation->rd_rel->relkind !=

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread David G. Johnston
On Monday, January 24, 2022, Amit Kapila wrote: > On Mon, Jan 24, 2022 at 1:30 PM David G. Johnston > wrote: > > > > That said, at present my two dislikes: > > > > 1) ALTER SYSTEM SKIP accepts any xid value (I need to consider further > the timing of when this resets to zero) > > > > I think

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread Peter Eisentraut
On 22.01.22 10:41, Amit Kapila wrote: Additionally, the description for pg_stat_subscription_workers should describe what happens once the transaction represented by last_error_xid has either been successfully processed or skipped. Does this "last error" stick around until another error

Re: PSA: Autoconf has risen from the dead

2022-01-24 Thread Tom Lane
Peter Eisentraut writes: > I have patches ready for this at > https://github.com/petere/postgresql/tree/autoconf-updates. > My thinking was to wait until Autoconf 2.71 has trickled down into the > OS versions that developers are likely to use. I find that kind of irrelevant, because we expect

Re: refactoring basebackup.c

2022-01-24 Thread Dipesh Pandit
Hi, > Here is a more detailed review. Thanks for the feedback, I have incorporated the suggestions and updated a new version of the patch (v3-0001). The required documentation changes are also incorporated in updated patch (v3-0001). > Interesting approach. This unfortunately has the effect of

Re: makefiles writing to $@ should first write to $@.new

2022-01-24 Thread Tom Lane
Julien Rouhaud writes: > On Mon, Jan 24, 2022 at 12:41:49PM +0900, Michael Paquier wrote: >> Honestly, I am not sure that this worth bothering about. This comes >> down to a balance between the code complexity and the likelihood of a >> failure, and the odds are not in favor of the later IMO.

Re: automatically generating node support functions

2022-01-24 Thread Peter Eisentraut
Rebased patch to resolve some merge conflicts On 29.12.21 12:08, Peter Eisentraut wrote: On 12.10.21 15:52, Andrew Dunstan wrote: I haven't been through the whole thing, but I did notice this: the comment stripping code looks rather fragile. I think it would blow up if there were a

Re: UNIQUE null treatment option

2022-01-24 Thread Peter Eisentraut
On 13.01.22 19:36, Peter Geoghegan wrote: I wonder if the logic for setting BTScanInsertData.anynullkeys inside _bt_mkscankey() is the place to put your test for rel->rd_index->indnullsnotdistinct -- not inside _bt_doinsert(). That would probably necessitate renaming anynullkeys, but that's

Re: PSA: Autoconf has risen from the dead

2022-01-24 Thread Peter Eisentraut
On 24.01.22 15:14, Tom Lane wrote: Do these versions fix any bugs that affect us (i.e., that we've not already created workarounds for)? The only thing that could be of interest is that the workaround we are carrying in config/check_decls.m4 was originally upstreamed by Noah, but was then

Re: Index Skip Scan (new UniqueKeys)

2022-01-24 Thread Dmitry Dolgov
> On Sun, Jan 23, 2022 at 04:25:04PM -0800, Zhihong Yu wrote: > On Sat, Jan 22, 2022 at 1:32 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Besides that the new patch version contains some cleaning up and > > addresses commentaries around leaf page pinning from [1]. The idea > > behind the

Re: Index Skip Scan (new UniqueKeys)

2022-01-24 Thread Zhihong Yu
On Mon, Jan 24, 2022 at 8:51 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Sun, Jan 23, 2022 at 04:25:04PM -0800, Zhihong Yu wrote: > > On Sat, Jan 22, 2022 at 1:32 PM Dmitry Dolgov <9erthali...@gmail.com> > wrote: > > > Besides that the new patch version contains some cleaning up and >

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Robert Haas
On Mon, Jan 24, 2022 at 2:27 PM Robert Haas wrote: > I really hate committing stuff that turns out to be broken. It's such > a fire drill when the build farm turns red. And there's a good chance it's about to break again, because I just committed the next patch in the series which, shockingly,

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-24 Thread Tom Lane
Andres Freund writes: > On 2022-01-14 17:51:52 -0500, Tom Lane wrote: >> FWIW, I'm just fine with reverting, particularly in the back branches. >> It seems clear that this dank corner of Windows contains even more >> creepy-crawlies than we thought. > Seems we should revert now-ish? There's a

Re: makefiles writing to $@ should first write to $@.new

2022-01-24 Thread Peter Eisentraut
On 24.01.22 04:23, Justin Pryzby wrote: There are many Makefile rules like foo: bar ./tool $< > $@ If the rule is interrupted (due to ^C or ENOSPC), foo can be 0 bytes or partially written, but won't be rebuilt until someone runs distclean or debugs it and removes the individual file,

Re: pg_upgrade should truncate/remove its logs before running

2022-01-24 Thread Justin Pryzby
On Mon, Jan 24, 2022 at 12:39:30PM -0500, Bruce Momjian wrote: > On Mon, Jan 24, 2022 at 10:59:40AM +0900, Michael Paquier wrote: > > On Thu, Jan 20, 2022 at 07:51:37PM +0900, Michael Paquier wrote: > > > Neat idea. That would work fine for my case. So I am fine to stick > > > with this

Re: pg_upgrade should truncate/remove its logs before running

2022-01-24 Thread Bruce Momjian
On Mon, Jan 24, 2022 at 10:59:40AM +0900, Michael Paquier wrote: > On Thu, Jan 20, 2022 at 07:51:37PM +0900, Michael Paquier wrote: > > Neat idea. That would work fine for my case. So I am fine to stick > > with this suggestion. > > I have been looking at this idea, and the result is quite

Re: refactoring basebackup.c

2022-01-24 Thread Robert Haas
On Mon, Jan 24, 2022 at 9:30 AM Dipesh Pandit wrote: > v13 patch does not apply on the latest head, it requires a rebase. I have > applied > it on commit dc43fc9b3aa3e0fa9c84faddad6d301813580f88 to validate gzip > decompression patches. It only needed trivial rebasing; I have committed it after

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2022-01-24 Thread Anton A. Melnikov
Hi, Andrey! I've checked the 5th version of the patch and there are some remarks. >I've created a new view named pg_stat_statements_aux. But for now both >views are using the same function pg_stat_statements which returns all >fields. It seems reasonable to me - if sampling solution will need

Re: [BUG]Update Toast data failure in logical replication

2022-01-24 Thread Euler Taveira
On Mon, Jan 24, 2022, at 12:58 AM, Michael Paquier wrote: > FWIW, I find the API changes of HeapDetermineModifiedColumns() and > ExtractReplicaIdentity() a bit grotty. Shouldn't we try to flatten > the old tuple instead? There are things like > toast_flatten_tuple_to_datum() for this purpose if

Re: [BUG]Update Toast data failure in logical replication

2022-01-24 Thread Robert Haas
On Tue, Aug 10, 2021 at 1:20 AM Amit Kapila wrote: > It seems to me this problem exists from the time we introduced > wal_level = logical in the commit e55704d8b2 [1], or another > possibility is that logical replication commit didn't consider > something to make it work. Andres, Robert, Petr,

Re: Catalog version access

2022-01-24 Thread Bossart, Nathan
Thanks for taking a look! On 1/23/22, 7:31 PM, "Michael Paquier" wrote: > On Mon, Aug 16, 2021 at 06:12:54PM +, Bossart, Nathan wrote: >> I was looking at the --check switch for the postgres binary recently >> [0], and this sounds like something that might fit in nicely there. >> In the

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-24 Thread Bruce Momjian
On Sat, Jan 22, 2022 at 12:47:35PM +0530, Shruthi Gowda wrote: > On Sat, Jan 22, 2022 at 12:27 AM Tom Lane wrote: > > > > Robert Haas writes: > > > It seems to me that what this comment is saying is that OIDs in the > > > second and third categories are doled out by counters. Therefore, we > > >

explain_regress, explain(MACHINE), and default to explain(BUFFERS) (was: BUFFERS enabled by default in EXPLAIN (ANALYZE))

2022-01-24 Thread Justin Pryzby
I'm renaming this thread for better visibility, since buffers is a small, optional part of the patches I sent. I made a CF entry here. https://commitfest.postgresql.org/36/3409/ On Wed, Dec 01, 2021 at 06:58:20PM -0600, Justin Pryzby wrote: > On Mon, Nov 15, 2021 at 01:09:54PM -0600, Justin

Re: CREATEROLE and role ownership hierarchies

2022-01-24 Thread Andrew Dunstan
On 1/22/22 16:20, Stephen Frost wrote: >> Subject: [PATCH v4 1/5] Add tests of the CREATEROLE attribute. > No particular issue with this one. > > I'm going to commit this piece forthwith so we get it out of the way. That will presumably make the cfbot unhappy until Mark submits a new patch set.

Re: pg_upgrade should truncate/remove its logs before running

2022-01-24 Thread Bruce Momjian
On Mon, Jan 24, 2022 at 11:41:17AM -0600, Justin Pryzby wrote: > On Mon, Jan 24, 2022 at 12:39:30PM -0500, Bruce Momjian wrote: > > On Mon, Jan 24, 2022 at 10:59:40AM +0900, Michael Paquier wrote: > > > On Thu, Jan 20, 2022 at 07:51:37PM +0900, Michael Paquier wrote: > > > > Neat idea. That would

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-24 Thread Robert Haas
On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: > Agree. In the latest patch, the template0 and postgres OIDs are fixed > to unused manually assigned OIDs 4 and 5 respectively. These OIDs are > no more listed as unused OIDs. Thanks. Committed with a few more cosmetic changes. -- Robert

Re: CREATEROLE and role ownership hierarchies

2022-01-24 Thread Robert Haas
On Sat, Jan 22, 2022 at 4:20 PM Stephen Frost wrote: > Whoah, really? No, I don't agree with this, it's throwing away the > entire concept around inheritance of role rights and how you can have > roles which you can get the privileges of by doing a SET ROLE to them > but you don't automatically

Re: Corruption during WAL replay

2022-01-24 Thread Daniel Shelepanov
On 27.09.2021 11:30, Kyotaro Horiguchi wrote: Thank you for the comments! (Sorry for the late resopnse.) At Tue, 10 Aug 2021 14:14:05 -0400, Robert Haas wrote in On Thu, Mar 4, 2021 at 10:01 PM Kyotaro Horiguchi wrote: The patch assumed that CHKPT_START/COMPLETE barrier are exclusively

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-24 Thread Peter Eisentraut
On 24.01.22 03:53, Andres Freund wrote: On 2022-01-23 21:31:52 -0500, Tom Lane wrote: Andres Freund writes: No, not really. There generally seems to be very little documentation about what one is supposed to use when embedding python (rather than building a python module). The only thing I

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Andrew Dunstan
On 1/23/22 22:52, Robert Haas wrote: > On Sun, Jan 23, 2022 at 4:09 PM Andrew Dunstan wrote: >> The most common issues we get are around this issue of virtualized paths >> in the TAP tests. If people followed the rule I suggested upthread, 99% >> of those problems would go away. > Well, that

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-24 Thread Andres Freund
Hi, On 2022-01-24 21:48:21 +0100, Peter Eisentraut wrote: > Also note that python-config is itself a Python script that uses sysconfig > and includes code like this: Huh. It's a shell script on my debian system. Looks like the python source tree has both. Not sure what / who decides which is

Re: Printing backtrace of postgres processes

2022-01-24 Thread Fujii Masao
On 2022/01/24 16:35, torikoshia wrote: On 2022-01-14 19:48, Bharath Rupireddy wrote: On Sat, Nov 20, 2021 at 11:50 AM Bharath Rupireddy wrote: On Fri, Nov 19, 2021 at 4:07 PM vignesh C wrote: > The Attached v15 patch has the fixes for the same. Thanks. The v15 patch LGTM and the cf bot

Re: DSA failed to allocate memory

2022-01-24 Thread Robert Haas
On Mon, Jan 24, 2022 at 4:59 AM Dongming Liu wrote: > Maybe we can use one of the following methods to fix it: > 1. re-bin segment to suitable segment index when called dsa_free > 2. get_best_segment search all bins (2) is definitely the wrong idea. The comments say: /* * What is the lowest

Re: Error running configure on Mac

2022-01-24 Thread Andres Freund
Hi, On 2022-01-24 08:41:39 -0800, samay sharma wrote: > I've also attached the test.i file. The problem is that you got a stdint.h in /usr/local/include/. And that stdint.h doesn't match the system one. Which explains why there's a compilation failure and also explains why others don't see this

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-24 Thread Andres Freund
Hi, On 2022-01-14 17:51:52 -0500, Tom Lane wrote: > FWIW, I'm just fine with reverting, particularly in the back branches. > It seems clear that this dank corner of Windows contains even more > creepy-crawlies than we thought. Seems we should revert now-ish? There's a minor release coming up and

TAP tests, directories and parameters

2022-01-24 Thread Andres Freund
Hi, Right now we run tap tests separately in each directory. Which is one of the reasons the make output is so unreadable - instead of having one 'prove' output listing all tests, we get a lot of different prove outputs, all interspersed. And it severely limits parallelism on windows right now.

Re: TAP tests, directories and parameters

2022-01-24 Thread Tom Lane
Andres Freund writes: > It's currently not possible to just run all tap tests in one prove run, > because a number of tests assume that they are run from specific directories > and/or with per-directory parameters. > For meson I "solved" this by running each individual test in a wrapper that >

Re: fairywren is generating bogus BASE_BACKUP commands

2022-01-24 Thread Robert Haas
On Mon, Jan 24, 2022 at 2:01 PM Andrew Dunstan wrote: > Well if we can get Andres' suggestion to work all this might go away, > which would keep everyone happy, especially me. You're right that I was > a little careless upthread. Mea culpa. Meanwhile I am committing a > minimal one line fix. I

Re: PublicationActions - use bit flags.

2022-01-24 Thread Peter Eisentraut
On 21.01.22 01:05, Greg Nancarrow wrote: On Tue, Dec 21, 2021 at 12:55 PM Greg Nancarrow wrote: On Tue, Dec 21, 2021 at 11:56 AM Tom Lane wrote: Removing this is not good: if (relation->rd_pubactions) - { pfree(relation->rd_pubactions); -

Re: Non-decimal integer literals

2022-01-24 Thread Robert Haas
On Mon, Jan 24, 2022 at 3:41 AM Peter Eisentraut wrote: > Rebased patch set What if someone finds this new behavior too permissive? -- Robert Haas EDB: http://www.enterprisedb.com

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread Amit Kapila
On Mon, Jan 24, 2022 at 7:36 PM Peter Eisentraut wrote: > > On 22.01.22 03:54, Amit Kapila wrote: > > Won't we already do that for Alter Subscription command which means > > nothing special needs to be done for this? However, it seems to me > > that the idea we are trying to follow here is that

Re: Schema variables - new implementation for Postgres 15

2022-01-24 Thread Julien Rouhaud
Hi, On Mon, Jan 24, 2022 at 12:33:11PM +0100, Pavel Stehule wrote: > > here is updated patch with locking support Thanks for updating the patch! While the locking is globally working as intended, I found a few problems with it. First, I don't think that acquiring the lock in

Re: row filtering for logical replication

2022-01-24 Thread Peter Smith
A couple more comments for the v69-0001 TAP tests. ~~~ 1. src/test/subscription/t/027_row_filter.pl +# The subscription of the ALL TABLES IN SCHEMA publication means there should be +# no filtering on the tablesync COPY, so all expect all 5 will be present. +$result =

Re: [BUG]Update Toast data failure in logical replication

2022-01-24 Thread Amit Kapila
On Tue, Jan 25, 2022 at 12:26 AM Euler Taveira wrote: > > On Mon, Jan 24, 2022, at 12:58 AM, Michael Paquier wrote: > > FWIW, I find the API changes of HeapDetermineModifiedColumns() and > ExtractReplicaIdentity() a bit grotty. Shouldn't we try to flatten > the old tuple instead? There are

Re: Fix BUG #17335: Duplicate result rows in Gather node

2022-01-24 Thread David Rowley
On Tue, 25 Jan 2022 at 17:35, Yura Sokolov wrote: > And another attempt to fix tests volatility. FWIW, I had not really seen the point in adding a test for this. I did however see a point in it with your original patch. It seemed useful there to verify that Gather and GatherMerge did what we

Re: GUC flags

2022-01-24 Thread Michael Paquier
On Mon, Jan 24, 2022 at 07:07:29PM -0600, Justin Pryzby wrote: > I think you'll find that this is how it's done elsewhere in postgres. > In the frontend, see appendPQExpBufferChar and appendPGArray and 3e6e86abc. > On the backend, see: git grep -F "'{'" |grep -w appendStringInfoChar Yeah, I was

RE: refactoring basebackup.c

2022-01-24 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hi, Thank you for committing a great feature. I have tested the committed features. The attached small patch fixes the output of the --help message. In the previous commit, only gzip and none were output, but in the attached patch, client-gzip and server-gzip are added. Regards, Noriyoshi

Re: Fix BUG #17335: Duplicate result rows in Gather node

2022-01-24 Thread Yura Sokolov
В Пн, 24/01/2022 в 16:24 +0300, Yura Sokolov пишет: > В Вс, 23/01/2022 в 14:56 +0300, Yura Sokolov пишет: > > В Чт, 20/01/2022 в 09:32 +1300, David Rowley пишет: > > > On Fri, 31 Dec 2021 at 00:14, Yura Sokolov > > > wrote: > > > > Suggested quick (and valid) fix in the patch attached: > > > > -

Re: CREATEROLE and role ownership hierarchies

2022-01-24 Thread Fujii Masao
On 2022/01/25 8:18, Mark Dilger wrote: On Jan 24, 2022, at 2:21 PM, Stephen Frost wrote: Superuser is a problem specifically because it gives people access to do absolutely anything, both for security and safety concerns. Disallowing a way to curtail that same risk when it comes to

Re: autovacuum prioritization

2022-01-24 Thread Dilip Kumar
On Fri, Jan 21, 2022 at 12:54 AM Robert Haas wrote: > So at a high level, I think that what we ought to do is, first, for > each table, estimate the time at which we think something bad will > occur. There are several bad events: too much bloat, XID wraparound, > MXID wraparound. We need to

RE: Support tab completion for upper character inputs in psql

2022-01-24 Thread tanghy.f...@fujitsu.com
On Monday, January 24, 2022 6:36 PM, Peter Eisentraut wrote: > The way your patch works now is that the case-insensitive behavior you > are implementing only works if the client encoding is a single-byte > encoding. This isn't what downcase_identifier() does; > downcase_identifier() always

Re: Correct error message for end-of-recovery record TLI

2022-01-24 Thread Michael Paquier
On Wed, Dec 01, 2021 at 07:09:34PM +, Bossart, Nathan wrote: > The patch no longer applied, so I went ahead and rebased it. This was on the CF stack for some time, so applied. I have also changed the messages produced for the shutdown and online checkpoint records as they used the same

Re: Printing backtrace of postgres processes

2022-01-24 Thread vignesh C
On Mon, Jan 24, 2022 at 10:06 PM Fujii Masao wrote: > > > > On 2022/01/24 16:35, torikoshia wrote: > > On 2022-01-14 19:48, Bharath Rupireddy wrote: > >> On Sat, Nov 20, 2021 at 11:50 AM Bharath Rupireddy > >> wrote: > >>> > >>> On Fri, Nov 19, 2021 at 4:07 PM vignesh C wrote: > >>> > The

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-01-24 Thread Michael Paquier
On Tue, Nov 09, 2021 at 03:23:07PM +0100, Daniel Gustafsson wrote: > Looking at this thread I think it makes sense to go ahead with this patch. > The > filter functionality worked on in another thread is dealing with > cherry-picking > certain objects where this is an all-or-nothing switch, so

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-24 Thread Tom Lane
Thomas Munro writes: > On Tue, Jan 25, 2022 at 10:28 AM Andres Freund wrote: >> On 2022-01-24 15:35:25 -0500, Tom Lane wrote: >>> Sure. Do we want to revert in HEAD too? >> Not sure. I'm also OK with trying to go with Thomas' patch to walreceiver and >> try a bit longer to get all this

Re: 2022-01 Commitfest

2022-01-24 Thread Julien Rouhaud
Hi, This is the 4th week of this commitfest. Since last week, 5 entries were committed. There are still overall 223 active patches, the vast majority needing review. If you signed up to review patches, you still have a whole week to help patch making progress and getting committed! Status

Re: Bug in ProcArrayApplyRecoveryInfo for snapshots crossing 4B, breaking replicas

2022-01-24 Thread Michael Paquier
On Mon, Jan 24, 2022 at 10:45:48PM +0100, Tomas Vondra wrote: > On 1/24/22 22:28, Bossart, Nathan wrote: >>> Attached patch is fixing this by just sorting the XIDs logically. The >>> xidComparator is meant for places that can't do logical ordering. But >>> these XIDs come from RUNNING_XACTS, so

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-01-24 Thread Shruthi Gowda
On Tue, Jan 25, 2022 at 1:14 AM Robert Haas wrote: > > On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: > > Agree. In the latest patch, the template0 and postgres OIDs are fixed > > to unused manually assigned OIDs 4 and 5 respectively. These OIDs are > > no more listed as unused OIDs. > >

Re: Printing backtrace of postgres processes

2022-01-24 Thread vignesh C
On Mon, Jan 24, 2022 at 1:05 PM torikoshia wrote: > > On 2022-01-14 19:48, Bharath Rupireddy wrote: > > On Sat, Nov 20, 2021 at 11:50 AM Bharath Rupireddy > > wrote: > >> > >> On Fri, Nov 19, 2021 at 4:07 PM vignesh C wrote: > >> > The Attached v15 patch has the fixes for the same. > >> > >>

Re: Correct error message for end-of-recovery record TLI

2022-01-24 Thread Amul Sul
On Tue, Jan 25, 2022 at 10:08 AM Michael Paquier wrote: > > On Wed, Dec 01, 2021 at 07:09:34PM +, Bossart, Nathan wrote: > > The patch no longer applied, so I went ahead and rebased it. > > This was on the CF stack for some time, so applied. I have also > changed the messages produced for

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-01-24 Thread Michael Paquier
On Mon, Jan 10, 2022 at 11:04:05AM +0530, Bharath Rupireddy wrote: > On Mon, Jan 10, 2022 at 10:58 AM Jaime Casanova > wrote: >> Now; I do think that the secondd patch, the one that just skips update >> of the state in control file, is the way to go. The other patch adds too >> much complexity

Re: PublicationActions - use bit flags.

2022-01-24 Thread Greg Nancarrow
On Tue, Jan 25, 2022 at 7:31 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > > Why can't GetRelationPublicationActions() have the PublicationActions as > a return value, instead of changing it to an output argument? That would be OK too, for now, for the current (small size,

Support escape sequence for cluster_name in postgres_fdw.application_name

2022-01-24 Thread Fujii Masao
Hi, Commit 6e0cb3dec1 allowed postgres_fdw.application_name to include escape sequences %a (application name), %d (database name), %u (user name) and %p (pid). In addition to them, I'd like to support the escape sequence (e.g., %C) for cluster name there. This escape sequence is helpful to

Foreign join search stops on the first try

2022-01-24 Thread Alexander Pyhalov
Hi. I was investigating why foreign join is not happening and found the following issue. When we search for foreign join path we stop on the first try. For example, in case A JOIN B JOIN C where all of them are foreign tables on the same server firstly we estimate A JOIN B, A JOIN C and B

Re: WIN32 pg_import_system_collations

2022-01-24 Thread Dmitry Koval
Hi Juan José, I a bit tested this feature and have small doubts about block: +/* + * Windows will use hyphens between language and territory, where POSIX + * uses an underscore. Simply make it POSIX looking. + */ + hyphen = strchr(localebuf, '-'); + if (hyphen) +*hyphen = '_'; After this

Re: Catalog version access

2022-01-24 Thread Michael Paquier
On Mon, Jan 24, 2022 at 08:40:08PM +, Bossart, Nathan wrote: > On 1/23/22, 7:31 PM, "Michael Paquier" wrote: >> On Mon, Aug 16, 2021 at 06:12:54PM +, Bossart, Nathan wrote: >>> I was looking at the --check switch for the postgres binary recently >>> [0], and this sounds like something

Re: Skipping logical replication transactions on subscriber side

2022-01-24 Thread Amit Kapila
On Mon, Jan 24, 2022 at 7:40 PM Peter Eisentraut wrote: > > On 22.01.22 10:41, Amit Kapila wrote: > >> Additionally, the description for pg_stat_subscription_workers should > >> describe what happens once the transaction represented by last_error_xid > >> has either been successfully processed

Design of pg_stat_subscription_workers vs pgstats

2022-01-24 Thread Andres Freund
Hi, I was looking the shared memory stats patch again. The rebase of which collided fairly heavily with the addition of pg_stat_subscription_workers. I'm concerned about the design of pg_stat_subscription_workers. The view was introduced in commit 8d74fc96db5fd547e077bf9bf4c3b67f821d71cd

Re: WIN32 pg_import_system_collations

2022-01-24 Thread Dmitry Koval
Hi Juan José, I a bit tested this feature and have small doubts about block: +/* + * Windows will use hyphens between language and territory, where POSIX + * uses an underscore. Simply make it POSIX looking. + */ + hyphen = strchr(localebuf, '-'); + if (hyphen) +*hyphen = '_'; After this

Re: [BUG]Update Toast data failure in logical replication

2022-01-24 Thread Andres Freund
Hi, On 2022-01-24 15:10:05 -0500, Robert Haas wrote: > I think we realized when we were working on the logical decoding stuff > that the key columns of the old tuple would have to be detoasted in > order for the mechanism to work, because I remember worrying about > whether it would potentially

Re: CREATEROLE and role ownership hierarchies

2022-01-24 Thread Robert Haas
On Mon, Jan 24, 2022 at 4:23 PM Stephen Frost wrote: > The idea behind this patch is to enable creation and dropping of roles, which > isn’t possible now without being effectively a superuser. > > Forcing owners to also implicitly have all rights of the roles they create is > orthogonal to that

Re: [BUG]Update Toast data failure in logical replication

2022-01-24 Thread Andres Freund
On 2022-01-24 16:31:08 -0500, Robert Haas wrote: > That seems consistent with what's been described on this thread so > far, but I still don't quite understand why the logic that reassembles > TOAST chunks doesn't solve it. There are no toast chunks to reassemble if the update didn't change the

Re: GUC flags

2022-01-24 Thread Justin Pryzby
On Wed, Jan 05, 2022 at 11:36:41PM -0600, Justin Pryzby wrote: > On Thu, Jan 06, 2022 at 02:19:08PM +0900, Michael Paquier wrote: > > > + initStringInfo(); > > > + appendStringInfoChar(, '{'); > > > + > > > + if (flags & GUC_NO_SHOW_ALL) > > > + appendStringInfo(, "NO_SHOW_ALL,"); > > > +

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-24 Thread Andres Freund
Hi, On 2022-01-23 19:49:57 -0800, Andres Freund wrote: > > To avoid too noisy breakages, we could have python.m4 emit INCLUDEPY and > > then > > search the bf logs in a day or three? > > Maybe something like the attached? Not particularly nice, but should give us > most of the relevant

Re: CREATEROLE and role ownership hierarchies

2022-01-24 Thread Mark Dilger
> On Jan 24, 2022, at 2:21 PM, Stephen Frost wrote: > > Being able to create and drop users is, in fact, effectively a superuser-only > task today. We could throw out the entire idea of role ownership, in fact, > as being entirely unnecessary when talking about that specific task. Wow,

Re: XTS cipher mode for cluster file encryption

2022-01-24 Thread Bruce Momjian
On Mon, Nov 29, 2021 at 08:37:31AM +0100, Antonin Houska wrote: > The changes to buffile.c are not trivial, but we haven't really changed the > API, as long as you mean BufFileCreateTemp(), BufFileWrite(), BufFileRead(). > > What our patch affects on the caller side is that

Re: CREATEROLE and role ownership hierarchies

2022-01-24 Thread Stephen Frost
Greetings, On Mon, Jan 24, 2022 at 15:33 Robert Haas wrote: > On Sat, Jan 22, 2022 at 4:20 PM Stephen Frost wrote: > > Whoah, really? No, I don't agree with this, it's throwing away the > > entire concept around inheritance of role rights and how you can have > > roles which you can get the

Re: Bug in ProcArrayApplyRecoveryInfo for snapshots crossing 4B, breaking replicas

2022-01-24 Thread Tomas Vondra
On 1/24/22 22:28, Bossart, Nathan wrote: On 1/22/22, 4:43 PM, "Tomas Vondra" wrote: There's a bug in ProcArrayApplyRecoveryInfo, introduced by 8431e296ea, which may cause failures when starting a replica, making it unusable. The commit message for 8431e296ea is not very clear about what

Re: CREATEROLE and role ownership hierarchies

2022-01-24 Thread Mark Dilger
> On Jan 24, 2022, at 2:21 PM, Stephen Frost wrote: > > Superuser is a problem specifically because it gives people access to do > absolutely anything, both for security and safety concerns. Disallowing a way > to curtail that same risk when it comes to role ownership invites exactly >

Re: Replace uses of deprecated Python module distutils.sysconfig

2022-01-24 Thread Tom Lane
Peter Eisentraut writes: > Also note that python-config is itself a Python script that uses > sysconfig and includes code like this: > elif opt in ('--includes', '--cflags'): > flags = ['-I' + sysconfig.get_path('include'), > '-I' +

Re: Collecting statistics about contents of JSONB columns

2022-01-24 Thread Tomas Vondra
On 1/23/22 01:24, Nikita Glukhov wrote: Hi! I am glad that you found my very old patch interesting and started to work on it. We failed to post it in 2016 mostly because we were not satisfied with JSONB storage. Also we decided to wait for completion of work on extended statistics as we

  1   2   >