Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Julien Rouhaud
On Wed, Feb 09, 2022 at 02:30:08PM -0500, Robert Haas wrote: > On Wed, Feb 9, 2022 at 1:34 PM Bruce Momjian wrote: > > Well, I think the worst case is that the checkpoint happens exactly > > between two checkpoints, so you are checkpointing twice as often, but if > > it happens just before or afte

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-09 Thread Nitin Jadhav
> > We need at least a trace of the number of buffers to sync (num_to_scan) > > before the checkpoint start, instead of just emitting the stats at the end. > > > > Bharat, it would be good to show the buffers synced counter and the total > > buffers to sync, checkpointer pid, substep it is runnin

Re: Replacing TAP test planning with done_testing()

2022-02-09 Thread Kyotaro Horiguchi
At Thu, 10 Feb 2022 09:58:27 +0900, Michael Paquier wrote in > On Wed, Feb 09, 2022 at 02:49:47PM -0500, Tom Lane wrote: > > =?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > >> Daniel Gustafsson writes: > >>> The attached patch removes all Test::More planning and instead ensures > >>> that

Re: Make mesage at end-of-recovery less scary.

2022-02-09 Thread Kyotaro Horiguchi
At Wed, 9 Feb 2022 17:31:02 +0530, Ashutosh Sharma wrote in > On Wed, Feb 9, 2022 at 1:14 PM Kyotaro Horiguchi > wrote: > > This means archive-recovery is requested but not started yet. That is, > > we've just finished crash recovery. The existing comment cited > > together is mentioning that.

Re: Move replication slot structures/enums/macros to a new header file for better usability by external tools/modules

2022-02-09 Thread Andres Freund
Hi, On 2022-02-07 10:52:08 +0530, Bharath Rupireddy wrote: > While working on pg_replslotdata tool [1], it was observed that some > of the replication slot structures/enums/macros such as > ReplicationSlotPersistentData, ReplicationSlotPersistency, > ReplicationSlotOnDisk, ReplicationSlotOnDiskXXX

Use return value of XLogRecGetBlockTag instead of explicit block ref checks and also use XLogRecHasBlockRef/Image macros instead of explicit checks

2022-02-09 Thread Bharath Rupireddy
Hi, I think we can use the return value of XLogRecGetBlockTag instead of an explicit check XLogRecHasBlockRef as the XLogRecGetBlockTag would anyways return false in such a case. Also, in some places the macros XLogRecHasBlockRef and XLogRecHasBlockImage aren't being used instead the checks are be

Re: Move replication slot structures/enums/macros to a new header file for better usability by external tools/modules

2022-02-09 Thread Bharath Rupireddy
On Wed, Feb 9, 2022 at 2:16 AM Peter Smith wrote: > > On Mon, Feb 7, 2022 at 4:22 PM Bharath Rupireddy > wrote: > > > > Hi, > > > > While working on pg_replslotdata tool [1], it was observed that some > > of the replication slot structures/enums/macros such as > > ReplicationSlotPersistentData, R

Re: [PATCH] Add min() and max() aggregate functions for xid8

2022-02-09 Thread Fujii Masao
On 2022/02/09 13:04, Kyotaro Horiguchi wrote: At Wed, 09 Feb 2022 12:04:51 +0900 (JST), Kyotaro Horiguchi wrote in At Wed, 9 Feb 2022 11:01:57 +0900, Fujii Masao wrote in Agreed. So barring any objection, I will commit that patch. Sorry for being late, but I don't like the function nam

Re: Database-level collation version tracking

2022-02-09 Thread Julien Rouhaud
On Wed, Feb 09, 2022 at 05:12:41PM +0100, Peter Eisentraut wrote: > On 08.02.22 13:55, Julien Rouhaud wrote: > > Apart from that I still think that we should check the collation version of > > the > > source database when creating a new database. It won't cost much but will > > give > > the DBA

Re: catalog access with reset GUCs during parallel worker startup

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 7:33 PM Andres Freund wrote: > > Can we arrange to absorb the leader's values before starting the > > worker's transaction, instead of inside it? > > I assume Robert did it this way for a reason? It'd not be surprising if there > were a bunch of extensions assuming its safe

Re: Logging in LockBufferForCleanup()

2022-02-09 Thread Michael Paquier
On Wed, Feb 09, 2022 at 06:22:05PM -0800, Andres Freund wrote: > Previously the code looked somewhat safe to use in critical section like > blocks (although whether it'd be good idea to use in one is a different > question), but not after. Even if not used in a critical section, adding new > failur

Re: Possible uninitialized use of the variables (src/backend/access/transam/twophase.c)

2022-02-09 Thread Michael Paquier
On Thu, Feb 10, 2022 at 10:18:15AM +0900, Kyotaro Horiguchi wrote: > Who repoerted that to you? Let's bet on Coverity. > StartPrepare and EndPrepare are virtually a single function that > accepts some additional operations in the middle. StartPrepare leaves > the "records" incomplete then EndPre

Re: [RFC] building postgres with meson - perl embedding

2022-02-09 Thread Tom Lane
Andres Freund writes: > On 2022-02-08 18:42:33 -0500, Tom Lane wrote: >> I'd be a little inclined to replace it with some rule about stripping '-bE:' >> switches out of the ldopts result. > Similar. That's a lot easier to understand than than -bE ending up stripped by > what we're doing. Should I

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-02-09 Thread Bharath Rupireddy
On Sun, Feb 6, 2022 at 9:15 PM Robert Haas wrote: > > On Mon, Jan 31, 2022 at 4:40 PM Greg Stark wrote: > > So I looked at this patch and I have the same basic question as Bruce. > > Do we really want to expose every binary tool associated with Postgres > > as an extension? Obviously this is temp

Logging in LockBufferForCleanup()

2022-02-09 Thread Andres Freund
Hi, I just noticed somewhat new code in LockBufferForCleanup(), added in commit 39b03690b529935a3c33024ee68f08e2d347cf4f Author: Fujii Masao Date: 2021-01-13 22:59:17 +0900 Log long wait time on recovery conflict when it's resolved. commit 64638ccba3a659d8b8a3a4bc5b47307685a64a8a Author

Re: Plug minor memleak in pg_dump

2022-02-09 Thread Michael Paquier
On Wed, Feb 09, 2022 at 02:48:35PM -0300, Ranier Vilela wrote: > IMO I think that still have troubles here. > > ReadStr can return NULL, so the fix can crash. - sscanf(tmp, "%u", &te->catalogId.tableoid); - free(tmp); + if (tmp) + { + sscanf(t

Re: [BUG]Update Toast data failure in logical replication

2022-02-09 Thread Amit Kapila
On Wed, Feb 9, 2022 at 11:08 AM tanghy.f...@fujitsu.com wrote: > > On Tue, Feb 8, 2022 3:18 AM Andres Freund wrote: > > > > On 2022-02-07 08:44:00 +0530, Amit Kapila wrote: > > > Right, and it is getting changed. We are just printing the first 200 > > > characters (by using SQL [1]) from the deco

Re: warn if GUC set to an invalid shared library

2022-02-09 Thread Justin Pryzby
On Fri, Jan 28, 2022 at 09:42:17AM -0500, Robert Haas wrote: > -1 from me on using "guc" in any user-facing error message. And even > guc -> setting isn't a big improvement. If we're going to structure > the reporting this way there, we should try to use a meaningful phrase > there, probably beginn

Re: [RFC] building postgres with meson - perl embedding

2022-02-09 Thread Andres Freund
Hi, On 2022-02-08 18:42:33 -0500, Tom Lane wrote: > I'd be a little inclined to replace it with some rule about stripping '-bE:' > switches out of the ldopts result. Similar. That's a lot easier to understand than than -bE ending up stripped by what we're doing. Should I do so, or do you want to?

Re: make MaxBackends available in _PG_init

2022-02-09 Thread Michael Paquier
On Wed, Feb 09, 2022 at 09:53:38AM -0800, Nathan Bossart wrote: > On Wed, Feb 09, 2022 at 12:31:16PM -0500, Robert Haas wrote: >> On Wed, Feb 9, 2022 at 12:18 AM Michael Paquier wrote: >>> Okay, I'd rather apply the same rule everywhere for consistency, then, >>> like in the attached. That's mini

Re: GetRelationPublicationActions. - Remove unreachable code

2022-02-09 Thread Peter Smith
On Thu, Feb 10, 2022 at 11:34 AM Tom Lane wrote: > > Peter Smith writes: > > There appears to be some unreachable code in the relcache function > > GetRelationPublicationActions. > > When the 'relation->rd_pubactions' is not NULL then the function > > unconditionally returns early (near the top).

Re: Possible uninitialized use of the variables (src/backend/access/transam/twophase.c)

2022-02-09 Thread Kyotaro Horiguchi
At Wed, 9 Feb 2022 08:15:45 -0300, Ranier Vilela wrote in > Hi, > > Commit > https://github.com/postgres/postgres/commit/1eb6d6527aae264b3e0b9c95aa70bb7a594ad1cf, > modified > data struct TwoPhaseFileHeader and added two new fields: > > XLogRecPtr origin_lsn; /* lsn of this record at origin nod

Re: Justin Pryzby

2022-02-09 Thread Michael Paquier
Hi Alexander, On Wed, Feb 09, 2022 at 03:18:12PM +0300, Alexander Pyhalov wrote: > I've looked at patches, introducing CREATE INDEX CONCURRENTLY for > partitioned tables - > https://www.postgresql.org/message-id/flat/20210226182019.GU20769%40telsasoft.com#da169a0a518bf8121604437d9ab053b3 > . > Th

Re: Replacing TAP test planning with done_testing()

2022-02-09 Thread Michael Paquier
On Wed, Feb 09, 2022 at 02:49:47PM -0500, Tom Lane wrote: > =?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: >> Daniel Gustafsson writes: >>> The attached patch removes all Test::More planning and instead ensures that >>> all >>> tests conclude with a done_testing() call. While there, I also r

Re: row filtering for logical replication

2022-02-09 Thread Peter Smith
On Tue, Jan 25, 2022 at 2:18 PM houzj.f...@fujitsu.com wrote: > ... > > 4. src/backend/utils/cache/relcache.c - RelationBuildPublicationDesc > > > > - if (relation->rd_pubactions) > > + if (relation->rd_pubdesc) > > { > > - pfree(relation->rd_pubactions); > > - relation->rd_pubactions = NULL; >

Re: GetRelationPublicationActions. - Remove unreachable code

2022-02-09 Thread Tom Lane
Peter Smith writes: > There appears to be some unreachable code in the relcache function > GetRelationPublicationActions. > When the 'relation->rd_pubactions' is not NULL then the function > unconditionally returns early (near the top). > Therefore, the following code (near the bottom) seems to ha

Re: catalog access with reset GUCs during parallel worker startup

2022-02-09 Thread Andres Freund
Hi, On 2022-02-09 18:56:41 -0500, Tom Lane wrote: > Andres Freund writes: > > Do we really need to reset GUCs to their boot value? Particularly for > > PGC_SIGHUP variables I don't think we can do that if we want to do > > RestoreGUCState() in a transaction. It'd obviously involve a bit more cod

GetRelationPublicationActions. - Remove unreachable code

2022-02-09 Thread Peter Smith
Hi hackers. There appears to be some unreachable code in the relcache function GetRelationPublicationActions. When the 'relation->rd_pubactions' is not NULL then the function unconditionally returns early (near the top). Therefore, the following code (near the bottom) seems to have no purpose be

Re: catalog access with reset GUCs during parallel worker startup

2022-02-09 Thread Tom Lane
Andres Freund writes: > Do we really need to reset GUCs to their boot value? Particularly for > PGC_SIGHUP variables I don't think we can do that if we want to do > RestoreGUCState() in a transaction. It'd obviously involve a bit more code, > but it seems entirely possible to just get rid of the

Re: New developer papercut - Makefile references INSTALL

2022-02-09 Thread Tom Lane
Andres Freund writes: > On 2022-02-09 22:32:59 +0100, Magnus Hagander wrote: >> post-commit hooks don't run on the git server, they run locally on >> your machine. There is a "post receive" hook that runs on the git >> server, but we definitely don't want that one to fabricate new commits >> I thi

Re: [PATCH] Improve function toast_delete_external (src/backend/access/table/toast_helper.c)

2022-02-09 Thread Ranier Vilela
Em qua., 9 de fev. de 2022 às 20:10, Dong Wook Lee escreveu: > Yes, now I understand it. > Thank you for letting me know about that. > You are welcome. Best regards, Ranier Vilela

Re: New developer papercut - Makefile references INSTALL

2022-02-09 Thread Andres Freund
Hi, On 2022-02-09 22:32:59 +0100, Magnus Hagander wrote: > On Fri, Jan 21, 2022 at 11:53 PM Andres Freund wrote: > > On 2022-01-21 17:25:08 -0500, Tom Lane wrote: > > > Perhaps this could be finessed by making updating of INSTALL > > > the responsibility of some post-commit hook on the git server

Re: [PATCH] nodeindexscan with reorder memory leak

2022-02-09 Thread Tom Lane
Alexander Korotkov writes: > I've rechecked that the now patched code branch is covered by > regression tests. I think the memory leak issue is independent of the > computational errors we've observed. > So, I'm going to push and backpatch this if no objections. +1. We should work on the roundo

Re: decoupling table and index vacuum

2022-02-09 Thread Peter Geoghegan
On Wed, Feb 9, 2022 at 1:41 PM Robert Haas wrote: > I'm not sure that we can. I mean, there's still only going to be ~3 > autovacuum workers, and there could be arbitrarily many tables. Even > if the vacuum load is within the bounds of what the system can > sustain, individual tables can't be assu

catalog access with reset GUCs during parallel worker startup

2022-02-09 Thread Andres Freund
Hi, I noticed that during ParallelWorkerMain()->RestoreGUCState() we perform catalog accesses with GUCs being set to wrong values, specifically values that aren't what's in postgresql.conf, and that haven't been set in the leader. The reason for this is that 1) RestoreGUCState() is called in a t

Re: [PATCH] nodeindexscan with reorder memory leak

2022-02-09 Thread Alexander Korotkov
On Thu, Feb 10, 2022 at 1:19 AM Aliaksandr Kalenik wrote: > On Mon, Feb 7, 2022 at 11:20 PM Alexander Korotkov > wrote: > > Regarding the memory leak, could you add a corresponding regression > > test to the patch (probably similar to Tom's query upthread)? > > Yes, added similar to Tom's query

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes

2022-02-09 Thread Andres Freund
Hi, On 2022-02-09 16:42:30 -0600, Justin Pryzby wrote: > On Wed, Feb 09, 2022 at 02:00:04PM -0800, Andres Freund wrote: > > On linux we can do so by a) checking if readlink(/proc/self/fd/$fd) points > > to > > a filename ending in " (deleted)", b) doing fstat(fd) and checking if > > st_nlink > >

Re: [PATCH] Improve function toast_delete_external (src/backend/access/table/toast_helper.c)

2022-02-09 Thread Dong Wook Lee
Yes, now I understand it. Thank you for letting me know about that. Thanks. 2022년 2월 10일 (목) 00:39, Ranier Vilela 님이 작성: > > Em qua., 9 de fev. de 2022 às 11:34, Dong Wook Lee > escreveu: > >> Hi, >> >> I've applied your patch. I think it's reasonable. >> but IMHO It looks more complicated to

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-02-09 Thread Justin Pryzby
On Wed, Feb 09, 2022 at 02:00:04PM -0800, Andres Freund wrote: > On linux we can do so by a) checking if readlink(/proc/self/fd/$fd) points to > a filename ending in " (deleted)", b) doing fstat(fd) and checking if st_nlink > == 0. You could also stat() the file in proc/self/fd/N and compare st_in

Re: [PATCH] nodeindexscan with reorder memory leak

2022-02-09 Thread Aliaksandr Kalenik
On Mon, Feb 7, 2022 at 11:20 PM Alexander Korotkov wrote: > Regarding the memory leak, could you add a corresponding regression > test to the patch (probably similar to Tom's query upthread)? Yes, added similar to Tom's query but with polygons instead of circles. 0002-nodeindexscan_with_reorder

wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-02-09 Thread Andres Freund
Hi, I was working on rebasing the AIO branch. Tests started to fail after, but it turns out that the problem exists independent of AIO. The problem starts with commit aa01051418f10afbdfa781b8dc109615ca785ff9 Author: Robert Haas Date: 2022-01-24 14:23:15 -0500 pg_upgrade: Preserve databas

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-09 Thread Joshua Brindle
On Wed, Feb 9, 2022 at 3:58 PM Robert Haas wrote: > > On Wed, Feb 9, 2022 at 1:13 PM Nathan Bossart > wrote: > > I do wonder if users find the differences between predefined roles and role > > attributes confusing. INHERIT doesn't govern role attributes, but it will > > govern predefined roles

Re: decoupling table and index vacuum

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 2:27 PM Peter Geoghegan wrote: > We should probably dispense with the idea that we'll be making these > decisions about what to do with an index like this (bloated in a way > that bottom-up index deletion just won't help with) in an environment > that is similar to how the c

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-09 Thread Joe Conway
On 2/9/22 13:13, Nathan Bossart wrote: On Tue, Feb 08, 2022 at 10:54:50PM -0500, Robert Haas wrote: On Tue, Feb 8, 2022 at 7:38 PM Joe Conway wrote: If we were to start all over again with this feature my vote would be to do things differently than we have done. I would not have called them pr

Re: New developer papercut - Makefile references INSTALL

2022-02-09 Thread Magnus Hagander
On Fri, Jan 21, 2022 at 11:53 PM Andres Freund wrote: > > Hi, > > On 2022-01-21 17:25:08 -0500, Tom Lane wrote: > > Perhaps this could be finessed by making updating of INSTALL > > the responsibility of some post-commit hook on the git server. > > Not sure that we want to go there, though. In any

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 1:13 PM Nathan Bossart wrote: > I do wonder if users find the differences between predefined roles and role > attributes confusing. INHERIT doesn't govern role attributes, but it will > govern predefined roles when this patch is applied. Maybe the role > attribute system s

Re: is the base backup protocol used by out-of-core tools?

2022-02-09 Thread Magnus Hagander
On Wed, Feb 9, 2022 at 2:02 PM Robert Haas wrote: > > On Wed, Feb 9, 2022 at 3:14 AM Kyotaro Horiguchi > wrote: > > Thanks for pining. AFAICS, as you see, pg_rman doesn't talk > > basebackup protocol (nor even pg_basebackup command) as it supports > > inremental backup. So there's no issue abou

Re: Replacing TAP test planning with done_testing()

2022-02-09 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > Daniel Gustafsson writes: >> The attached patch removes all Test::More planning and instead ensures that >> all >> tests conclude with a done_testing() call. While there, I also removed a few >> exit(0) calls from individual tests making them

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 1:34 PM Bruce Momjian wrote: > Well, I think the worst case is that the checkpoint happens exactly > between two checkpoints, so you are checkpointing twice as often, but if > it happens just before or after a checkpoint, I assume the effect would > be minimal. I agree for

Re: decoupling table and index vacuum

2022-02-09 Thread Peter Geoghegan
On Wed, Feb 9, 2022 at 6:13 AM Robert Haas wrote: > Just to be clear, when I say that the dead index tuples don't matter > here, I mean from the point of view of the index. From the point of > view of the table, the presence of dead index tuples (or even the > potential presence of dead tuples) po

Re: Avoiding smgrimmedsync() during nbtree index builds

2022-02-09 Thread Melanie Plageman
Hi, v5 attached and all email feedback addressed below On Thu, Jan 13, 2022 at 12:18 PM Robert Haas wrote: > > On Wed, Sep 29, 2021 at 2:36 PM Melanie Plageman > wrote: > > unbuffered_write() and unbuffered_extend() might be able to be used even > > if unbuffered_prep() and unbuffered_finish() a

Re: How to get started with contribution

2022-02-09 Thread Daniel Gustafsson
> On 9 Feb 2022, at 18:13, 603_Annada Dash > wrote: > Respected Sir/Ma'am, > I am Annada Dash, a computer science undergraduate, currently about to finish > the first semester of University (MKSSS Cummins College of Engineering for > Women, Pune). I am new to open source contributions but I am

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Bruce Momjian
On Wed, Feb 9, 2022 at 11:00:06AM -0500, Robert Haas wrote: > Try running pgbench with the --progress option and enough concurrent > jobs to keep a moderately large system busy and watching what happens > to the tps each time a checkpoint occurs. It's extremely dramatic, or > at least it was the l

Re: Identify missing publications from publisher while create/alter subscription.

2022-02-09 Thread Euler Taveira
On Wed, Feb 9, 2022, at 12:06 PM, Ashutosh Sharma wrote: > Just wondering if we should also be detecting the incorrect conninfo > set with ALTER SUBSCRIPTION command as well. See below: > > -- try creating a subscription with incorrect conninfo. the command fails. > postgres=# create subscription

Re: Unnecessary call to resetPQExpBuffer in getIndexes

2022-02-09 Thread Nathan Bossart
On Wed, Feb 09, 2022 at 10:50:07AM +0800, Julien Rouhaud wrote: > I just noticed that e2c52beecd (adding PeterE in Cc) added a > resetPQExpBuffer() > which seems unnecessary since the variable is untouched since the initial > createPQExpBuffer(). > > Simple patch attached. LGTM -- Nathan Bossa

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-09 Thread Nathan Bossart
On Tue, Feb 08, 2022 at 10:54:50PM -0500, Robert Haas wrote: > On Tue, Feb 8, 2022 at 7:38 PM Joe Conway wrote: >> If we were to start all over again with this feature my vote would be to >> do things differently than we have done. I would not have called them >> predefined roles, and I would have

Re: Typo in archive modules docs

2022-02-09 Thread Nathan Bossart
On Wed, Feb 09, 2022 at 03:04:34PM +0100, Daniel Gustafsson wrote: > Spotted a typo when reading the new archive modules documentation, will apply > the attached shortly to fix. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

How to get started with contribution

2022-02-09 Thread 603_Annada Dash
Respected Sir/Ma'am, I am Annada Dash, a computer science undergraduate, currently about to finish the first semester of University (MKSSS Cummins College of Engineering for Women, Pune). I am new to open source contributions but I am well aware of python, C and R. I would love to contribute to you

Re: make MaxBackends available in _PG_init

2022-02-09 Thread Nathan Bossart
On Wed, Feb 09, 2022 at 12:31:16PM -0500, Robert Haas wrote: > On Wed, Feb 9, 2022 at 12:18 AM Michael Paquier wrote: >> Okay, I'd rather apply the same rule everywhere for consistency, then, >> like in the attached. That's minimal, still. > > That's fine with me. In the interest of full disclos

Re: Plug minor memleak in pg_dump

2022-02-09 Thread Ranier Vilela
>No, but I was distracted by other things leaving this on the TODO list. It's >been pushed now. Hi, IMO I think that still have troubles here. ReadStr can return NULL, so the fix can crash. regards, Ranier Vilela v1_fix_possible_null_dereference_pg_backup_archiver.patch Description: Binary

Re: make MaxBackends available in _PG_init

2022-02-09 Thread Nathan Bossart
On Tue, Feb 08, 2022 at 04:12:26PM -0500, Robert Haas wrote: > After some investigation I've determined that it's no longer Friday > afternoon. This matches my analysis. > I also spent time investigating whether the patch had > problems that would make me uncomfortable with the idea of committing

Re: make MaxBackends available in _PG_init

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 12:18 AM Michael Paquier wrote: > Okay, I'd rather apply the same rule everywhere for consistency, then, > like in the attached. That's minimal, still. That's fine with me. In the interest of full disclosure, I did kind of notice this when reviewing the patch, though perha

Re: [PATCH] Add reloption for views to enable RLS

2022-02-09 Thread walther
Laurenz Albe: So even though the view owner "duff" has no permissions on the schema "viewtest", we can still select from the table. Permissions on the schema containing the table are not checked, only permissions on the table itself. [...] If not, I don't know if it is the business of this patc

Re: Database-level collation version tracking

2022-02-09 Thread Alvaro Herrera
On 2022-Feb-08, Julien Rouhaud wrote: > On Tue, Feb 08, 2022 at 12:14:02PM +0100, Peter Eisentraut wrote: > > I don't think you can run ALTER DATABASE from the regression test scripts, > > since the database name is not fixed. You'd have to paste the command > > together using psql tricks or som

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Robert Haas
On Tue, Feb 8, 2022 at 11:47 PM Dilip Kumar wrote: > Now, one bigger question is can we proceed with this patch without > fixing [2], IMHO, if we are deciding to keep the old method also > intact then one option could be that for now only change CREATE > DATABASE to support both old and new way of

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Andrew Dunstan
On 2/9/22 10:58, Dilip Kumar wrote: > On Wed, Feb 9, 2022 at 9:25 PM Andrew Dunstan wrote: >> >> On 6/16/21 03:52, Dilip Kumar wrote: >>> On Tue, Jun 15, 2021 at 7:01 PM Andrew Dunstan wrote: Rather than use size, I'd be inclined to say use this if the source database is marked as a t

Re: Database-level collation version tracking

2022-02-09 Thread Peter Eisentraut
On 08.02.22 13:55, Julien Rouhaud wrote: Apart from that I still think that we should check the collation version of the source database when creating a new database. It won't cost much but will give the DBA a chance to recreate the indexes before risking invalid index usage. A question on thi

Re: [PATCH] Add reloption for views to enable RLS

2022-02-09 Thread Laurenz Albe
On Fri, 2022-02-04 at 22:28 +0100, walt...@technowledgy.de wrote: > This is a feature I have long been looking for. I tested the patch (v5) > and found two cases that I feel need to be either fixed or documented > explicitly. Thanks for testing and weighing in! > Case 1 - Schema privileges: >

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 10:59 AM Dilip Kumar wrote: > On Wed, Feb 9, 2022 at 9:25 PM Andrew Dunstan wrote: > > On 6/16/21 03:52, Dilip Kumar wrote: > > > On Tue, Jun 15, 2021 at 7:01 PM Andrew Dunstan > > > wrote: > > >> Rather than use size, I'd be inclined to say use this if the source > > >>

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 9:19 AM Bruce Momjian wrote: > Honestly, I never understood why the checkpoint during CREATE DATABASE > was as problem --- we checkpoint by default every five minutes anyway, > so why is an additional two a problem --- it just means the next > checkpoint will do less work.

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Dilip Kumar
On Wed, Feb 9, 2022 at 9:25 PM Andrew Dunstan wrote: > > > On 6/16/21 03:52, Dilip Kumar wrote: > > On Tue, Jun 15, 2021 at 7:01 PM Andrew Dunstan wrote: > >> Rather than use size, I'd be inclined to say use this if the source > >> database is marked as a template, and use the copydir approach fo

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Dilip Kumar
On Wed, Feb 9, 2022 at 7:49 PM Bruce Momjian wrote: > > Honestly, I never understood why the checkpoint during CREATE DATABASE > was as problem --- we checkpoint by default every five minutes anyway, > so why is an additional two a problem --- it just means the next > checkpoint will do less work.

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Andrew Dunstan
On 6/16/21 03:52, Dilip Kumar wrote: > On Tue, Jun 15, 2021 at 7:01 PM Andrew Dunstan wrote: >> Rather than use size, I'd be inclined to say use this if the source >> database is marked as a template, and use the copydir approach for >> anything that isn't. > Yeah, that is possible, on the other

Re: [PATCH] Improve function toast_delete_external (src/backend/access/table/toast_helper.c)

2022-02-09 Thread Alvaro Herrera
Our style is that we group all declarations in a block to appear at the top. We don't mix declarations and statements. -- Álvaro Herrera 39°49'30"S 73°17'W — https://www.EnterpriseDB.com/

Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR

2022-02-09 Thread Tomas Vondra
On 2/9/22 06:41, Noah Misch wrote: The explanation was more boring than that. v13 and earlier have an additional InvalidateSystemCaches() call site, which I neglected to update. Here's the fix I intend to push. I tried this patch on 10 and 13, and it seems to fix the issue. So +1. regards

Re: Identify missing publications from publisher while create/alter subscription.

2022-02-09 Thread Ashutosh Sharma
Just wondering if we should also be detecting the incorrect conninfo set with ALTER SUBSCRIPTION command as well. See below: -- try creating a subscription with incorrect conninfo. the command fails. postgres=# create subscription sub1 connection 'host=localhost port=5490 dbname=postgres' publicat

Re: [PATCH] Improve function toast_delete_external (src/backend/access/table/toast_helper.c)

2022-02-09 Thread Dong Wook Lee
Oops, I realized that I made a mistake in my patch and now I have corrected it. 2022년 2월 9일 (수) 오후 11:33, Dong Wook Lee 님이 작성: > > Hi, > > I've applied your patch. I think it's reasonable. > but IMHO It looks more complicated to read because of many conditions > in if statement. > so what about j

Re: Replacing TAP test planning with done_testing()

2022-02-09 Thread Dagfinn Ilmari Mannsåker
Daniel Gustafsson writes: > Whether or not to explicitly plan the number of TAP tests per suite has been > discussed a number of times on this list, often as a side-note in an unrelated > thread which adds/modifies a test. The concensus has so far weighed towards > not doing manual bookkeeping o

Re: [PATCH] Improve function toast_delete_external (src/backend/access/table/toast_helper.c)

2022-02-09 Thread Dong Wook Lee
Hi, I've applied your patch. I think it's reasonable. but IMHO It looks more complicated to read because of many conditions in if statement. so what about just moving up if-statement? Thanks. Dong Wook Lee 2022년 2월 9일 (수) 오후 7:56, Ranier Vilela 님이 작성: > > Hi, > > I think this change can improve

Re: Add tag/category to the commitfest app

2022-02-09 Thread Julien Rouhaud
On Wed, Feb 09, 2022 at 03:14:30PM +0100, Daniel Gustafsson wrote: > > On 8 Feb 2022, at 03:10, Julien Rouhaud wrote: > > > I was thinking that we could add a tag system, probably with some limited > > predefined tags, to address that problem. > > > > Do you think it's worth adding, or do you ha

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-09 Thread Bruce Momjian
On Tue, Feb 8, 2022 at 12:09:08PM -0500, Robert Haas wrote: > Sadly, it doesn't appear to me that anyone has done any performance > testing of this patch, along the lines suggested above or otherwise, > and I think it's a crucial question for the patch. My reading of this > thread is that nobody r

Re: Replacing TAP test planning with done_testing()

2022-02-09 Thread Julien Rouhaud
Hi, On Wed, Feb 09, 2022 at 03:01:36PM +0100, Daniel Gustafsson wrote: > Whether or not to explicitly plan the number of TAP tests per suite has been > discussed a number of times on this list, often as a side-note in an unrelated > thread which adds/modifies a test. The concensus has so far weig

Re: Add tag/category to the commitfest app

2022-02-09 Thread Daniel Gustafsson
> On 8 Feb 2022, at 03:10, Julien Rouhaud wrote: > I was thinking that we could add a tag system, probably with some limited > predefined tags, to address that problem. > > Do you think it's worth adding, or do you have a better idea to help newer > contributors? We already have the categories

Re: decoupling table and index vacuum

2022-02-09 Thread Robert Haas
On Wed, Feb 9, 2022 at 1:18 AM Dilip Kumar wrote: > I agree with the point that we should be focusing more on index size > growth compared to dead tuples. But I don't think that we can > completely ignore the number of dead tuples. Although we have the > bottom-up index deletion but whether the

Typo in archive modules docs

2022-02-09 Thread Daniel Gustafsson
Spotted a typo when reading the new archive modules documentation, will apply the attached shortly to fix. -- Daniel Gustafsson https://vmware.com/ typo-recyling.diff Description: Binary data

Replacing TAP test planning with done_testing()

2022-02-09 Thread Daniel Gustafsson
Whether or not to explicitly plan the number of TAP tests per suite has been discussed a number of times on this list, often as a side-note in an unrelated thread which adds/modifies a test. The concensus has so far weighed towards not doing manual bookkeeping of test plans but to let Test::More d

Re: [PATCH] Improve function toast_delete_external (src/backend/access/table/toast_helper.c)

2022-02-09 Thread Julien Rouhaud
Hi, On Wed, Feb 09, 2022 at 07:56:35AM -0300, Ranier Vilela wrote: > > I think this change can improve this particular function by avoiding > touching value if not needed. > Test if not isnull is cheaper than test TupleDescAttr is -1. It looks sensible.

Re: is the base backup protocol used by out-of-core tools?

2022-02-09 Thread Bernd Helmle
Am Mittwoch, dem 09.02.2022 um 08:01 -0500 schrieb Robert Haas: > On Wed, Feb 9, 2022 at 3:14 AM Kyotaro Horiguchi > wrote: > > Thanks for pining.  AFAICS, as you see, pg_rman doesn't talk > > basebackup protocol (nor even pg_basebackup command) as it supports > > inremental backup.  So there's no

Re: refactoring basebackup.c

2022-02-09 Thread Abhijit Menon-Sen
At 2022-02-02 10:55:53 -0500, robertmh...@gmail.com wrote: > > On Tue, Jan 18, 2022 at 1:55 PM Robert Haas wrote: > > 0001 adds "server" and "blackhole" as backup targets. It now has some > > tests. This might be more or less ready to ship, unless somebody else > > sees a problem, or I find one. >

Re: Database-level collation version tracking

2022-02-09 Thread Julien Rouhaud
On Wed, Feb 09, 2022 at 12:48:35PM +0100, Peter Eisentraut wrote: > On 08.02.22 13:55, Julien Rouhaud wrote: > > I'm just saying that without such a lock you can easily trigger the "cache > > lookup" error, and that's something that's supposed to happen with normal > > usage I think. So it should

Re: Refactoring SSL tests

2022-02-09 Thread Andrew Dunstan
On 2/9/22 08:11, Daniel Gustafsson wrote: >> On 8 Feb 2022, at 16:46, Andrew Dunstan wrote: >> There a capitalization typo in SSL/Backend/OpenSSL.pm - looks like >> that's my fault: >> >> + my $backend = SSL::backend::OpenSSL->new(); > Fixed. > >> Also, I think we should document that SSL::Serv

Re: I would like to participate for postgresql projects

2022-02-09 Thread Ashesh Vashi
On Wed, Feb 9, 2022 at 6:52 PM Daniel Gustafsson wrote: > > On 8 Feb 2022, at 06:12, luminate 22 wrote: > > > > Hi! my name is Mario Mercado. I'm from California western pacific time. > I would like to participate in your projects. I'm a software developer from > California. > > Welcome aboard,

Re: SQL/JSON: JSON_TABLE

2022-02-09 Thread Himanshu Upadhyaya
On Wed, Feb 2, 2022 at 12:44 AM Andrew Dunstan wrote: > > > rebased with some review comments attended to. I am in process of reviewing these patches, initially, have started with 0002-JSON_TABLE-v55.patch. Tested many different scenarios with various JSON messages and these all are working as ex

Re: I would like to participate for postgresql projects

2022-02-09 Thread Daniel Gustafsson
> On 8 Feb 2022, at 06:12, luminate 22 wrote: > > Hi! my name is Mario Mercado. I'm from California western pacific time. I > would like to participate in your projects. I'm a software developer from > California. Welcome aboard, everyone is welcome. I would recommend starting with cloning th

Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR

2022-02-09 Thread Tom Lane
Michael Paquier writes: > On Tue, Feb 08, 2022 at 05:43:34PM -0800, Andres Freund wrote: >> It's stamped, not tagged, so we could send out new tarballs. Or we could skip >> a release number. IIRC we had to do something along those lines before. > It does not matter now, but the release is stamped

Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR

2022-02-09 Thread Noah Misch
On Tue, Feb 08, 2022 at 06:04:03PM -0800, Noah Misch wrote: > On Tue, Feb 08, 2022 at 04:43:47PM -0800, Andres Freund wrote: > > On 2022-02-08 22:13:01 +0100, Tomas Vondra wrote: > > > On 10/24/21 03:40, Noah Misch wrote: > > > > Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY

Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR

2022-02-09 Thread Michael Paquier
On Tue, Feb 08, 2022 at 05:43:34PM -0800, Andres Freund wrote: > It's stamped, not tagged, so we could send out new tarballs. Or we could skip > a release number. IIRC we had to do something along those lines before. It does not matter now, but the release is stamped and tagged. > What do you mea

Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR

2022-02-09 Thread Noah Misch
On Tue, Feb 08, 2022 at 04:43:47PM -0800, Andres Freund wrote: > On 2022-02-08 22:13:01 +0100, Tomas Vondra wrote: > > On 10/24/21 03:40, Noah Misch wrote: > > > Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURRENTLY. > > > > > > CIC and REINDEX CONCURRENTLY assume backends see thei

Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR

2022-02-09 Thread Andres Freund
Hi, On 2022-02-09 02:25:09 +0100, Tomas Vondra wrote: > AFAICS this only affects builds with CLOBBER_CACHE_ALWAYS, and anyone > running such build in production clearly likes painful things anyway. Yea, realistically nobody does that. > But really, for the infinite loop to happen, building a re

Re: pgsql: Avoid race in RelationBuildDesc() affecting CREATE INDEX CONCURR

2022-02-09 Thread Andres Freund
Hi, On 2022-02-09 10:23:06 +0900, Michael Paquier wrote: > On Tue, Feb 08, 2022 at 04:43:47PM -0800, Andres Freund wrote: > > Ugh. Do we need to do something about WRT the next set of minor > > releases? > > The set of minor releases of this week has already been stamped, so > that's too late :/

  1   2   >