Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-07-20 Thread Alvaro Herrera
v26 here. I spent some time fighting the readdir() stuff for Windows (so that get_dirent_type returns LNK for junction points) but couldn't make it to work and was unable to figure out why. So I ended up doing what do_pg_backup_start is already doing: an #ifdef to call pgwin32_is_junction

Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?

2022-07-20 Thread Bharath Rupireddy
Hi, After the commit [1], is it correct to say errmsg("invalid data in file \"%s\"", BACKUP_LABEL_FILE))); in do_pg_backup_stop() when we hold the contents in backend global memory, not actually reading from backup_label file? However, it is correct to say that in read_backup_label. IMO, we can

Re: Allowing REINDEX to have an optional name

2022-07-20 Thread Justin Pryzby
On Tue, Jul 19, 2022 at 01:13:34PM +0900, Michael Paquier wrote: > > It looks like you named the table "toast_relfilenodes", but then also store > > to it data for non-toast tables. > > How about naming that index_relfilenodes? One difference with what I > posted previously and 5fb5b6 is the

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-07-20 Thread Marco Slot
On Mon, Jul 18, 2022 at 8:29 AM Amit Kapila wrote: > IIUC, this proposal is to optimize cases where users can't have a > unique/primary key for a relation on the subscriber and those > relations receive lots of updates or deletes? I think this patch optimizes for all non-trivial cases of

Re: Memory leak fix in psql

2022-07-20 Thread Alvaro Herrera
More on the same tune. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ "This is what I like so much about PostgreSQL. Most of the surprises are of the "oh wow! That's cool" Not the "oh shit!" kind. :)" Scott Marlowe,

Re: Windows default locale vs initdb

2022-07-20 Thread Juan José Santamaría Flecha
On Tue, Jul 19, 2022 at 12:59 AM Thomas Munro wrote: > Now that museum-grade Windows has been defenestrated, we are free to > call GetUserDefaultLocaleName(). Here's a patch. > This LGTM. > > I think we should also convert to POSIX format when making the > collname in your

Re: Windows default locale vs initdb

2022-07-20 Thread Thomas Munro
On Wed, Jul 20, 2022 at 10:27 PM Juan José Santamaría Flecha wrote: > On Tue, Jul 19, 2022 at 4:47 AM Thomas Munro wrote: >> As for whether "accordingly" still applies, by the logic of of >> win32_langinfo()... Windows still considers WIN1252 to be the default >> ANSI code page for "en-US",

Re: Windows default locale vs initdb

2022-07-20 Thread Juan José Santamaría Flecha
On Tue, Jul 19, 2022 at 4:47 AM Thomas Munro wrote: > As for whether "accordingly" still applies, by the logic of of > win32_langinfo()... Windows still considers WIN1252 to be the default > ANSI code page for "en-US", though it'd work with UTF-8 too. I'm not > sure what to make of that. The

Re: Collect ObjectAddress for ATTACH DETACH PARTITION to use in event trigger

2022-07-20 Thread Amit Kapila
On Fri, Jul 15, 2022 at 11:39 AM houzj.f...@fujitsu.com wrote: > > On Friday, July 15, 2022 11:41 AM Michael Paquier wrote: > > Hi, > > > > On Fri, Jul 15, 2022 at 03:21:30AM +, kuroda.hay...@fujitsu.com wrote: > > > Sounds good. I grepped ATExecXXX() functions called in ATExecCmd(), > > >

Re: Remove fls(), use pg_bitutils.h facilities instead?

2022-07-20 Thread Tom Lane
Thomas Munro writes: > On Wed, Jul 20, 2022 at 4:52 PM Tom Lane wrote: >> I think we could probably just drop fls() entirely. It doesn't look >> to me like any of the existing callers expect a zero argument, so they >> could be converted to use pg_leftmost_one_pos32() pretty trivially. > That

Re: GetStandbyFlushRecPtr() : OUT param is not optional like elsewhere.

2022-07-20 Thread Aleksander Alekseev
Hi Amul, > - *tli = replayTLI; > + if (tli) > + *tli = replayTLI; I would guess the difference here is that GetStandbyFlushRecPtr is static. It is used 3 times in walsender.c and in all cases the argument can't be NULL. So I'm not certain what we will gain from the proposed check. --

Re: Use "WAL segment" instead of "log segment" consistently in user-facing messages

2022-07-20 Thread Bharath Rupireddy
On Wed, Jul 20, 2022 at 12:55 PM Kyotaro Horiguchi wrote: > > At Wed, 20 Jul 2022 10:02:22 +0530, Bharath Rupireddy > wrote in > > Done. PSA v6 patch set. > > Thanks! > > -Specifies the minimum size of past log file segments kept in the > +Specifies the minimum size of past WAL

Re: Make name optional in CREATE STATISTICS

2022-07-20 Thread Matthias van de Meent
On Wed, 13 Jul 2022 at 08:07, Simon Riggs wrote: > > On Thu, 7 Jul 2022 at 11:58, Matthias van de Meent > wrote: > > A more correct version would be > > > > + *CREATE STATISTICS [ [IF NOT EXISTS] stats_name ] > > [(stat types)] > > There you go Thanks! I think this is ready for

GetStandbyFlushRecPtr() : OUT param is not optional like elsewhere.

2022-07-20 Thread Amul Sul
Hi, If you look at GetFlushRecPtr() function the OUT parameter for TimeLineID is optional and this is not only one, see GetWalRcvFlushRecPtr(), GetXLogReplayRecPtr(), etc. I think we have missed that for GetStandbyFlushRecPtr(), to be inlined, we should change this as follow: ---

Re: Making pg_rewind faster

2022-07-20 Thread Justin Kwan
Hi Michael, Thank you for your feedback, I've incorporated your suggestions by scanning the logs produced from pg_rewind when asserting that certain WAL segment files were skipped from being copied over to the target server. I've also updated the pg_rewind patch file to target the Postgres

Re: PATCH: Add Table Access Method option to pgbench

2022-07-20 Thread Alexander Korotkov
Hi! On Tue, Jul 19, 2022 at 4:47 AM Michael Paquier wrote: > On Mon, Jul 18, 2022 at 01:53:21PM +0300, Alexander Korotkov wrote: > > Looks good to me as well. I'm going to push this if no objections. > > FWIW, I find the extra mention of PGOPTIONS with the specific point of > table AMs added

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-20 Thread Amit Kapila
On Wed, Jul 20, 2022 at 1:28 PM Masahiko Sawada wrote: > > On Wed, Jul 20, 2022 at 2:19 PM Amit Kapila wrote: > > > > On Wed, Jul 20, 2022 at 9:01 AM Masahiko Sawada > > wrote: > > > > > Another idea would be to have functions, say > > > SnapBuildCommitTxnWithXInfo() and

Re: Pluggable toaster

2022-07-20 Thread Nikita Malakhov
Hi hackers! We really need your feedback on the last patchset update! On a previous question about TOAST API overhead - please check script in attach, we tested INSERT, UPDATE and SELECT operations, and ran it on vanilla master and on patched master (vanilla with untouched TOAST implementation

Re: Is select_outer_pathkeys_for_merge() too strict now we have Incremental Sorts?

2022-07-20 Thread Richard Guo
On Wed, Jul 20, 2022 at 11:03 AM David Rowley wrote: > I think we can relax this now that we have incremental sort. I think > a better way to limit this is to allow a prefix of the query_pathkeys > providing that covers *all* of the join pathkeys. That way, for the > above query, it leaves it

Re: Backup command and functions can cause assertion failure and segmentation fault

2022-07-20 Thread Michael Paquier
On Wed, Jul 20, 2022 at 02:00:00PM +0900, Fujii Masao wrote: > I reported two trouble cases; they are the cases where BASE_BACKUP > is canceled and terminated, respectively. But you added the test > only for one of them. Is this intentional? Nope. The one I have implemented was the fanciest case

RE: Memory leak fix in psql

2022-07-20 Thread tanghy.f...@fujitsu.com
On Wednesday, July 20, 2022 12:52 PM, Michael Paquier wrote: > What about the argument of upthread where we could use a goto in > functions where there are multiple pattern validation checks? Per se > v4 attached. Thanks for your kindly remind and modification. I checked v4 patch, it looks

Re: Fast COPY FROM based on batch insert

2022-07-20 Thread Etsuro Fujita
On Tue, Jul 19, 2022 at 6:35 PM Andrey Lepikhov wrote: > On 18/7/2022 13:22, Etsuro Fujita wrote: > > I rewrote the decision logic to something much simpler and much less > > invasive, which reduces the patch size significantly. Attached is an > > updated patch. What do you think about that? >

Re: Transparent column encryption

2022-07-20 Thread Masahiko Sawada
On Tue, Jul 19, 2022 at 10:52 PM Peter Eisentraut wrote: > > On 12.07.22 20:29, Peter Eisentraut wrote: > > Updated patch, to resolve some merge conflicts. > > Rebased patch, no new functionality Thank you for working on this and updating the patch! I've mainly looked at the documentation and

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-07-20 Thread Kyotaro Horiguchi
At Tue, 19 Jul 2022 09:53:39 -0700, Nathan Bossart wrote in > On Tue, Jul 19, 2022 at 12:55:14AM -0500, Steve Chavez wrote: > > Taking your options into consideration, for me the correct behaviour should > > be: > > > > - The ALTER ROLE placeholder should always be stored with a PGC_USERSET >

Re: replacing role-level NOINHERIT with a grant-level option

2022-07-20 Thread tushar
On 7/19/22 12:56 AM, Robert Haas wrote: Another good catch. Here is v5 with a fix for that problem. Thanks, the issue is fixed now. -- regards,tushar EnterpriseDB https://www.enterprisedb.com/ The Enterprise PostgreSQL Company

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-20 Thread Masahiko Sawada
On Wed, Jul 20, 2022 at 4:16 PM Kyotaro Horiguchi wrote: > > At Wed, 20 Jul 2022 10:58:16 +0900, Masahiko Sawada > wrote in > > On Wed, Jul 20, 2022 at 9:58 AM Kyotaro Horiguchi > > wrote: > > > Mmm. the patch changed that behavior. AllocateSnapshotBuilder always > > > allocate the array with

Re: Use "WAL segment" instead of "log segment" consistently in user-facing messages

2022-07-20 Thread Kyotaro Horiguchi
At Wed, 20 Jul 2022 10:02:22 +0530, Bharath Rupireddy wrote in > Done. PSA v6 patch set. Thanks! -Specifies the minimum size of past log file segments kept in the +Specifies the minimum size of past WAL files kept in the -log file by reducing the value of this

Re: Memory leak fix in psql

2022-07-20 Thread Junwang Zhao
Thanks for your explanation, this time I know how it works, thanks ;) On Wed, Jul 20, 2022 at 3:04 PM tanghy.f...@fujitsu.com wrote: > > > On Wed, Jul 20, 2022 at 12:51:24PM +0800, Junwang Zhao wrote: > > > Though the patch looks good, I myself think the patch should be edited > > > and

Re: i.e. and e.g.

2022-07-20 Thread Kyotaro Horiguchi
At Thu, 14 Jul 2022 15:38:37 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 14 Jul 2022 09:40:25 +0700, John Naylor > wrote in > > On Wed, Jul 13, 2022 at 4:13 PM Kyotaro Horiguchi > > wrote: > > > > > > At Wed, 13 Jul 2022 18:09:43 +0900 (JST), Kyotaro Horiguchi < > >

Re: Handle infinite recursion in logical replication setup

2022-07-20 Thread vignesh C
On Wed, Jul 20, 2022 at 10:38 AM Peter Smith wrote: > > On Tue, Jul 19, 2022 at 11:34 PM Amit Kapila wrote: > > > > On Mon, Jul 18, 2022 at 9:46 PM vignesh C wrote: > > > > > > I have updated the patch to handle the origin value case > > > insensitively. The attached patch has the changes for

RE: Memory leak fix in psql

2022-07-20 Thread tanghy.f...@fujitsu.com
> On Wed, Jul 20, 2022 at 12:51:24PM +0800, Junwang Zhao wrote: > > Though the patch looks good, I myself think the patch should be edited > > and submitted by Tang > > It's easy to attach a fixed patch based on the comments of the thread, > > but coins should be > > given to Tang since he is the

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-20 Thread Kyotaro Horiguchi
At Wed, 20 Jul 2022 10:58:16 +0900, Masahiko Sawada wrote in > On Wed, Jul 20, 2022 at 9:58 AM Kyotaro Horiguchi > wrote: > > Mmm. the patch changed that behavior. AllocateSnapshotBuilder always > > allocate the array with a fixed size. SnapBuildAddCommittedTxn still > > assumes

Re: Memory leak fix in psql

2022-07-20 Thread Tom Lane
Japin Li writes: > Thanks for updating the patch. It looks good. However, it cannot be > applied on 14 stable. The attached patches are for 10-14. While I think this is good cleanup, I'm doubtful that any of these leaks are probable enough to be worth back-patching into stable branches. The

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Alvaro Herrera
On 2022-Jul-20, Tom Lane wrote: > I'll try to do some research later today to identify anything else > we need to mark in plpgsql. I recall doing some work specifically > creating functions for pldebugger's use, but I'll need to dig. I suppose you're probably thinking of commit 53ef6c40f1e7;

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-07-20 Thread Amit Langote
On Wed, Jul 20, 2022 at 12:37 AM Andres Freund wrote: > On 2022-07-19 20:40:11 +0900, Amit Langote wrote: > > About that, I was wondering if the blocks in llvm_compile_expr() need > > to be hand-coded to match what's added in ExecInterpExpr() or if I've > > missed some tool that can be used

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Tom Lane
Robert Haas writes: > On Wed, Jul 20, 2022 at 9:39 AM Tom Lane wrote: >> ISTM that a comment pointing out that the functions marked PGDLLEXPORT >> are meant to be externally accessible should be sufficient. > The name PGDLLEXPORT is actually slightly misleading, now, because > there's no longer

[PATCH] Renumber confusing value for GUC_UNIT_BYTE

2022-07-20 Thread Justin Pryzby
The GUC units are currently defined like: #define GUC_UNIT_KB 0x1000 /* value is in kilobytes */ #define GUC_UNIT_BLOCKS 0x2000 /* value is in blocks */ #define GUC_UNIT_XBLOCKS0x3000 /* value is in xlog blocks */ #define GUC_UNIT_MB

Re: shared-memory based stats collector - v70

2022-07-20 Thread Melanie Plageman
On Wed, Jul 20, 2022 at 11:35 AM Greg Stark wrote: > On the one hand the rest of Postgres seems to be designed on the > assumption that the number of tables and database objects is limited > only by disk space. The catalogs are stored in relational storage > which is read through the buffer

Re: shared-memory based stats collector - v70

2022-07-20 Thread Tom Lane
Melanie Plageman writes: > On Wed, Jul 20, 2022 at 11:35 AM Greg Stark wrote: >> It seems like if we really think the total number of database objects >> is reasonably limited to scales that fit in RAM there would be a much >> simpler database design that would just store the catalog tables in

Re: GetStandbyFlushRecPtr() : OUT param is not optional like elsewhere.

2022-07-20 Thread Alvaro Herrera
Hello On 2022-Jul-20, Amul Sul wrote: > If you look at GetFlushRecPtr() function the OUT parameter for > TimeLineID is optional and this is not only one, see > GetWalRcvFlushRecPtr(), GetXLogReplayRecPtr(), etc. > > I think we have missed that for GetStandbyFlushRecPtr(), to be > inlined, we

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Tom Lane
Alvaro Herrera writes: > I suppose you're probably thinking of commit 53ef6c40f1e7; that didn't > expose functions directly, but through plpgsql_plugin_ptr. Maybe that > one does need to be made PGDLLEXPORT, since currently it isn't. Hm. Not sure if the rules are the same for global variables

Re: shared-memory based stats collector - v70

2022-07-20 Thread Andres Freund
Hi, On 2022-07-20 12:08:35 -0400, Tom Lane wrote: > AFAIR, the previous stats collector implementation had no such provision > either: it'd just keep adding hashtable entries as it received info about > new objects. Yep. > The only thing that's changed is that now those entries are in shared >

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-07-20 Thread Önder Kalacı
Hi, > > - Add a new class of invalidation callbacks: Today, if we do ALTER TABLE > or CREATE INDEX on a table, the CacheRegisterRelcacheCallback helps us to > re-create the cache entries. In this case, as far as I can see, we need a > callback that is called when table "ANALYZE"d, because that

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Andres Freund
Hi, On July 20, 2022 4:20:04 PM GMT+02:00, Tom Lane wrote: >Alvaro Herrera writes: >> I suppose you're probably thinking of commit 53ef6c40f1e7; that didn't >> expose functions directly, but through plpgsql_plugin_ptr. Maybe that >> one does need to be made PGDLLEXPORT, since currently it

Re: Remove useless arguments in ReadCheckpointRecord().

2022-07-20 Thread Bharath Rupireddy
On Wed, Jul 20, 2022 at 8:21 PM Fujii Masao wrote: > > Hi, > > I'd like to propose to remove "whichChkpt" and "report" arguments in > ReadCheckpointRecord(). "report" is obviously useless because it's always > true, i.e., there are two callers of the function and they always specify > true as

Re: shared-memory based stats collector - v70

2022-07-20 Thread Greg Stark
So I'm finally wrapping my head around this new code. There is something I'm surprised by that perhaps I'm misreading or perhaps I shouldn't be surprised by, not sure. Is it true that the shared memory allocation contains the hash table entry and body of every object in every database? I guess I

Re: shared-memory based stats collector - v70

2022-07-20 Thread Andres Freund
Hi, On 2022-07-20 11:35:13 -0400, Greg Stark wrote: > Is it true that the shared memory allocation contains the hash table > entry and body of every object in every database? Yes. However, note that that was already the case with the old stats collector - it also kept everything in memory. In

Re: Logical insert/update/delete WAL records for custom table AMs

2022-07-20 Thread Jeff Davis
On Mon, 2022-03-21 at 17:43 -0700, Andres Freund wrote: > Currently this doesn't apply: > http://cfbot.cputube.org/patch_37_3394.log Withdrawn for now. With custom WAL resource managers this is less important to me. I still think it has value, and I'm willing to work on it if more use cases

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-07-20 Thread Alvaro Herrera
On 2022-Jul-20, Alvaro Herrera wrote: > On 2022-Jul-20, Alvaro Herrera wrote: > > > I also change the use of allow_invalid_pages to > > allow_in_place_tablespaces. We could add a > > separate GUC for this, but it seems overengineering. > > Oh, but allow_in_place_tablespaces doesn't exist in

Re: Remove fls(), use pg_bitutils.h facilities instead?

2022-07-20 Thread Tom Lane
Thomas Munro writes: > That double eval macro wasn't nice. This time with a static inline function. Seems like passing a size_t to pg_leftmost_one_pos32 isn't great. It was just as wrong before (if the caller-supplied argument is indeed a size_t), but no time like the present to fix it. We

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Robert Haas
On Wed, Jul 20, 2022 at 9:39 AM Tom Lane wrote: > ISTM that a comment pointing out that the functions marked PGDLLEXPORT > are meant to be externally accessible should be sufficient. The name PGDLLEXPORT is actually slightly misleading, now, because there's no longer anything about it that is

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Andres Freund
Hi, On July 20, 2022 3:54:03 PM GMT+02:00, Robert Haas wrote: >On Wed, Jul 20, 2022 at 9:39 AM Tom Lane wrote: >> ISTM that a comment pointing out that the functions marked PGDLLEXPORT >> are meant to be externally accessible should be sufficient. > >The name PGDLLEXPORT is actually slightly

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-07-20 Thread Alvaro Herrera
On 2022-Jul-20, Alvaro Herrera wrote: > I also change the use of allow_invalid_pages to > allow_in_place_tablespaces. We could add a > separate GUC for this, but it seems overengineering. Oh, but allow_in_place_tablespaces doesn't exist in versions 14 and older, so this strategy doesn't really

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-07-20 Thread Tom Lane
Kyotaro Horiguchi writes: > At Tue, 19 Jul 2022 09:53:39 -0700, Nathan Bossart > wrote in >> Hm. I would expect ALTER ROLE to store the PGC_SUSET context when executed >> by a superuser or a role with privileges via pg_parameter_acl. Storing all >> placeholder GUC settings as PGC_USERSET

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Tom Lane
Alvaro Herrera writes: > No immediate plans for splitting plpgsql.h, so if anyone wants to take a > stab at that, be my guest. ISTM that a comment pointing out that the functions marked PGDLLEXPORT are meant to be externally accessible should be sufficient. I'll try to do some research later

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2022-07-20 Thread Aleksander Alekseev
Hi Pavel, > Rebased. PFA v13. > Your reviews are very much welcome! I noticed that this patch is in "Needs Review" state and it has been stuck for some time now, so I decided to take a look. ``` +SELECT bt_index_parent_check('bttest_a_idx', true, true, true); +SELECT

Re: Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Tom Lane
Andres Freund writes: > On July 20, 2022 3:54:03 PM GMT+02:00, Robert Haas > wrote: >> The name PGDLLEXPORT is actually slightly misleading, now, because >> there's no longer anything about it that is specific to DLLs. > How so? Right now it's solely used for making symbols in DLLs as

Remove useless arguments in ReadCheckpointRecord().

2022-07-20 Thread Fujii Masao
Hi, I'd like to propose to remove "whichChkpt" and "report" arguments in ReadCheckpointRecord(). "report" is obviously useless because it's always true, i.e., there are two callers of the function and they always specify true as "report". "whichChkpt" indicates where the specified checkpoint

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Tom Lane
Alvaro Herrera writes: > On 2022-Jul-20, Tom Lane wrote: >> I'll try to do some research later today to identify anything else >> we need to mark in plpgsql. I recall doing some work specifically >> creating functions for pldebugger's use, but I'll need to dig. > I suppose you're probably

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Dave Page
On Wed, 20 Jul 2022 at 16:12, Tom Lane wrote: > Alvaro Herrera writes: > > On 2022-Jul-20, Tom Lane wrote: > >> I'll try to do some research later today to identify anything else > >> we need to mark in plpgsql. I recall doing some work specifically > >> creating functions for pldebugger's

make -C libpq check fails obscurely if tap tests are disabled

2022-07-20 Thread Justin Pryzby
make -C ./src/interfaces/libpq check PATH=... && @echo "TAP tests not enabled. Try configuring with --enable-tap-tests" /bin/sh: 1: @echo: not found make is telling the shell to run "@echo" , rather than running "echo" silently. Since: commit 6b04abdfc5e0653542ac5d586e639185a8c61a39 Author:

Re: shared-memory based stats collector - v70

2022-07-20 Thread Greg Stark
On Wed, 20 Jul 2022 at 12:08, Tom Lane wrote: > > AFAIR, the previous stats collector implementation had no such provision > either: it'd just keep adding hashtable entries as it received info about > new objects. The only thing that's changed is that now those entries are > in shared memory

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-07-20 Thread Andres Freund
Hi, On 2022-07-14 18:44:48 -0400, Melanie Plageman wrote: > Subject: [PATCH v26 1/4] Add BackendType for standalone backends > Subject: [PATCH v26 2/4] Remove unneeded call to pgstat_report_wal() LGTM. > Subject: [PATCH v26 3/4] Track IO operation statistics > @@ -978,8 +979,17 @@

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-07-20 Thread Melanie Plageman
On Wed, Jul 20, 2022 at 12:50 PM Andres Freund wrote: > > On 2022-07-14 18:44:48 -0400, Melanie Plageman wrote: > > > @@ -1427,8 +1445,10 @@ pgstat_read_statsfile(void) > > FILE *fpin; > > int32 format_id; > > boolfound; > > +

Re: pg_auth_members.grantor is bunk

2022-07-20 Thread Robert Haas
On Fri, Jun 24, 2022 at 4:46 PM Robert Haas wrote: > On Fri, Jun 24, 2022 at 4:30 PM David G. Johnston > wrote: > >> Upthread, I proposed that "drop role baz" should fail here > > > > I concur with this. > > > > I think that the grantor owns the grant, and that REASSIGNED OWNED should > > be

Re: shared-memory based stats collector - v70

2022-07-20 Thread Andres Freund
Hi, On July 20, 2022 8:41:53 PM GMT+02:00, Greg Stark wrote: >On Wed, 20 Jul 2022 at 12:08, Tom Lane wrote: >> >> AFAIR, the previous stats collector implementation had no such provision >> either: it'd just keep adding hashtable entries as it received info about >> new objects. The only

Re: make -C libpq check fails obscurely if tap tests are disabled

2022-07-20 Thread Andrew Dunstan
On 2022-07-20 We 13:23, Justin Pryzby wrote: > make -C ./src/interfaces/libpq check > PATH=... && @echo "TAP tests not enabled. Try configuring with > --enable-tap-tests" > /bin/sh: 1: @echo: not found > > make is telling the shell to run "@echo" , rather than running "echo" > silently. > >

Re: Is select_outer_pathkeys_for_merge() too strict now we have Incremental Sorts?

2022-07-20 Thread David Rowley
On Wed, 20 Jul 2022 at 21:19, Richard Guo wrote: > So the idea is if the ECs used by the mergeclauses are prefix of the > query_pathkeys, we use this prefix as pathkeys for the mergejoin. Why > not relax this further that if the ECs in the mergeclauses and the > query_pathkeys have common prefix,

Re: Memory leak fix in psql

2022-07-20 Thread Michael Paquier
On Wed, Jul 20, 2022 at 10:05:47AM +0200, Alvaro Herrera wrote: > More on the same tune. Thanks. I have noticed that as well. I'll include all that in the set. -- Michael signature.asc Description: PGP signature

Re: Memory leak fix in psql

2022-07-20 Thread Japin Li
On Wed, 20 Jul 2022 at 21:54, Tom Lane wrote: > Japin Li writes: >> Thanks for updating the patch. It looks good. However, it cannot be >> applied on 14 stable. The attached patches are for 10-14. > > While I think this is good cleanup, I'm doubtful that any of these > leaks are probable

Re: Refactor to make use of a common function for GetSubscriptionRelations and GetSubscriptionNotReadyRelations.

2022-07-20 Thread Michael Paquier
On Wed, Jul 20, 2022 at 02:32:44PM +0530, Amit Kapila wrote: > On Thu, Jul 14, 2022 at 4:34 AM Peter Smith wrote: >> By using a bitmask I think there is an implication that the flags can >> be combined... >> >> Perhaps it is not a problem today, but later you may want more flags. e.g. >> #define

Re: Allowing REINDEX to have an optional name

2022-07-20 Thread Michael Paquier
On Wed, Jul 20, 2022 at 07:27:07AM -0500, Justin Pryzby wrote: > Looks fine Thanks for checking, applied. -- Michael signature.asc Description: PGP signature

Re: Improve description of XLOG_RUNNING_XACTS

2022-07-20 Thread Fujii Masao
On 2022/07/21 10:13, Masahiko Sawada wrote: Hi, I realized that standby_desc_running_xacts() in standbydesc.c doesn't describe subtransaction XIDs. I've attached the patch to improve the description. +1 The patch looks good to me. Regards, -- Fujii Masao Advanced Computing Technology

Fwd: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-20 Thread Gurjeet Singh
Moving the report from security to -hackers on Noah's advice. Since the function(s) involved in the crash are not present in any of the released versions, it is not considered a security issue. I can confirm that this is reproducible on the latest commit on master, 3c0bcdbc66. Below is the

Improve description of XLOG_RUNNING_XACTS

2022-07-20 Thread Masahiko Sawada
Hi, I realized that standby_desc_running_xacts() in standbydesc.c doesn't describe subtransaction XIDs. I've attached the patch to improve the description. Here is an example by pg_wlaldump: * HEAD rmgr: Standby len (rec/tot): 58/58, tx: 0, lsn: 0/01D0C608, prev 0/01D0C5D8,

Re: System catalog documentation chapter

2022-07-20 Thread Bruce Momjian
On Wed, Jul 20, 2022 at 04:32:46PM -0400, Bruce Momjian wrote: > On Wed, Jul 20, 2022 at 04:23:21PM -0400, Isaac Morland wrote: > > Will there be a comprehensive list somewhere? Even if it just lists the > > views, > > gives maybe a one-sentence description, and links to the relevant chapter, I >

Re: Memory leak fix in psql

2022-07-20 Thread Michael Paquier
On Thu, Jul 21, 2022 at 09:10:43AM +0800, Japin Li wrote: > Yeah, we should take care of the backpatch risk. However, I think > it makes sense to backpatch. We are talking about 256 bytes being leaked in each loop when a validation pattern or when a query fails, so I don't see a strong argument

Re: Is select_outer_pathkeys_for_merge() too strict now we have Incremental Sorts?

2022-07-20 Thread Richard Guo
On Thu, Jul 21, 2022 at 3:47 AM David Rowley wrote: > On Wed, 20 Jul 2022 at 21:19, Richard Guo wrote: > > So the idea is if the ECs used by the mergeclauses are prefix of the > > query_pathkeys, we use this prefix as pathkeys for the mergejoin. Why > > not relax this further that if the ECs in

Re: Remove useless arguments in ReadCheckpointRecord().

2022-07-20 Thread Fujii Masao
On 2022/07/21 0:29, Bharath Rupireddy wrote: How about we transform the following messages into something like below? (errmsg("could not locate a valid checkpoint record"))); after ReadCheckpointRecord() for control file cases to "could not locate valid checkpoint record in control file"

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-20 Thread Masahiko Sawada
On Wed, Jul 20, 2022 at 2:19 PM Amit Kapila wrote: > > On Wed, Jul 20, 2022 at 9:01 AM Masahiko Sawada wrote: > > > > On Wed, Jul 20, 2022 at 12:11 PM Amit Kapila > > wrote: > > > > > > On Tue, Jul 19, 2022 at 7:28 PM Masahiko Sawada > > > wrote: > > > > > > Why do you think we can't remove

Re: Memory leak fix in psql

2022-07-20 Thread Japin Li
On Wed, 20 Jul 2022 at 14:21, tanghy.f...@fujitsu.com wrote: > On Wednesday, July 20, 2022 12:52 PM, Michael Paquier > wrote: >> What about the argument of upthread where we could use a goto in >> functions where there are multiple pattern validation checks? Per se >> v4 attached. > > Thanks

Re: pgsql: Default to hidden visibility for extension libraries where possi

2022-07-20 Thread Alvaro Herrera
Hello On 2022-Jul-19, Andres Freund wrote: > On 2022-07-19 16:28:07 +0200, Alvaro Herrera wrote: > > Anyway, the minimal patch that makes plpgsql_check tests pass is > > attached. > > Do you want to commit that or should I? Done. No immediate plans for splitting plpgsql.h, so if anyone wants

Re: Refactor to make use of a common function for GetSubscriptionRelations and GetSubscriptionNotReadyRelations.

2022-07-20 Thread Amit Kapila
On Thu, Jul 14, 2022 at 4:34 AM Peter Smith wrote: > > On Wed, Jul 13, 2022 at 7:55 PM vignesh C wrote: > > > > On Wed, Jul 13, 2022 at 1:13 PM Michael Paquier wrote: > > > > > > On Wed, Jul 13, 2022 at 12:22:06PM +0530, vignesh C wrote: > > > > Most of the code is common between

Re: [PATCH] Log details for client certificate failures

2022-07-20 Thread Jacob Champion
On Tue, Jul 19, 2022 at 3:38 PM Andres Freund wrote: > Or alternatively, perhaps we can just make pg_clean_ascii() return NULL > if allocation failed and then guc_strdup() the result in guc.c? The guc_strdup() approach really reduces the amount of code, so that's what I did in v3. I'm not

Re: [PATCH] Log details for client certificate failures

2022-07-20 Thread Tom Lane
Jacob Champion writes: > The guc_strdup() approach really reduces the amount of code, so that's > what I did in v3. I'm not following why we need to return NULL on > failure, though -- both palloc() and guc_malloc() ERROR on failure, so > is it okay to keep those semantics the same? guc_malloc's

Re: [PATCH] Log details for client certificate failures

2022-07-20 Thread Tom Lane
Jacob Champion writes: > I'm currently hardcoding an elevel of ERROR on the new guc_strdup()s, > because that seems to be a common case for the check hooks. Really? That's almost certainly NOT okay. As an example, if you have a problem with a new value loaded from postgresql.conf during SIGHUP

Re: System catalog documentation chapter

2022-07-20 Thread Bruce Momjian
On Tue, Jul 19, 2022 at 01:41:44PM -0400, Bruce Momjian wrote: > I am going to look at moving system views that make sense into the > chapters where their contents are mentioned. I don't think having a > central list of views is really helping us because we expect the views > to be used in ways

Re: System catalog documentation chapter

2022-07-20 Thread Isaac Morland
On Wed, 20 Jul 2022 at 16:08, Bruce Momjian wrote: > On Tue, Jul 19, 2022 at 01:41:44PM -0400, Bruce Momjian wrote: > > I am going to look at moving system views that make sense into the > > chapters where their contents are mentioned. I don't think having a > > central list of views is really

Re: System catalog documentation chapter

2022-07-20 Thread Bruce Momjian
On Wed, Jul 20, 2022 at 04:23:21PM -0400, Isaac Morland wrote: > On Wed, 20 Jul 2022 at 16:08, Bruce Momjian wrote: > > On Tue, Jul 19, 2022 at 01:41:44PM -0400, Bruce Momjian wrote: > > I am going to look at moving system views that make sense into the > > chapters where their

Re: [PATCH] Log details for client certificate failures

2022-07-20 Thread Jacob Champion
On Wed, Jul 20, 2022 at 3:15 PM Tom Lane wrote: > guc_malloc's behavior varies depending on elevel. It's *not* > equivalent to palloc. Right, sorry -- a better way for me to ask the question: I'm currently hardcoding an elevel of ERROR on the new guc_strdup()s, because that seems to be a

Re: i.e. and e.g.

2022-07-20 Thread John Naylor
On Wed, Jul 20, 2022 at 3:40 PM Kyotaro Horiguchi wrote: > By the way, I forgot about back-branches. Don't we need to fix the > same in back-branches? The intent of the messages is pretty clear to me, so I don't really see a need to change back branches. It does make sense for v15, though, and I

Re: i.e. and e.g.

2022-07-20 Thread Kyotaro Horiguchi
At Thu, 21 Jul 2022 10:20:30 +0700, John Naylor wrote in > On Wed, Jul 20, 2022 at 3:40 PM Kyotaro Horiguchi > wrote: > > By the way, I forgot about back-branches. Don't we need to fix the > > same in back-branches? > > The intent of the messages is pretty clear to me, so I don't really see a

Re: Refactor to make use of a common function for GetSubscriptionRelations and GetSubscriptionNotReadyRelations.

2022-07-20 Thread Amit Kapila
On Thu, Jul 21, 2022 at 7:10 AM Michael Paquier wrote: > > On Wed, Jul 20, 2022 at 02:32:44PM +0530, Amit Kapila wrote: > > On Thu, Jul 14, 2022 at 4:34 AM Peter Smith wrote: > >> By using a bitmask I think there is an implication that the flags can > >> be combined... > >> > >> Perhaps it is

Re: making relfilenodes 56 bits

2022-07-20 Thread Thomas Munro
On Wed, Jul 20, 2022 at 11:27 PM Dilip Kumar wrote: > [v10 patch set] Hi Dilip, I'm experimenting with these patches and will hopefully have more to say soon, but I just wanted to point out that this builds with warnings and failed on 3/4 of the CI OSes on cfbot's last run. Maybe there is the

Re: Add LZ4 compression in pg_dump

2022-07-20 Thread Michael Paquier
On Tue, Jul 05, 2022 at 01:22:47PM +, gkokola...@pm.me wrote: > I have updated for "some" of the comments. This is not an unwillingness to > incorporate those specific comments. Simply this patchset had started to > divert > heavily already based on comments from Mr. Paquier who had already

Re: Remove useless arguments in ReadCheckpointRecord().

2022-07-20 Thread Kyotaro Horiguchi
I agree to removing the two parameters. And agree to let ReadCheckpointRecord not conscious of the location source. At Thu, 21 Jul 2022 11:45:23 +0900, Fujii Masao wrote in > Agreed. Attached is the updated version of the patch. > Thanks for the review! - (errmsg("could not locate

Re: Custom tuplesorts for extensions

2022-07-20 Thread John Naylor
On Tue, Jul 12, 2022 at 3:23 PM Alexander Korotkov wrote: > There are some places, which potentially could cause a slowdown. I'm > going to make some experiments with that. I haven't looked at the patches, so I don't know of a specific place to look for a slowdown, but I thought it might help

Re: GetStandbyFlushRecPtr() : OUT param is not optional like elsewhere.

2022-07-20 Thread Amul Sul
Thanks Aleksander and Álvaro for your inputs. I understand this change is not making any improvement to the current code. I was a bit concerned regarding the design and consistency of the function that exists for the server in recovery and for the server that is not in recovery. I was trying to

Re: Use "WAL segment" instead of "log segment" consistently in user-facing messages

2022-07-20 Thread Kyotaro Horiguchi
At Wed, 20 Jul 2022 17:25:33 +0530, Bharath Rupireddy wrote in > On Wed, Jul 20, 2022 at 12:55 PM Kyotaro Horiguchi > wrote: > PSA v7 patch set. Thanks. Looks perfect, but (sorry..) in the final checking, I found "log archive" in the doc. If you agree to it please merge the attached (or

Re: Refactor to make use of a common function for GetSubscriptionRelations and GetSubscriptionNotReadyRelations.

2022-07-20 Thread Michael Paquier
On Thu, Jul 21, 2022 at 09:54:05AM +0530, Amit Kapila wrote: > Yeah, it is not very clear to me either. I think this won't be > difficult to change one or another way depending on future needs. At > this stage, it appeared to me that bitmask is a better way to > represent this information but if

  1   2   >