Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-17 Thread Alexander Korotkov
On Thu, Jan 18, 2024 at 4:16 AM torikoshia wrote: > On 2024-01-18 10:10, jian he wrote: > > On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada > > wrote: > >> On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote: > >> > Kyotaro-san's suggestion isn't bad, though I might shorten it to > >> > error_action

Re: Make all Perl warnings fatal

2024-01-17 Thread Peter Eisentraut
On 16.01.24 12:08, Bharath Rupireddy wrote: On Fri, Jan 12, 2024 at 9:21 PM Bharath Rupireddy wrote: On Fri, Jan 12, 2024 at 9:03 PM Peter Eisentraut wrote: I would put this code my $core = $ret & 128 ? " (core dumped)" : ""; die "psql exited with signal " . ($ret &

Re: [17] CREATE SUBSCRIPTION ... SERVER

2024-01-17 Thread Jeff Davis
On Tue, 2024-01-16 at 09:23 +0530, Bharath Rupireddy wrote: > 1. > May be a more descriptive note is > worth here instead of just saying "Load the library providing us > libpq calls."? OK, will be in the next patch set. > 2. Why not typedef keyword before the ConnectionOption structure?

Re: Catalog domain not-null constraints

2024-01-17 Thread Peter Eisentraut
On 17.01.24 13:15, vignesh C wrote: One of the test has failed in CFBot at [1] with: diff -U3 /tmp/cirrus-ci-build/src/test/regress/expected/domain.out /tmp/cirrus-ci-build/src/test/regress/results/domain.out --- /tmp/cirrus-ci-build/src/test/regress/expected/domain.out 2024-01-14

Re: Reduce useless changes before reassembly during logical replication

2024-01-17 Thread Bharath Rupireddy
On Wed, Jan 17, 2024 at 11:45 AM li jie wrote: > > Hi hackers, > > During logical replication, if there is a large write transaction, some > spill files will be written to disk, depending on the setting of > logical_decoding_work_mem. > > This behavior can effectively avoid OOM, but if the

Re: A performance issue with Memoize

2024-01-17 Thread Arne Roland
Hi Richard, I can tell this a real world problem. I have seen this multiple times in production. The fix seems surprisingly simple. I hope my questions here aren't completely off. I still struggle to think about the implications. I wonder, if there is any stuff we are breaking by bluntly

Re: heavily contended lwlocks with long wait queues scale badly

2024-01-17 Thread Michael Paquier
On Tue, Jan 16, 2024 at 03:11:48PM +0900, Michael Paquier wrote: > I'd like to apply that, just let me know if you have any comments > and/or objections. And done on 12~15. While on it, I have also looked at source code references on github and debian that involve lwWaiting, and all of them rely

Re: subscription disable_on_error not working after ALTER SUBSCRIPTION set bad conninfo

2024-01-17 Thread Peter Smith
On Thu, Jan 18, 2024 at 12:55 PM Masahiko Sawada wrote: > ... > > Although we can improve it to handle this case too, I'm not sure it's > a bug. The doc says[1]: > > Specifies whether the subscription should be automatically disabled if > any errors are detected by subscription workers during

Re: Strange Bitmapset manipulation in DiscreteKnapsack()

2024-01-17 Thread Andy Fan
Hi, David Rowley writes: > On Thu, 18 Jan 2024 at 14:58, Andy Fan wrote: >> I want to know if "user just want to zero out the flags in bitmapset >> but keeping the memory allocation" is a valid requirement. Commit >> 00b41463c makes it is hard IIUC. > > Looking at your patch, I see: > > +/*

Re: Adding facility for injection points (or probe points?) for more advanced tests

2024-01-17 Thread Ashutosh Bapat
Hi Michael, There is some overlap between Dtrace functionality and this functionality. But I see differences too. E.g. injection points offer deeper integration whereas dtrace provides more information to the probe like callstack and argument values etc. We need to assess whether these

Re: Fix search_path for all maintenance commands

2024-01-17 Thread Jeff Davis
On Thu, 2024-01-18 at 09:24 +0530, Shubham Khanna wrote: > I tried to apply the patch but it is failing at the Head. It is > giving > the following error: I am withdrawing this patch from the CF until it's more clear that this is what we really want to do. Thank you for looking into it.

Re: Synchronizing slots from primary to standby

2024-01-17 Thread Peter Smith
I have one question about the new code in v63-0002. == src/backend/replication/logical/slotsync.c 1. ReplSlotSyncWorkerMain + Assert(SlotSyncWorker->pid == InvalidPid); + + /* + * Startup process signaled the slot sync worker to stop, so if meanwhile + * postmaster ended up starting the

Re: initdb's -c option behaves wrong way?

2024-01-17 Thread Kyotaro Horiguchi
Thank you for upicking this up. At Wed, 17 Jan 2024 23:47:41 +0100, Daniel Gustafsson wrote in > > On 17 Jan 2024, at 21:33, Tom Lane wrote: > > > > I wrote: > >> However ... I don't like the patch much. It seems to have left > >> the code in a rather random state. Why, for example, didn't

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-17 Thread John Naylor
On Thu, Jan 18, 2024 at 8:31 AM Masahiko Sawada wrote: > It seems we cannot make this work nicely. IIUC VacDeadItems is > allocated in DSM and TidStore is embedded there. However, > dead_items->items.area is a local pointer to dsa_area. So we cannot > include dsa_area in neither TidStore nor

Re: Strange Bitmapset manipulation in DiscreteKnapsack()

2024-01-17 Thread David Rowley
On Thu, 18 Jan 2024 at 14:58, Andy Fan wrote: > I want to know if "user just want to zero out the flags in bitmapset > but keeping the memory allocation" is a valid requirement. Commit > 00b41463c makes it is hard IIUC. Looking at your patch, I see: +/* + * does this break commit

Re: Fix search_path for all maintenance commands

2024-01-17 Thread Shubham Khanna
On Thu, Jan 18, 2024 at 9:19 AM Jeff Davis wrote: > > On Mon, 2023-07-17 at 12:16 -0700, Jeff Davis wrote: > > Based on feedback, I plan to commit soon. > > Attached is a new version. > > Changes: > > * Also switch the search_path during CREATE MATERIALIZED VIEW, so that > it's consistent with

Re: Strange Bitmapset manipulation in DiscreteKnapsack()

2024-01-17 Thread David Rowley
Thanks for having a look at this again. On Thu, 18 Jan 2024 at 15:22, Richard Guo wrote: > Do you think we can use 'memcpy(a, b, BITMAPSET_SIZE(b->nwords))' > directly in the new bms_replace_members() instead of copying the > bitmapwords one by one, like: > > - i = 0; > - do > - { > -

Re: Strange Bitmapset manipulation in DiscreteKnapsack()

2024-01-17 Thread Richard Guo
On Thu, Jan 18, 2024 at 8:35 AM David Rowley wrote: > The functions's header comment mentions "The bitmapsets are all > pre-initialized with an unused high bit so that memory allocation is > done only once.". Ah, I neglected to notice this when reviewing the v1 patch. I guess it's implemented

Re: Add system identifier to backup manifest

2024-01-17 Thread Michael Paquier
On Wed, Jan 17, 2024 at 08:46:09AM -0500, Robert Haas wrote: > On Wed, Jan 17, 2024 at 6:45 AM Alvaro Herrera > wrote: >> Hmm, okay, but what if I take a full backup from a primary server and >> later I want an incremental from a standby, or the other way around? >> Will this prevent me from

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-17 Thread torikoshia
On 2024-01-18 10:10, jian he wrote: On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada wrote: On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote: > > Alexander Korotkov writes: > > On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi > > wrote: > >> On the other hand, SAVE_ERROR_TO takes 'error' or

Re: Add last_commit_lsn to pg_stat_database

2024-01-17 Thread James Coleman
On Sun, Jan 14, 2024 at 6:01 AM vignesh C wrote: > > On Sat, 10 Jun 2023 at 07:57, James Coleman wrote: > > > > I've previously noted in "Add last commit LSN to > > pg_last_committed_xact()" [1] that it's not possible to monitor how > > many bytes of WAL behind a logical replication slot is

Re: Add last_commit_lsn to pg_stat_database

2024-01-17 Thread James Coleman
Hello, Thanks for reviewing! On Tue, Sep 19, 2023 at 8:16 AM Aleksander Alekseev wrote: > > Hi, > > > [...] > > As I was thinking about how to improve things, I realized that this > > information (since it's for monitoring anyway) fits more naturally > > into the stats system. I'd originally

Re: Strange Bitmapset manipulation in DiscreteKnapsack()

2024-01-17 Thread Andy Fan
Hi, David Rowley writes: > On Tue, 16 Jan 2024 at 16:32, David Rowley wrote: >> >> >> Now that 00b41463c changed Bitmapset to have NULL be the only valid >> representation of an empty set, this code no longer makes sense. We >> may as well just bms_free() the original set and bms_copy() in

Re: subscription disable_on_error not working after ALTER SUBSCRIPTION set bad conninfo

2024-01-17 Thread Masahiko Sawada
On Thu, Jan 18, 2024 at 8:16 AM Peter Smith wrote: > > Hi, > > I had reported a possible subscription 'disable_on_error' bug found > while reviewing another patch. > > I am including my initial report and Nisha's analysis again here so > that this topic has its own thread. > > ==

linux cachestat in file Readv and Prefetch

2024-01-17 Thread Cédric Villemain
Hi, I was testing the index prefetch and streamIO patches and I added cachestat() syscall to get a better view of the prefetching. It's a new linux syscall, it requires 6.5, it provides numerous interesting information from the VM for the range of pages examined. It's way way faster than the

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-17 Thread Masahiko Sawada
On Wed, Jan 17, 2024 at 11:37 AM John Naylor wrote: > > On Wed, Jan 17, 2024 at 8:39 AM Masahiko Sawada wrote: > > > > On Wed, Jan 17, 2024 at 9:20 AM John Naylor wrote: > > > > > > On Tue, Jan 16, 2024 at 1:18 PM Masahiko Sawada > > > wrote: > > > > Just changing "items" to be the local

Re: [PATCH] Add additional extended protocol commands to psql: \parse and \bindx

2024-01-17 Thread Michael Paquier
On Wed, Jan 17, 2024 at 10:05:33AM +0100, Anthonin Bonnefoy wrote: > > I do realize the same is true for plain \bind, but it seems > > like a bug there too. > > The unscanned bind's parameters are discarded later in the > HandleSlashCmds functions. So adding the ignore_slash_options() for >

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-17 Thread Michael Paquier
On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote: > I agree with your points. While the other I/O related work is > happening we can discuss what we should do in the variable op_byte > cases. Also, this is happening only for WAL right now but if we try to > extend pg_stat_io in

Re: CI and test improvements

2024-01-17 Thread Michael Paquier
On Wed, Jan 17, 2024 at 05:34:00PM +0530, vignesh C wrote: > Are we planning to do anything more in this thread, the thread has > been idle for more than 7 months. If nothing more is planned for this, > I'm planning to close this commitfest entry in this commitfest. Oops, this went through the

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-17 Thread jian he
On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada wrote: > > On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote: > > > > Alexander Korotkov writes: > > > On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi > > > wrote: > > >> On the other hand, SAVE_ERROR_TO takes 'error' or 'none', which > > >>

Re: Where can I find the doxyfile?

2024-01-17 Thread vignesh C
On Tue, 7 Nov 2023 at 12:23, John Morris wrote: > > Another update, this time with an abbreviated Doxyfile. Rather than including > the full Doxyfile, this updated version only includes modified settings. It > is more compact and more likely to survive across future doxygen versions. Meson

Re: Add system identifier to backup manifest

2024-01-17 Thread Sravan Kumar
I have also done a review of the patch and some testing. The patch looks good, and I agree with Robert's comments. On Wed, Jan 17, 2024 at 8:40 PM Robert Haas wrote: > > On Wed, Jan 17, 2024 at 6:31 AM Amul Sul wrote: > > With the attached patch, the backup manifest will have a new key item as

Re: pgsql: Clean up role created in new subscription test.

2024-01-17 Thread vignesh C
On Fri, 1 Dec 2023 at 18:23, Daniel Gustafsson wrote: > > > On 1 Dec 2023, at 13:19, Alvaro Herrera wrote: > > > > Isn't it simpler to use DROP OWNED BY in 0001? > > I suppose it is, I kind of like the explicit drops but we do use OWNED BY > quite > a lot in the regression tests so changed to

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-17 Thread Masahiko Sawada
On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote: > > Alexander Korotkov writes: > > On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi > > wrote: > >> On the other hand, SAVE_ERROR_TO takes 'error' or 'none', which > >> indicate "immediately error out" and 'just ignore the failure' > >>

Re: Teach predtest about IS [NOT] proofs

2024-01-17 Thread James Coleman
On Fri, Dec 22, 2023 at 2:48 PM Tom Lane wrote: > > James Coleman writes: > > I've not yet applied all of your feedback, but I wanted to get an > > initial read on your thoughts on how using switch statements ends up > > looking. Attached is a single (pure refactor) patch that converts the > >

Re: Strange Bitmapset manipulation in DiscreteKnapsack()

2024-01-17 Thread David Rowley
On Tue, 16 Jan 2024 at 16:32, David Rowley wrote: > > While working on [1], I noticed some strange code in > DiscreteKnapsack() which seems to be aiming to copy the Bitmapset. > > It's not that obvious at a casual glance, but: > > sets[j] = bms_del_members(sets[j], sets[j]); > > this is aiming to

linux cachestat in file Readv and Prefetch

2024-01-17 Thread Cedric Villemain
Hi, I was testing the index prefetch and streamIO patches and I added cachestat() syscall to get a better view of the prefetching. It's a new linux syscall, it requires 6.5, it provides numerous interesting information from the VM for the range of pages examined. It's way way faster than the

Re: Improve the connection failure error messages

2024-01-17 Thread Peter Smith
On Wed, Jan 17, 2024 at 7:15 PM Nisha Moond wrote: > > > > > ~~ > > > > BTW, while experimenting with the bad connection ALTER I also tried > > setting 'disable_on_error' like below: > > > > ALTER SUBSCRIPTION sub4 SET (disable_on_error); > > ALTER SUBSCRIPTION sub4 CONNECTION 'port = -1'; > > >

subscription disable_on_error not working after ALTER SUBSCRIPTION set bad conninfo

2024-01-17 Thread Peter Smith
Hi, I had reported a possible subscription 'disable_on_error' bug found while reviewing another patch. I am including my initial report and Nisha's analysis again here so that this topic has its own thread. == INITIAL REPORT [1] == ... I see now that any ALTER

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Peter Geoghegan
On Wed, Jan 17, 2024 at 5:47 PM Melanie Plageman wrote: > > On Wed, Jan 17, 2024 at 4:31 PM Peter Geoghegan wrote: > > > > On Wed, Jan 17, 2024 at 4:25 PM Peter Geoghegan wrote: > > > I tend to suspect that VACUUM_FSM_EVERY_PAGES is fundamentally the > > > wrong idea. If it's such a good idea

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Melanie Plageman
On Wed, Jan 17, 2024 at 4:31 PM Peter Geoghegan wrote: > > On Wed, Jan 17, 2024 at 4:25 PM Peter Geoghegan wrote: > > I tend to suspect that VACUUM_FSM_EVERY_PAGES is fundamentally the > > wrong idea. If it's such a good idea then why not apply it all the > > time? That is, why not apply it

Re: initdb's -c option behaves wrong way?

2024-01-17 Thread Daniel Gustafsson
> On 17 Jan 2024, at 21:33, Tom Lane wrote: > > I wrote: >> However ... I don't like the patch much. It seems to have left >> the code in a rather random state. Why, for example, didn't you >> keep all the code that constructs the "newline" value together? > > After thinking about it a bit

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Melanie Plageman
On Wed, Jan 17, 2024 at 4:25 PM Peter Geoghegan wrote: > > On Wed, Jan 17, 2024 at 3:58 PM Robert Haas wrote: > > > Ah, I realize I was not clear. I am now talking about inconsistencies > > > in vacuuming the FSM itself. FreeSpaceMapVacuumRange(). Not updating > > > the freespace map during the

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Melanie Plageman
On Wed, Jan 17, 2024 at 3:58 PM Robert Haas wrote: > > On Wed, Jan 17, 2024 at 3:12 PM Melanie Plageman > wrote: > > > Yes, I also spent some time thinking about this. In master, we do > > always call lazy_scan_new_or_empty() before calling > > lazy_scan_noprune(). The code is aiming to ensure

modify first-word capitalisation of some messages

2024-01-17 Thread Peter Smith
Hi. PSA a small patch to adjust the first-word capitalisation of some errmsg/ errdetail/ errhint so they comply with the guidelines. == Kind Regards, Peter Smith. Fujitsu Australia v1-0001-Error-message-capitalisation.patch Description: Binary data

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-17 Thread Tom Lane
Alexander Korotkov writes: > On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi > wrote: >> On the other hand, SAVE_ERROR_TO takes 'error' or 'none', which >> indicate "immediately error out" and 'just ignore the failure' >> respectively, but these options hardly seem to denote a 'location', >>

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Peter Geoghegan
On Wed, Jan 17, 2024 at 4:25 PM Peter Geoghegan wrote: > I tend to suspect that VACUUM_FSM_EVERY_PAGES is fundamentally the > wrong idea. If it's such a good idea then why not apply it all the > time? That is, why not apply it independently of whether nindexes==0 > in the current VACUUM

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Peter Geoghegan
On Wed, Jan 17, 2024 at 3:58 PM Robert Haas wrote: > > Ah, I realize I was not clear. I am now talking about inconsistencies > > in vacuuming the FSM itself. FreeSpaceMapVacuumRange(). Not updating > > the freespace map during the course of vacuuming the heap relation. > > Fair enough, but I'm

Re: On login trigger: take three

2024-01-17 Thread Alexander Korotkov
On Mon, Jan 15, 2024 at 11:29 AM Daniel Gustafsson wrote: > > On 13 Jan 2024, at 17:00, Alexander Lakhin wrote: > > > I suspected that this failure was caused by autovacuum, and I've managed to > > reproduce it without Valgrind or slowing down a machine. > > This might be due to the fact that

Re: Assertion failure with epoch when replaying standby records for 2PC

2024-01-17 Thread Alexander Korotkov
Hi, Michael! On Wed, Jan 17, 2024 at 7:47 AM Michael Paquier wrote: > rorqual has failed today with a very interesting failure: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=rorqual=2024-01-17%2005%3A06%3A31 > > This has caused an assertion failure for a 2PC transaction when >

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-17 Thread Alexander Korotkov
On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi wrote: > At Wed, 17 Jan 2024 14:38:54 +0900, torikoshia > wrote in > > Hi, > > > > Thanks for applying! > > > > > + errmsg_plural("%zd row were skipped due to data type > > > incompatibility", > > > > Sorry, I just noticed it, but 'were' should

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-17 Thread Alexander Korotkov
On Wed, Jan 17, 2024 at 7:38 AM torikoshia wrote: > > Hi, > > Thanks for applying! > > > + errmsg_plural("%zd row were skipped due > > to data type incompatibility", > > Sorry, I just noticed it, but 'were' should be 'was' here? Sure, the fix is pushed. --

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Robert Haas
On Wed, Jan 17, 2024 at 3:12 PM Melanie Plageman wrote: > > Reviewing 0001, consider the case where a table has no indexes. > > Pre-patch, PageTruncateLinePointerArray() will happen when > > lazy_vacuum_heap_page() is called; post-patch, it will not occur. > > That's a loss. Should we care? On

Re: initdb's -c option behaves wrong way?

2024-01-17 Thread Tom Lane
I wrote: > However ... I don't like the patch much. It seems to have left > the code in a rather random state. Why, for example, didn't you > keep all the code that constructs the "newline" value together? After thinking about it a bit more, I don't see why you didn't just

Re: initdb's -c option behaves wrong way?

2024-01-17 Thread Tom Lane
Robert Haas writes: > On Wed, Jan 17, 2024 at 2:31 PM Daniel Gustafsson wrote: >> Agreed, I think the patch as it stands now where it replaces case >> insensitive, >> while keeping the original casing, is the best path forward. The issue exist >> in 16 as well so I propose a backpatch to

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Melanie Plageman
On Wed, Jan 17, 2024 at 12:17 PM Robert Haas wrote: > > On Tue, Jan 16, 2024 at 6:07 PM Melanie Plageman > wrote: > > Attached v8 patch set is rebased over this. > > Reviewing 0001, consider the case where a table has no indexes. > Pre-patch, PageTruncateLinePointerArray() will happen when >

Re: cleanup patches for incremental backup

2024-01-17 Thread Robert Haas
On Wed, Jan 17, 2024 at 1:42 PM Matthias van de Meent wrote: > Sure, added in attached. I think this mostly looks good now but I just realized that I think this needs rephrasing: + To restore incremental backups the tool + is used, which combines incremental backups with a base backup

Re: initdb's -c option behaves wrong way?

2024-01-17 Thread Robert Haas
On Wed, Jan 17, 2024 at 2:31 PM Daniel Gustafsson wrote: > Agreed, I think the patch as it stands now where it replaces case insensitive, > while keeping the original casing, is the best path forward. The issue exist > in 16 as well so I propose a backpatch to there. I like that approach, too.

Re: psql JSON output format

2024-01-17 Thread Robert Haas
On Wed, Jan 17, 2024 at 4:30 AM Laurenz Albe wrote: > As mentioned in my other mail, I was talking about the psql output > format "csv" rather than about COPY. Oh. Well, I think it's sad that the psql format csv has that property. Why doesn't it adopt COPY's handling? > I agree that it is

Re: More new SQL/JSON item methods

2024-01-17 Thread Peter Eisentraut
On 17.01.24 10:03, Jeevan Chalke wrote: I added unary '+' and '-' support as well and thus thought of having separate rules altogether rather than folding those in. Per SQL standard, the precision and scale arguments are unsigned integers, so unary plus and minus signs are not

Re: initdb's -c option behaves wrong way?

2024-01-17 Thread Daniel Gustafsson
> On 17 Jan 2024, at 18:05, Tom Lane wrote: > > Alvaro Herrera writes: >> Hmm, how about raising an error if multiple options are given targetting >> the same GUC? > > I don't see any reason to do that. The underlying configuration > files don't complain about duplicate entries, they just

Re: cleanup patches for incremental backup

2024-01-17 Thread Matthias van de Meent
On Tue, 16 Jan 2024 at 21:49, Robert Haas wrote: > > On Tue, Jan 16, 2024 at 3:22 PM Matthias van de Meent > wrote: > + A special base > backup > + that for some WAL-logged relations only contains the pages that were > + modified since a previous backup, as opposed to the full

Re: New Table Access Methods for Multi and Single Inserts

2024-01-17 Thread Bharath Rupireddy
On Tue, Aug 1, 2023 at 10:32 PM Jacob Champion wrote: > > On Tue, Aug 1, 2023 at 9:31 AM Bharath Rupireddy > wrote: > > Thanks. Finally, I started to spend time on this. Just curious - may > > I know the discussion in/for which this patch is referenced? What was > > the motive? Is it captured

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-17 Thread Robert Haas
On Tue, Jan 16, 2024 at 6:07 PM Melanie Plageman wrote: > Attached v8 patch set is rebased over this. Reviewing 0001, consider the case where a table has no indexes. Pre-patch, PageTruncateLinePointerArray() will happen when lazy_vacuum_heap_page() is called; post-patch, it will not occur.

Re: initdb's -c option behaves wrong way?

2024-01-17 Thread Tom Lane
Alvaro Herrera writes: > Hmm, how about raising an error if multiple options are given targetting > the same GUC? I don't see any reason to do that. The underlying configuration files don't complain about duplicate entries, they just take the last setting. regards, tom

Re: More new SQL/JSON item methods

2024-01-17 Thread Andrew Dunstan
On 2024-01-17 We 04:03, Jeevan Chalke wrote: On Mon, Jan 15, 2024 at 7:41 PM Peter Eisentraut wrote: Overall, I think it would be better if you combined all three of these patches into one.  Right now, you have arranged these as incremental features, and as a result of

Re: Build the docs if there are changes in docs and don't run other tasks if the changes are only in docs

2024-01-17 Thread Andrew Dunstan
On 2023-10-06 Fr 10:07, Tom Lane wrote: Peter Eisentraut writes: I don't have a good sense of what you are trying to optimize for. If it's the mainline build-on-every-commit type, then I wonder how many commits would really be affected by this. Like, how many commits touch only a README

Re: psql JSON output format

2024-01-17 Thread Andrew Dunstan
On 2024-01-17 We 03:52, Laurenz Albe wrote: On Tue, 2024-01-16 at 11:49 -0500, Andrew Dunstan wrote: On 2024-01-16 Tu 11:07, Laurenz Albe wrote: On Tue, 2024-01-09 at 16:51 +, Dean Rasheed wrote: On Tue, 9 Jan 2024 at 14:35, Christoph Berg wrote: Getting it print numeric/boolean

[pg16]Question about building snapshot in logical decoding

2024-01-17 Thread cca5507
If txn1 begin after SNAPBUILD_BUILDING_SNAPSHOT and commit before SNAPBUILD_FULL_SNAPSHOT(so txn1 will not be processed by DecodeCommit), and txn2 begin after SNAPBUILD_FULL_SNAPSHOT and commit after SNAPBUILD_CONSISTENT(so txn2 will be replayed), how to ensure that txn2 could see the changes made

Re: initdb's -c option behaves wrong way?

2024-01-17 Thread Alvaro Herrera
On 2024-Jan-16, Daniel Gustafsson wrote: > > On 28 Sep 2023, at 09:49, Kyotaro Horiguchi wrote: > > > I noticed that -c option of initdb behaves in an unexpected > > manner. Identical variable names with variations in letter casing are > > treated as distinct variables. > > > > $ initdb

Re: Add pgindent test to check if codebase is correctly formatted

2024-01-17 Thread Tristan Partin
On Wed Jan 17, 2024 at 3:50 AM CST, Alvaro Herrera wrote: Hmm, should this also install typedefs.list and pgindent.man? What about the tooling to reformat Perl code? Good point about pgindent.man. It would definitely be good to install alongside pgindent and pg_bsd_indent. I don't know if

Re: Fix a possible socket leak at Windows (src/backend/port/win32/socket.c)

2024-01-17 Thread Ranier Vilela
Em qua., 17 de jan. de 2024 09:54, Daniel Gustafsson escreveu: > > On 17 Jan 2024, at 07:26, Michael Paquier wrote: > > On Tue, Jan 16, 2024 at 05:25:39PM -0300, Ranier Vilela wrote: > > >> Do you have plans or should I register for a commitfest? > > > > Daniel has stated that he would take

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-17 Thread Jelte Fennema-Nio
On Wed, 17 Jan 2024 at 14:39, Robert Haas wrote: > I think it's hard to say for sure what API is going to work well here, > because we just don't have much experience with this. Agreed, but I strongly believe PQunsupportedProtocolExtensions() is useful regardless of the API choice. > I also

Re: Add system identifier to backup manifest

2024-01-17 Thread Robert Haas
On Wed, Jan 17, 2024 at 6:31 AM Amul Sul wrote: > With the attached patch, the backup manifest will have a new key item as > "System-Identifier" 64-bit integer whose value is derived from pg_control > while > generating it, and the manifest version bumps to 2. > > This helps to identify the

Re: Change GUC hashtable to use simplehash?

2024-01-17 Thread Heikki Linnakangas
On 17/01/2024 09:15, John Naylor wrote: /* * hashfn_unstable.h * * Building blocks for creating fast inlineable hash functions. The * unstable designation is in contrast to hashfn.h, which cannot break * compatibility because hashes can be written to disk and so must produce * the same

Re: Build the docs if there are changes in docs and don't run other tasks if the changes are only in docs

2024-01-17 Thread Daniel Gustafsson
> On 14 Dec 2023, at 14:40, Nazir Bilal Yavuz wrote: > On Fri, 6 Oct 2023 at 17:07, Tom Lane wrote: >> Not sure what that tells us about the value of changing the CI >> logic, but it does seem like it could be worth the one-liner >> change needed to teach buildfarm animals to ignore READMEs. >

Re: [PATCH] Compression dictionaries for JSONB

2024-01-17 Thread Nikita Malakhov
Hi, Aleksander, there was a quite straightforward answer regarding Pluggable TOAST in other thread - the Pluggable TOAST feature is not desired by the community, and advanced TOAST mechanics would be accepted as parts of problematic datatypes extended functionality, on a par with in and out

Re: MERGE ... RETURNING

2024-01-17 Thread jian he
On Sat, Nov 18, 2023 at 8:55 PM Dean Rasheed wrote: > > The v13 patch still applies on top of this, so I won't re-post it. > Hi. minor issues based on v13. + +MERGING ( property ) + + The following are valid property values specifying what to return: + + + + ACTION + + +

Re: [PATCH] Compression dictionaries for JSONB

2024-01-17 Thread Aleksander Alekseev
Hi again, > Yes it does for a while now. Until we reach any agreement regarding > questions (1) and (2) personally I don't see the point in submitting > rebased patches. We can continue the discussion but mark the CF entry > as RwF for now it will be helpful. Sorry, what I actually meant were

Re: [PATCH] Compression dictionaries for JSONB

2024-01-17 Thread Aleksander Alekseev
Hi Shubham, > > > 8272749e added a few more arguments to CastCreate(). Here is the rebased > > > patch. > > > > After merging afbfc029 [1] the patch needed a rebase. PFA v10. > > > > The patch is still in a PoC state and this is exactly why comments and > > suggestions from the community are

Re: Add system identifier to backup manifest

2024-01-17 Thread Robert Haas
On Wed, Jan 17, 2024 at 6:45 AM Alvaro Herrera wrote: > Hmm, okay, but what if I take a full backup from a primary server and > later I want an incremental from a standby, or the other way around? > Will this prevent me from using such a combination? The system identifier had BETTER match in

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-17 Thread Robert Haas
On Tue, Jan 16, 2024 at 9:21 PM Jelte Fennema-Nio wrote: > It's understandable you're worried about breaking clients, but afaict > **you are actually misunderstanding the situation**. I totally agree > that we cannot bump the protocol version without also merging 0002 > (that's why the version

Re: Fix a possible socket leak at Windows (src/backend/port/win32/socket.c)

2024-01-17 Thread Daniel Gustafsson
> On 17 Jan 2024, at 07:26, Michael Paquier wrote: > On Tue, Jan 16, 2024 at 05:25:39PM -0300, Ranier Vilela wrote: >> Do you have plans or should I register for a commitfest? > > Daniel has stated that he would take care of it, so why not letting > him a few days? I don't think that a CF

Re: generate syscache info automatically

2024-01-17 Thread Peter Eisentraut
On 10.01.24 09:00, John Naylor wrote: I suppose a sensible alternative could be to leave the DECLARE_..._INDEX... alone and make a separate statement, like MAKE_SYSCACHE(pg_type_oid_index, TYPEOID, 64); That's at least visually easier, because some of those DECLARE_... lines are getting pretty

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-17 Thread Nazir Bilal Yavuz
Hi, On Mon, 15 Jan 2024 at 09:27, Michael Paquier wrote: > > On Fri, Jan 12, 2024 at 04:23:26PM +0300, Nazir Bilal Yavuz wrote: > > On Thu, 11 Jan 2024 at 17:28, Melanie Plageman > > wrote: > >> Even if we made a separate view for WAL I/O stats, we would still have > >> this issue of variable

Re: Catalog domain not-null constraints

2024-01-17 Thread vignesh C
On Wed, 29 Nov 2023 at 01:14, Peter Eisentraut wrote: > > On 23.11.23 14:13, Aleksander Alekseev wrote: > > =# create domain connotnull1 integer; > > =# create domain connotnull2 integer; > > =# alter domain connotnull1 add not null value; > > =# alter domain connotnull2 set not null; > > =# \dD

Re: Add system identifier to backup manifest

2024-01-17 Thread Amul Sul
On Wed, Jan 17, 2024 at 5:15 PM Alvaro Herrera wrote: > On 2024-Jan-17, Amul Sul wrote: > > > This helps to identify the correct database server and/or backup for the > > subsequent backup operations. pg_verifybackup validates the manifest > system > > identifier against the backup control file

Re: CI and test improvements

2024-01-17 Thread vignesh C
On Wed, 12 Jul 2023 at 11:38, Michael Paquier wrote: > > On Wed, Jul 12, 2023 at 12:56:17AM -0500, Justin Pryzby wrote: > > And, since 681d9e462: > > > > security is missing from contrib/seg/meson.build > > Ugh. Good catch! Are we planning to do anything more in this thread, the thread has been

Re: Add system identifier to backup manifest

2024-01-17 Thread Alvaro Herrera
On 2024-Jan-17, Amul Sul wrote: > This helps to identify the correct database server and/or backup for the > subsequent backup operations. pg_verifybackup validates the manifest system > identifier against the backup control file and fails if they don’t match. > Similarly, pg_basebackup

Add system identifier to backup manifest

2024-01-17 Thread Amul Sul
Hi All, With the attached patch, the backup manifest will have a new key item as "System-Identifier" 64-bit integer whose value is derived from pg_control while generating it, and the manifest version bumps to 2. This helps to identify the correct database server and/or backup for the subsequent

Re: [PATCH] Compression dictionaries for JSONB

2024-01-17 Thread Shubham Khanna
On Wed, Jan 17, 2024 at 4:16 PM Aleksander Alekseev wrote: > > Hi hackers, > > > 8272749e added a few more arguments to CastCreate(). Here is the rebased > > patch. > > After merging afbfc029 [1] the patch needed a rebase. PFA v10. > > The patch is still in a PoC state and this is exactly why

Re: Implement missing join selectivity estimation for range types

2024-01-17 Thread vignesh C
On Fri, 5 Jan 2024 at 23:09, Schoemans Maxime wrote: > > On 05/01/2024 11:37, vignesh C wrote: > > One of the tests was aborted at [1], kindly post an updated patch for > the same: > > Thank you for notifying us. > I believe I fixed the issue but it is hard to be certain as the issue > did not

Re: Synchronizing slots from primary to standby

2024-01-17 Thread shveta malik
On Wed, Jan 17, 2024 at 3:08 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Jan 16, 2024 at 05:27:05PM +0530, shveta malik wrote: > > PFA v62. Details: > > Thanks! > > > v62-003: > > It is a new patch which attempts to implement slot-sync worker as a > > special process which is neither a

Re: Add pgindent test to check if codebase is correctly formatted

2024-01-17 Thread Alvaro Herrera
Hmm, should this also install typedefs.list and pgindent.man? What about the tooling to reformat Perl code? -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "Linux transformó mi computadora, de una `máquina para hacer cosas', en un aparato realmente entretenido,

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-01-17 Thread Richard Guo
On Wed, Jan 17, 2024 at 5:01 PM Richard Guo wrote: > On Tue, Jan 16, 2024 at 2:30 AM Robert Haas wrote: > >> On Mon, Jan 8, 2024 at 3:32 AM Richard Guo >> wrote: > > > Fair point. I think we can back-patch a more minimal fix, as Tom >> > proposed in [1], which disallows the reparameterization

Re: Synchronizing slots from primary to standby

2024-01-17 Thread Bertrand Drouvot
Hi, On Tue, Jan 16, 2024 at 05:27:05PM +0530, shveta malik wrote: > PFA v62. Details: Thanks! > v62-003: > It is a new patch which attempts to implement slot-sync worker as a > special process which is neither a bgworker nor an Auxiliary process. > Here we get the benefit of converting

Re: make pg_ctl more friendly

2024-01-17 Thread Junwang Zhao
Hi Alvaro, On Wed, Jan 17, 2024 at 4:54 PM Alvaro Herrera wrote: > > I think this needs more comments. First, in the WaitPMResult enum, we > currently have three values -- READY, STILL_STARTING, FAILED. These are > all pretty self-explanatory. But this patch adds SHUTDOWN_IN_RECOVERY, > and

Re: psql JSON output format

2024-01-17 Thread Laurenz Albe
On Tue, 2024-01-16 at 14:12 -0500, Robert Haas wrote: > On Tue, Jan 16, 2024 at 11:07 AM Laurenz Albe > wrote: > > "Round-trip safety" is not so important. If you want to move data from > > PostgreSQL to PostgreSQL, you use the plain or the binary format. > > The CSV format by default renders

Re: [PATCH] Add additional extended protocol commands to psql: \parse and \bindx

2024-01-17 Thread Anthonin Bonnefoy
> I do realize the same is true for plain \bind, but it seems > like a bug there too. The unscanned bind's parameters are discarded later in the HandleSlashCmds functions. So adding the ignore_slash_options() for inactive branches scans and discards them earlier. I will add it to match what's

Re: More new SQL/JSON item methods

2024-01-17 Thread Jeevan Chalke
On Mon, Jan 15, 2024 at 7:41 PM Peter Eisentraut wrote: > Attached are two small fixup patches for your patch set. > Thanks, Peter. > > In the first one, I simplified the grammar for the .decimal() method. > It seemed a bit overkill to build a whole list structure when all we > need are 0, 1,

  1   2   >