Re: per backend WAL statistics

Hi, On Mon, Feb 17, 2025 at 04:25:46PM +0900, Michael Paquier wrote: > On Mon, Feb 17, 2025 at 06:59:40AM +, Bertrand Drouvot wrote: > > There is still something that would simplify what is done here: it's the > > "the elimination of the write & sync columns for pg_stat_wal" mentioned in > >

Re: SQL:2011 application time

On 2/13/25 05:23, Peter Eisentraut wrote: On 23.01.25 16:40, Peter Eisentraut wrote: I think my interpretation of what RESTRICT should do is different. The clause "Execution of referential actions" in the SQL standard only talks about referenced and referencing columns, not periods.  So this w

Re: per backend WAL statistics

On Mon, Feb 17, 2025 at 06:59:40AM +, Bertrand Drouvot wrote: > There is still something that would simplify what is done here: it's the > "the elimination of the write & sync columns for pg_stat_wal" mentioned in > [2]. Yeah, still you cannot just remove them because the data tracked in pg_s

Re: Proposal to CREATE FOREIGN TABLE LIKE

On Tue, Feb 11, 2025 at 10:07:48AM -0600, Sami Imseih wrote: > I agree. I was just saying the test cases you provided are > enough. No changes needed for the tests. > > I have no further comments. The checks you are adding in the parse analysis of the LIKE clauses is surprisingly light. + * For

Re: Introduce XID age and inactive timeout based replication slot invalidation

On Fri, Feb 14, 2025 at 5:30 PM Nisha Moond wrote: > > Here is a summary of changes in v78: > A few minor comments: 1. Slots that appear idle due to a disrupted connection between +the publisher and subscriber are also excluded, as they are managed by +wal_sender_timeout. ... How

Re: generic plans and "initial" pruning

Hi Junwang, On Sun, Feb 16, 2025 at 1:37 PM Junwang Zhao wrote: > On Sat, Feb 15, 2025 at 3:51 PM Amit Langote wrote: > > Thanks! It looks like I missed updating the MERGE-related lists in > > ModifyTable. > > > > I've attached a fix with a test added based on your example. I plan to > > push t

Re: per backend WAL statistics

Hi, Thank you for working on this! On Mon, 17 Feb 2025 at 09:59, Bertrand Drouvot wrote: > > There is still something that would simplify what is done here: it's the > "the elimination of the write & sync columns for pg_stat_wal" mentioned in > [2]. > > I'll add it in the new patch serie for th

Re: per backend WAL statistics

Hi, On Thu, Feb 06, 2025 at 10:28:48AM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Feb 06, 2025 at 10:38:55AM +0900, Michael Paquier wrote: > > On Wed, Feb 05, 2025 at 02:28:08PM +, Bertrand Drouvot wrote: > > > Agree, I'll start a dedicated thread for that. > > > > Thanks. > > Done in

Re: Virtual generated columns

On Sat, Feb 15, 2025 at 8:37 PM Dean Rasheed wrote: > > On Fri, 14 Feb 2025 at 10:59, Peter Eisentraut wrote: > > > > On 13.02.25 14:06, jian he wrote: > > > I didn't solve the out join semantic issue. > > > i am wondering, can we do the virtual generated column expansion in > > > the rewrite sta

Re: Introduce XID age and inactive timeout based replication slot invalidation

On Mon, Feb 17, 2025 at 11:29 AM Amit Kapila wrote: > > On Fri, Feb 14, 2025 at 5:30 PM Nisha Moond wrote: > > > > Here is a summary of changes in v78: > > > > A few minor comments: > 1. > Slots that appear idle due to a disrupted connection between > +the publisher and subscriber are als

Re: Move wal_buffers_full to WalUsage (and report it in pgss/explain)

On Fri, Feb 14, 2025 at 03:02:51PM +0900, Michael Paquier wrote: > Not sure. FWIW, it makes sense to me to group them by "family" in > this case, as they would belong to the same instrument structure. Looked at the full set again, and in the PGSS patch I have finished by grouping all the WAL fiel

Re: Some read stream improvements

On Mon, Feb 17, 2025 at 6:55 PM Kirill Reshke wrote: > Just out of curiosity, should we `Assert(*index + n < > stream->queue_size);` in `read_stream_index_advance_n`? No: it is allowed to be >= queue_size temporarily, but if so we subtract queue_size. The result should be equal to (index + n) %

Re: Some read stream improvements

On Mon, Feb 17, 2025 at 5:55 PM Thomas Munro wrote: > The solution we agreed on is to introduce a way for StartReadBuffers() > to communicate with future calls, and "forward" pinned buffers between > calls. The function arguments don't change, but its "buffers" > argument becomes an in/out array:

Re: Some read stream improvements

On Mon, 17 Feb 2025 at 09:55, Thomas Munro wrote: > > Hi, > > Here are some patches that address some of Andres's feedback since the > AIO v2 rebase[1], anticipate out-of-order streams, and make some other > minor improvements. They are independent of the main AIO patch set > and apply to master,

Re: Restrict copying of invalidated replication slots

Hi. Some review comments for patch v1-0001. == 1. DOCS? Shouldn't the documentation [1] for pg_copy_logical_replication_slot() and pg_copy_physical_replication_slot() be updated to mention this? == src/backend/replication/slotfuncs.c 2. + /* We should not copy invalidated replication sl

Re: Get rid of WALBufMappingLock

Hi! I spotted a typo in v10: + /* + * Page at nextidx wasn't initialized yet, so we cann't move + * InitializedUpto further. It will be moved by backend which + * will initialize nextidx. + */ cann't - > can't moved by backend -> moved by the backend -- Best regards, Kirill Reshke

Some read stream improvements

Hi, Here are some patches that address some of Andres's feedback since the AIO v2 rebase[1], anticipate out-of-order streams, and make some other minor improvements. They are independent of the main AIO patch set and apply to master, hence separate thread. 0001-Refactor-read_stream.c-s-circular-

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

Hi Shubham Some review comments for v7-0001. (I am late to this thread. If some of my comments have already been discussed and rejected please let me know). == 1 GENERAL. Option Name? Wondering why the patch is introducing more terminology like "cleanup"; if we are dropping publications the

Re: NOT ENFORCED constraint feature

On Fri, Feb 14, 2025 at 8:41 PM Ashutosh Bapat wrote: > > On Thu, Feb 13, 2025 at 5:27 PM Álvaro Herrera > wrote: > > > > On 2025-Feb-13, Ashutosh Bapat wrote: > > > > > > So considering that, I think a three-state system makes more sense. > > > > Something like: > > > > > > > > 1) NOT ENFORCED

Re: Add pg_accept_connections_start_time() for better uptime calculation

On Sun, Feb 16, 2025 at 07:53:06PM -0500, Tom Lane wrote: > Yeah. Making that happen would require extending the contents of > postmaster.pid, which is likely to break assorted peoples' tooling. > I doubt that this feature clears the bar for justifying that. Sure, agreed to not touch postmaster.p

Re: Add Postgres module info

On Mon, Feb 17, 2025 at 03:41:56AM +0200, Alexander Korotkov wrote: > 1) Is it intended to switch all in-core libraries to use > PG_MODULE_MAGIC_EXT()? > 2) Once we have module version information, it looks natural to > specify the required version for dependant objects, e.g. SQL-funcions > implem

Re: Introduce XID age and inactive timeout based replication slot invalidation

On Wed, Feb 12, 2025 at 1:16 PM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, February 12, 2025 11:56 AM Amit Kapila > wrote: > > > > On Tue, Feb 11, 2025 at 9:39 PM Nathan Bossart > > wrote: > > > > > > On Tue, Feb 11, 2025 at 03:22:49PM +0100, Álvaro Herrera wrote: > > > > I find this propose

Re: BackgroundPsql swallowing errors on windows

On Mon, Feb 17, 2025 at 7:02 AM Andres Freund wrote: > I don't really know enough about IPC::Run's internals to answer. My > interpretation of how it might work, purely from observation, is that it opens > one tcp connection for each "pipe" and that that's what's introducing the > potential of reo

Re: Adjust tuples estimate for appendrels

On Mon, Feb 10, 2025 at 5:22 PM Richard Guo wrote: > Thanks for all the reviews. Attached is an updated patch that > resolves the review comments. Pushed. Thanks for all the reviews. Thanks Richard

Re: UUID v7

On Sun, Feb 16, 2025 at 6:21 PM Andrew Alsup wrote: > Sergey, > > I took a look at your patch for chapter 9.14 "UUID Functions" docs page. > You've added some really good content here. I think section 9.14.4. > "Deciding Whether and Which UUID to Use" would be better suited for Chapter > 8: "Data

Re: BackgroundPsql swallowing errors on windows

Noah Misch writes: > On Sun, Feb 16, 2025 at 08:42:50PM -0500, Andres Freund wrote: >> On macos we can't currently use images, so we just cache all the installed >> macports packages. The cache is keyed by OS version and list of packages to >> be installed, with no other forced invalidation righ

Re: BackgroundPsql swallowing errors on windows

Hi, On February 16, 2025 8:42:50 PM EST, Andres Freund wrote: >On Windows cpan is used, so it should pick that new version fairly quickly if >a release has been made. Looks like no release has happened since late 2023: https://github.com/cpan-authors/IPC-Run/blob/master/Changelog So it won't

Re: BackgroundPsql swallowing errors on windows

On Sun, Feb 16, 2025 at 08:42:50PM -0500, Andres Freund wrote: > On February 16, 2025 7:50:18 PM EST, Tom Lane wrote: > >Noah Misch writes: > >> On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: > >>> I think that > >>> IPC::Run may be screwing up here, because I have seen non-Windows > >

Re: BackgroundPsql swallowing errors on windows

Hi, On February 16, 2025 7:50:18 PM EST, Tom Lane wrote: >Noah Misch writes: >> On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: >>> I think that >>> IPC::Run may be screwing up here, because I have seen non-Windows >>> CI failures that look like it didn't read all the stderr output. >

Re: Add Postgres module info

Hi, Andrei! On Mon, Dec 16, 2024 at 7:02 AM Andrei Lepikhov wrote: >ly > On 12/13/24 10:17, Tom Lane wrote: > > Andrei Lepikhov writes: > >> On 12/12/24 21:02, Yurii Rashkovskii wrote: > >>> 2. Any reasons to dictate MAJ.MIN format? With semantic versioning > >>> abound, it's rather common to us

Re: UUID v7

Sergey, I took a look at your patch for chapter 9.14 "UUID Functions" docs page. You've added some really good content here. I think section 9.14.4. "Deciding Whether and Which UUID to Use" would be better suited for Chapter 8: "Data Types" -- specifically, 8.12. "UUID Type", since the content see

Re: Improve statistics estimation considering GROUP-BY as a 'uniqueiser'

On Thu, Nov 28, 2024 at 4:39 AM Andrei Lepikhov wrote: > Thanks to take a look! > > On 11/25/24 23:45, Heikki Linnakangas wrote: > > On 24/09/2024 08:08, Andrei Lepikhov wrote: > >> + * proves the var is unique for this query. However, we'd better > >> still > >> + * believe the null-frac

Re: Add pg_accept_connections_start_time() for better uptime calculation

Michael Paquier writes: > On Sun, Feb 16, 2025 at 11:47:46AM +0100, Laurenz Albe wrote: >> Would it make sense to add that information to the output of >> "pg_ctl status" as well? Perhaps as a new option, so that default output >> format doesn't change. > A separate function that only returns th

Re: BackgroundPsql swallowing errors on windows

Noah Misch writes: > On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: >> I think that >> IPC::Run may be screwing up here, because I have seen non-Windows >> CI failures that look like it didn't read all the stderr output. >> For example, this pgbench test failure on macOS from [1]: > ht

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

Hi, Andrei! On Tue, Oct 8, 2024 at 8:00 AM Andrei Lepikhov wrote: > On 7/8/24 19:45, Andrei Lepikhov wrote: > > On 3/11/2023 23:43, Tomas Vondra wrote: > >> On 9/11/23 10:04, Lepikhov Andrei wrote: > >> * Determine bucketsize fraction and MCV frequency for the inner > >> * relation. We use th

Re: BackgroundPsql swallowing errors on windows

On Sun, Feb 16, 2025 at 07:03:39PM -0500, Andres Freund wrote: > git {show|log|...} --abbrev-commit Ah, thanks for the hint. This can also be controlled by core.abbrev at repo level with %h. Didn't suspect this keyword. > Cool, will after the minor release freeze. Thanks, Andres. If you'd pre

Re: Add pg_accept_connections_start_time() for better uptime calculation

On Sun, Feb 16, 2025 at 11:47:46AM +0100, Laurenz Albe wrote: > I myself have never felt the need for such a function - but perhaps it > can be useful in these times of hosted database services, when accessing > the log file might be difficult. Yeah. I'm not surprised by that. > I would probably

Re: BackgroundPsql swallowing errors on windows

Hi, On 2025-02-17 08:52:58 +0900, Michael Paquier wrote: > On Sat, Feb 15, 2025 at 04:13:37PM -0500, Andres Freund wrote: > > On 2025-02-14 09:52:24 -0800, Jacob Champion wrote: > >> On Fri, Feb 14, 2025 at 8:53 AM Andres Freund wrote: > >>> commit 70291a3c66e > > (Side question entirely unrelat

Re: BackgroundPsql swallowing errors on windows

Hi, On 2025-02-16 18:18:44 -0500, Tom Lane wrote: > Noah Misch writes: > > From the slow proxy's perspective, it can't rule out the program under test > > having done those two write() calls. The proxy doesn't have enough > > information to reconstruct the original four write() calls. What prev

Re: BackgroundPsql swallowing errors on windows

On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: > Noah Misch writes: > > From the slow proxy's perspective, it can't rule out the program under test > > having done those two write() calls. The proxy doesn't have enough > > information to reconstruct the original four write() calls. Wh

Re: BackgroundPsql swallowing errors on windows

On Sat, Feb 15, 2025 at 04:13:37PM -0500, Andres Freund wrote: > On 2025-02-14 09:52:24 -0800, Jacob Champion wrote: >> On Fri, Feb 14, 2025 at 8:53 AM Andres Freund wrote: >>> commit 70291a3c66e (Side question entirely unrelated as I'm reading that..) What's your magic recipe for showing up with

Re: Proposal - Allow extensions to set a Plan Identifier

On Sat, Feb 15, 2025 at 10:29:41AM +0100, Andrei Lepikhov wrote: > I have already implemented it twice in different ways as a core patch. > In my projects, we need to track queryId and plan node ID for two reasons: Are these available in the public somewhere or is that simply what Sami is proposin

Re: WAL-logging facility for pgstats kinds

On Sat, Feb 15, 2025 at 10:31:33AM +0100, Cédric Villemain wrote: > Additionally, there is significant potential for external applications to > leverage the PostgreSQL stats system to collect metrics instead of relying > on relational tables, promising a highly efficient solution. In such cases, >

Re: BackgroundPsql swallowing errors on windows

Noah Misch writes: > From the slow proxy's perspective, it can't rule out the program under test > having done those two write() calls. The proxy doesn't have enough > information to reconstruct the original four write() calls. What prevents > that anomaly? Yeah, I think it's hopeless to expect

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

Hi Ajin. FYI - Patch set v13* no longer applies cleanly. Needs rebasing. == Kind Regards, Peter Smith. Fujitsu Australia

Re: BackgroundPsql swallowing errors on windows

On Sun, Feb 16, 2025 at 03:55:10PM -0500, Andres Freund wrote: > On 2025-02-16 10:47:40 -0800, Noah Misch wrote: > > On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote: > > > On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > > > > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund w

Re: Introduce XID age and inactive timeout based replication slot invalidation

Some review comments for v78-0001. == src/backend/replication/slot.c ReportSlotInvalidation: 1. + int minutes; + int secs; The variables 'minutes' and 'seconds' are only used by case RS_INVAL_IDLE_TIMEOUT, so I think it would be better to make a new code block for that case where you can de

Re: Implement waiting for wal lsn replay: reloaded

Hi, Yura! On Thu, Feb 6, 2025 at 10:31 AM Yura Sokolov wrote: > I briefly looked into patch and have couple of minor remarks: > > 1. I don't like `palloc` in the `WaitLSNWakeup`. I believe it wont issue > problems, but still don't like it. I'd prefer to see local fixed array, say > of 16 element

Re: BackgroundPsql swallowing errors on windows

Hi, On 2025-02-16 10:47:40 -0800, Noah Misch wrote: > On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote: > > On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > > > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > > > > I suspect what's happening is that the communication

Re: DOCS - inactive_since field readability

On Fri, Feb 14, 2025 at 4:06 PM Amit Kapila wrote: > ... > > No, I meant to say that the description didn't looked any better to me > even after your 0001 patch. The second paragraph started immediately > in the next line which doesn't make it look any better. If we really > want to make it look b

Re: BackgroundPsql swallowing errors on windows

On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote: > On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > > > I suspect what's happening is that the communication with the > > > external process allows for reordering betwee

Re: Confine vacuum skip logic to lazy_scan_skip

Thomas Munro writes: > Thanks! It's green again. The security team's Coverity instance complained about this patch: *** CID 1642971: Null pointer dereferences (FORWARD_NULL) /srv/coverity/git/pgsql-git/postgresql/src/backend/access/heap/vacuumlazy.c: 1295 in lazy_scan_heap() 1289

Re: BackgroundPsql swallowing errors on windows

Hi, On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > > I suspect what's happening is that the communication with the > > external process allows for reordering between stdout/stderr. > > > > And indeed, changing BackgroundPsql::que

Re: BackgroundPsql swallowing errors on windows

On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > My understanding is that IPC::Run uses a proxy process on windows to execute > subprocesses and then communicates with that over TCP (or something along > those lines). Right. > I suspect what's happening is that the communication w

Re: psql: Add tab completion for ALTER USER RESET

On 2/15/25 12:14, Robins Tharakan wrote: > Hi Tomas, > > Thanks for taking a look - apologies for the delay here. > > On Tue, 10 Dec 2024 at 09:09, Tomas Vondra > wrote: >> >> 1) Does it make sense to still show "ALL" when the query returns >> nothing? Not sure if we a

Re: Add Option To Check All Addresses For Matching target_session_attr

Hi Andrew! cc Jelte, I suspect he might be interested. > On 20 Nov 2024, at 20:51, Andrew Jackson wrote: > > Would appreciate any feedback on the applicability/relevancy of the goal here > or the implementation. Thank you for raising the issue. Following our discussion in Discord I'm putting

Re: Add pg_accept_connections_start_time() for better uptime calculation

On Sun, 2025-02-16 at 17:35 +1030, Robins Tharakan wrote: > This patch introduces a new function pg_accept_connections_start_time(). > > Currently, pg_postmaster_start_time() is used to determine when the > database started. However, this is not accurate since the postmaster > process can sometime