Re: Make COPY format extendable: Extract COPY TO format implementations

2025-03-29 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Sat, 29 Mar 2025 09:48:22 -0700, "David G. Johnston" wrote: > I don't think this module should be responsible for testing the validity of > "qualified names in a string literal" behavior. Having some of th

Re: pg_stat_database.checksum_failures vs shared relations

2025-03-29 Thread Andres Freund
Hi, On 2025-03-29 13:17:44 -0400, Andres Freund wrote: > On 2025-03-28 13:47:16 -0400, Andres Freund wrote: > > Attached is a fix for the issue. > > I plan to push this fix soon, unless somebody protests... And done. Greetings, Andres Freund

Re: Amcheck verification of GiST and GIN

2025-03-29 Thread Tom Lane
Tomas Vondra writes: > I've pushed all the parts of this patch series, except for the stress > test - which I think was not meant for commit. > buildfarm seems happy so far, except for a minor indentation issue > (forgot to reindent after merging the separate fix patch). Not so much here: https:

Re: AIO v2.5

2025-03-29 Thread Noah Misch
On Sat, Mar 29, 2025 at 08:39:54PM -0400, Andres Freund wrote: > On 2025-03-29 14:29:29 -0700, Noah Misch wrote: > > On Wed, Mar 26, 2025 at 09:07:40PM -0400, Andres Freund wrote: > > The choice between LOG and LOG_SERVER_ONLY doesn't matter much for $SUBJECT. > > If a client has decided to set cl

Re: speedup COPY TO for partitioned table.

2025-03-29 Thread vignesh C
On Sat, 29 Mar 2025 at 12:08, jian he wrote: > > > I consolidated it into a new function: CopyThisRelTo. Few comments: 1) Here the error message is not correct, we are printing the original table from where copy was done which is a regular table and not a foreign table, we should use childreloid

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-03-29 Thread Alena Rybakina
Hi! On 29.03.2025 14:03, Alexander Korotkov wrote: On Wed, Mar 12, 2025 at 8:11 PM Alena Rybakina wrote: On 06.03.2025 11:23, Alexander Korotkov wrote: Hi, Alena! On Sat, Mar 1, 2025 at 1:39 PM Alena Rybakina wrote: On 09.02.2025 18:38, Alexander Korotkov wrote: Also, aren't we too restr

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

2025-03-29 Thread Srinath Reddy
./psql postgres postgres=# BEGIN; Hi, On Fri, Mar 28, 2025 at 8:13 PM Nathan Bossart wrote: > > + /* Report error if dbname have newline or carriage return in name. > */ > + if (strpbrk(dbname, "\n\r")) > + ereport(ERROR, > + (errcode(ERRC

Re: Using read stream in autoprewarm

2025-03-29 Thread Melanie Plageman
On Sat, Mar 29, 2025 at 4:44 PM Andres Freund wrote: > > How about having an iterator function operating on a pointer to iterator state > that's used both by the main loop and the read stream callback? If the > iterator reaches the next relation, it returns InvalidBlockNumber and the main > loop s

Re: AIO v2.5

2025-03-29 Thread Andres Freund
Hi, On 2025-03-29 14:29:29 -0700, Noah Misch wrote: > Flushing half-baked review comments before going offline for a few hours: > > On Wed, Mar 26, 2025 at 09:07:40PM -0400, Andres Freund wrote: > > Attached v2.13, with the following changes: > > > 5) The WARNING in the callback is now a LOG, as

[PATCH] Avoid useless prefetches in case of recent FPI WAL records

2025-03-29 Thread Michael Zhilin
Hi, One customer complained that physical standby has significant replay lag (few hours) on modern powerful bare metal servers. According to output of "top", startup process and few kernel workers utilized CPU, mostly in kernel space. The incident has been resolved by disabling recovery_pref

Re: Partial aggregates pushdown

2025-03-29 Thread Bruce Momjian
On Fri, Mar 28, 2025 at 02:00:44AM +, fujii.y...@df.mitsubishielectric.co.jp wrote: > Hi Bruce, Jelte, hackers. > > I apologize for my late response. > > > From: Jelte Fennema-Nio > > Sent: Thursday, August 8, 2024 8:49 PM > > SUMMARY OF THREAD > > > > The design of patch 0001 is agreed up

Re: AIO v2.5

2025-03-29 Thread Noah Misch
Flushing half-baked review comments before going offline for a few hours: On Wed, Mar 26, 2025 at 09:07:40PM -0400, Andres Freund wrote: > Attached v2.13, with the following changes: > 5) The WARNING in the callback is now a LOG, as it will be sent to the > client as a WARNING explicitly w

Re: AIO v2.5

2025-03-29 Thread Andres Freund
Hi, On 2025-03-29 10:48:10 -0400, Andres Freund wrote: > Attached is v2.14: FWIW, there was a last minute change in the test that fails in one task on CI, due to reading across the smaller segment size configured for one of the runs. Doesn't quite seem worth posting a new version for. > - push

Re: Using read stream in autoprewarm

2025-03-29 Thread Melanie Plageman
On Fri, Nov 29, 2024 at 6:20 AM Nazir Bilal Yavuz wrote: > > v4 is attached. I've started looking at this version. What I don't like about this structure is that we are forced to increment the position in the array of BlockInfoRecords in both the callback and the main loop in autoprewarm_database

Re: Using read stream in autoprewarm

2025-03-29 Thread Andres Freund
Hi, On 2025-03-29 16:09:56 -0400, Melanie Plageman wrote: > I've started looking at this version. What I don't like about this > structure is that we are forced to increment the position in the array > of BlockInfoRecords in both the callback and the main loop in > autoprewarm_database_main(). The

Re: Using read stream in autoprewarm

2025-03-29 Thread Melanie Plageman
On Sat, Mar 29, 2025 at 4:09 PM Melanie Plageman wrote: > > One alternative is to loop through the array of BlockInfoRecords and > get the start and end positions of the blocks in the arary for a > single relation/fork combo. Then we could make the read stream and > pass those two positions and th

Re: Why does wait_for_log() return current file size

2025-03-29 Thread Andres Freund
Hi, On 2025-03-29 12:09:58 -0400, Andrew Dunstan wrote: > On 2025-03-29 Sa 11:15 AM, Andres Freund wrote: > > Hi, > > > > In a test I'd like to use wait_for_log() to find a bunch of log messages > > emitted in sequence. A reasonable looking pattern for that would be: > > > > $log_location = -s

Re: AIO v2.5

2025-03-29 Thread Melanie Plageman
On Sat, Mar 29, 2025 at 2:25 PM Andres Freund wrote: > > I think I found an issue with this one - as it stands the view was viewable by > everyone. While it doesn't provide a *lot* of insight, it still seems a bit > too much for an unprivileged user to learn what part of a relation any other > use

Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

2025-03-29 Thread Andrew Dunstan
On 2025-03-29 Sa 2:58 PM, David G. Johnston wrote: On Sat, Mar 29, 2025 at 11:56 AM Andrew Dunstan wrote: I don't believe that the premise supports the conclusion. Regardless, I do support this patch and probably any similar ones proposed in the future.  Do you have an opinion on that?

Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

2025-03-29 Thread David G. Johnston
On Sat, Mar 29, 2025 at 12:27 PM Kirill Reshke wrote: > On Sat, 29 Mar 2025 at 19:59, David G. Johnston > wrote: > > Regardless, I do support this patch and probably any similar ones > proposed in the future. Do you have an opinion on that? > > > > I do also support what this patch aims to do,

Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

2025-03-29 Thread Kirill Reshke
On Sat, 29 Mar 2025 at 19:59, David G. Johnston wrote: > Regardless, I do support this patch and probably any similar ones proposed in > the future. Do you have an opinion on that? > > David J. > I do also support what this patch aims to do, how do you like v1? -- Best regards, Kirill Reshke

Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

2025-03-29 Thread Andrew Dunstan
On 2025-03-29 Sa 12:17 PM, David G. Johnston wrote: On Sat, Mar 29, 2025 at 9:06 AM Andrew Dunstan wrote: On 2025-03-29 Sa 10:40 AM, David G. Johnston wrote: On Saturday, March 29, 2025, Kirill Reshke wrote: On Sat, 29 Mar 2025 at 09:47, jian he wrote:

Re: Speed up ICU case conversion by using ucasemap_utf8To*()

2025-03-29 Thread Andres Freund
On 2025-03-17 12:16:11 +0530, vignesh C wrote: > On Fri, 20 Dec 2024 at 10:50, Andreas Karlsson wrote: > > > > Hi, > > > > Jeff pointed out to me that the case conversion functions in ICU have > > UTF-8 specific versions which means we can call those directly if the > > database encoding is UTF-8

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

2025-03-29 Thread Srinath Reddy
./psql postgres Hi, On Fri, Mar 28, 2025 at 5:08 PM Mahendra Singh Thalor wrote: > > v04_001* has the changes for CREATE DATABASE/ROLE/USER and > v04_002* has the changes into pg_upgrade to give ALERTS for invalid names. > i have reviewed these patches,in v04_002* i think it would be better if

Re: Proposal: Progressive explain

2025-03-29 Thread Rafael Thofehrn Castro
> (2) Merge progressive_explain = explain and progressive_explain = > analyze into a single value, progressive_explain = on. To tell which > is intended, just check progresive_explain_interval. If it's zero, > then there's no repeat, so we mean what you currently call > progressive_explain = explai

Re: 64 bit numbers vs format strings

2025-03-29 Thread Peter Eisentraut
On 10.03.25 10:49, Peter Eisentraut wrote: On 02.03.25 22:08, Thomas Munro wrote: On Mon, Mar 3, 2025 at 6:21 AM Peter Eisentraut wrote: On 05.12.24 23:18, Thomas Munro wrote:   Old: errmsg("hello %llu", (unsigned long long) x)   New: errmsg("hello %" PRIu64, x) I have committed the

Re: pg_stat_database.checksum_failures vs shared relations

2025-03-29 Thread Andres Freund
Hi, On 2025-03-28 13:47:16 -0400, Andres Freund wrote: > Attached is a fix for the issue. I plan to push this fix soon, unless somebody protests... Greetings, Andres Freund

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-03-29 Thread David G. Johnston
On Wed, Mar 26, 2025 at 8:28 PM Sutou Kouhei wrote: > > The attached v39 patch set uses the followings: > > 0001: Create copyto_internal.h and change COPY_XXX to > COPY_SOURCE_XXX and COPY_DEST_XXX accordingly. > (Same as 1. in your suggestion) > 0002: Support custom format for both C

Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

2025-03-29 Thread David G. Johnston
On Sat, Mar 29, 2025 at 9:06 AM Andrew Dunstan wrote: > > On 2025-03-29 Sa 10:40 AM, David G. Johnston wrote: > > On Saturday, March 29, 2025, Kirill Reshke wrote: > >> On Sat, 29 Mar 2025 at 09:47, jian he >> wrote: >> > >> > will use {table_beginscan, table_scan_getnextslot, table_endscan} >>

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-03-29 Thread David G. Johnston
On Sat, Mar 29, 2025 at 1:57 AM Sutou Kouhei wrote: > * I still think that someone may don't like defining COPY > handlers for built-in formats. If we don't define COPY > handlers for built-in formats finally, we can just drop > 0004. > We should (and usually do) dog-food APIs when reason

Re: Amcheck verification of GiST and GIN

2025-03-29 Thread Tomas Vondra
On 3/28/25 20:51, Kirill Reshke wrote: > On Fri, 28 Mar 2025 at 21:26, Tomas Vondra wrote: >> >> Here's a polished version of the patches. If you have any >> comments/objections, please speak now. >> -- >> Tomas Vondra > > Hi, no objections, lgtm > I've pushed all the parts of this patch series

why in appendShellStringNoError() loop still continues even after it found CR or LF?

2025-03-29 Thread Srinath Reddy
./psql postgres postgres=# BEGIN; Hi, i have naive doubt $SUBJECT,AFAIK The reasoning would be this function is designed to silently filter out \n and \r while still producing a usable shell-safe argument. It informs the caller of the issue (false return value) but does not abruptly stop executio

Why does wait_for_log() return current file size

2025-03-29 Thread Andres Freund
Hi, In a test I'd like to use wait_for_log() to find a bunch of log messages emitted in sequence. A reasonable looking pattern for that would be: $log_location = -s $node->logfile; $log_location = $node->wait_for_log(qr/first-message/, $log_location); $log_location = $nod

Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

2025-03-29 Thread Kirill Reshke
On Sat, 29 Mar 2025 at 09:47, jian he wrote: > > will use {table_beginscan, table_scan_getnextslot, table_endscan} > to output the data. > but views don't have storage, table_beginscan mechanism won't work. > > so i don't think this is possible for view. Well... So you are saying that let us have

Re: support ALTER TABLE DROP EXPRESSION for virtual generated column

2025-03-29 Thread jian he
On Thu, Mar 27, 2025 at 11:44 AM David G. Johnston wrote: > > On Wednesday, March 26, 2025, Tom Lane wrote: >> >> jian he writes: >> > the attached patch is to implement $subject. >> >> Why would this be a good idea? I don't see any principled fallback >> definition of the column. (No, "NULL"

Re: AIO v2.5

2025-03-29 Thread Noah Misch
On Fri, Mar 28, 2025 at 11:35:23PM -0400, Andres Freund wrote: > The number of combinations is annoyingly large. It's e.g. plausible to use > ignore_checksum_failure=on and zero_damaged_pages=on at the same time for > recovery. That's intricate indeed. > But I finally got to a point where the cod

Re: Thread-safe nl_langinfo() and localeconv()

2025-03-29 Thread Andres Freund
Hi, On 2025-03-28 09:13:54 +0100, Peter Eisentraut wrote: > On 27.03.25 11:16, Peter Eisentraut wrote: > > > Patch 0002 also looks okay, except that the error handling could be > > > unified with existing code, as I had previously pointed out.  Patch > > > 0005 fixes that. > > > > I plan to commi

Re: SQLFunctionCache and generic plans

2025-03-29 Thread Tom Lane
Alexander Pyhalov writes: > After writing some comments, looking at it once again, I've found that > one assumption is wrong - function can be discarded from cache during > its execution. Yeah. You really need a use-count on the shared cache object. I've been working on pulling out plpgsql's

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-03-29 Thread Ryo Kanbayashi
On Sat, Mar 29, 2025 at 3:35 PM Ryo Kanbayashi wrote: > On Fri, Mar 28, 2025 at 8:57 AM Michael Paquier wrote: > > > I am not sure that I'll have the time to look at 0002 for this release > > > cycle, could it be possible to get a rebase for it? > > Here is a simple rebase that I have been able t

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2025-03-29 Thread Tatsuo Ishii
> Hi, > Patch applies and compiles, all included tests passed and after the latest > fixes for non-nulls array, performance is near to lead/lag without support > of "ignore nulls". > I have been using the last version for more than one month in a production > environment with real data and didn't f

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-03-29 Thread Alexander Korotkov
On Wed, Mar 12, 2025 at 8:11 PM Alena Rybakina wrote: > On 06.03.2025 11:23, Alexander Korotkov wrote: > > Hi, Alena! > > On Sat, Mar 1, 2025 at 1:39 PM Alena Rybakina > wrote: > > On 09.02.2025 18:38, Alexander Korotkov wrote: > > Also, aren't we too restrictive while requiring is_simple_values

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-03-29 Thread Ashutosh Bapat
On Fri, Mar 28, 2025 at 10:46 PM Alvaro Herrera wrote: > > On 2025-Mar-28, David Rowley wrote: > > > I experimented by applying your v4 along with 0001-0003 of Yuya's v35 > > patchset from [2]. See the attached bz2 for my results run on an AMD > > Zen2 machine. The CREATE TABLE statement is in th

Re: Update Unicode data to Unicode 16.0.0

2025-03-29 Thread Robert Haas
On Tue, Mar 18, 2025 at 2:55 PM Jeff Davis wrote: > Continuing on with Unicode 15.1 and accepting the unassigned code point > *cannot* prevent breakage. Under your definition, this is true, but I think Jeremy would define breakage differently. His primary concern, I expect, is *stability*. Breaka

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2025-03-29 Thread Krasiyan Andreev
Hi, Patch applies and compiles, all included tests passed and after the latest fixes for non-nulls array, performance is near to lead/lag without support of "ignore nulls". I have been using the last version for more than one month in a production environment with real data and didn't find any bugs

High CPU consumption in cascade replication with large number of walsenders and ConditionVariable broadcast issues

2025-03-29 Thread Alexey Makhmutov
Hello, Recently we’ve faced an interesting situation on a standby environment with configured cascade replication and large number (~100) of configured walsenders. We’ve noticed a very high CPU consumption on such environment with the most time-consuming operation being signal delivery from s

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-03-29 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Fri, 28 Mar 2025 22:37:03 -0700, Masahiko Sawada wrote: >> I've added the following tests: >> >> * Wrong input type handler without namespace >> * Wrong input type handler with namespace >> * Wrong return t

Re: SQLFunctionCache and generic plans

2025-03-29 Thread Alexander Pyhalov
Alexander Pyhalov писал(а) 2025-03-28 15:22: Tom Lane писал(а) 2025-03-14 23:52: I spent some time today going through the actual code in this patch. I realized that there's no longer any point in 0001: the later patches don't move or repeatedly-call that bit of code, so it can be left as-is.

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-03-29 Thread Ryo Kanbayashi
On Fri, Mar 28, 2025 at 10:44 AM Michael Paquier wrote: > > On Thu, Mar 27, 2025 at 06:31:14PM +0900, Michael Paquier wrote: > > With all that in mind and more documentation added to the test, I've > > applied 0001, so let's see what the buildfarm has to say. The CI was > > stable, so it's a star

Re: Allow table AMs to define their own reloptions

2025-03-29 Thread Julien Tachoires
On Sun, Mar 02, 2025 at 02:23:54PM +0100, Julien Tachoires wrote: > On Sun, Mar 02, 2025 at 09:56:41AM +0100, Julien Tachoires wrote: > > With the help of the new TAM routine 'relation_options', table access > > methods can with this patch define their own reloptions > > parser/validator. > > >