Re: Using read stream in autoprewarm

2025-04-04 Thread Daniel Gustafsson
> On 3 Apr 2025, at 21:25, Melanie Plageman wrote: > > On Thu, Apr 3, 2025 at 11:17 AM Heikki Linnakangas wrote: >> >> I had a quick look at this. Looks good overall Same here, this seemed like a good piece to bite into with my limited AIO knowledge to learn more, and reading it over it seems

Re: Change log level for notifying hot standby is waiting non-overflowed snapshot

2025-04-04 Thread Yugo Nagata
On Mon, 31 Mar 2025 12:51:06 +0900 Fujii Masao wrote: > > > On 2025/03/28 0:13, torikoshia wrote: > > On 2025-03-27 11:06, torikoshia wrote: > >> Hi, > >> > >> I had another off-list discussion with Fujii-san, and according to the > >> following manual[1], it seems that a transaction with an ov

Re: RFC: Logging plan of the running query

2025-04-04 Thread torikoshia
On 2025-04-02 03:52, Robert Haas wrote: Thank you for review! On Fri, Mar 21, 2025 at 8:40 AM torikoshia wrote: Rebased it again. On 2025-03-10 14:10, torikoshia wrote: > BTW the patch adds about 400 lines to explain.c and it may be better > to split the file as well as 9173e8b6046, but I leav

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2025-04-04 Thread Alena Rybakina
On 12.03.2025 23:50, Peter Geoghegan wrote: On Wed, Mar 12, 2025 at 4:28 PM Alena Rybakina wrote: Thank you for the explanation! Now I see why these changes were made. After your additional explanations, everything really became clear and I fully agree with the current code regarding this p

Doc: Fixup misplaced filelist.sgml entities and add some commentary

2025-04-04 Thread David G. Johnston
Hi. Having been in filelist.sgml a bit recently I've noticed that the original alphabetical ordering of the entities therein hasn't been adhered to. Partly, I suspect, because there is no guidance about these files and how they are organized. The attached puts things back into alphabetical order

Re: Draft for basic NUMA observability

2025-04-04 Thread Tomas Vondra
OK, here's v25 after going through the patches once more, fixing the issues mentioned by Bertrand, etc. I think 0001 and 0002 are fine, I have a couple minor questions about 0003. 0002 - Adds the new types to typedefs.list, to make pgindent happy. - Improves comment for pg_buffercache_numa_p

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-04-04 Thread Michael Paquier
On Sat, Mar 22, 2025 at 10:43:00AM +0100, Christoph Berg wrote: > Are we at the point where the patch is already Ready for Committer? I'll think a bit more about how to document all that. Anyway, yes, I'm OK with the per-field custom_query_jumble, so let's move on with that, so I will do somethin

Re: New criteria for autovacuum

2025-04-04 Thread Konstantin Knizhnik
On 04/04/2025 7:10 pm, Melanie Plageman wrote: On Fri, Apr 4, 2025 at 1:53 AM Konstantin Knizhnik wrote: What is needed to reproduce the problem? 1. Table with populated data 2. Presence of transaction with assigned XID which prevents vacuum from marking pages of this table as all visible 3. Va

Re: Fix infinite loop from setting scram_iterations to INT_MAX

2025-04-04 Thread Michael Paquier
On Wed, Mar 26, 2025 at 03:41:10PM +0900, Richard Guo wrote: > I plan to go ahead and push Kevin's fix, barring any objections. Thanks, Richard. -- Michael signature.asc Description: PGP signature

Some problems regarding the self-join elimination code

2025-04-04 Thread Richard Guo
While working on another patch, I looked at ChangeVarNodes() and found some issues introduced by the self-join elimination commit. I think it'd better to fix or improve them. * ChangeVarNodes_walker includes special handling for RestrictInfo nodes. And it adjusts RestrictInfo.orclause only if th

Re: Use XLOG_CONTROL_FILE macro everywhere?

2025-04-04 Thread Anton A. Melnikov
Hi! On 04.04.2025 19:32, Fujii Masao wrote: I'm not sure there's clear consensus yet on the changes in the 0001 and 0002 patches, and it might not be worth rushing them in right before the feature freeze. So for now, I reviewed and updated only the 0003 patch, since there seems to be agreement o

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-04 Thread Yuya Watari
Hello David, Thank you for your prompt reply, and apologies for my late response. On Mon, Mar 24, 2025 at 2:38 PM David Rowley wrote: > > I understand Ashutosh would like to handle the RestrictInfo speedup > another way, so there's additional review work to do there to > determine the merits of

Re: Not-terribly-safe checks for CRC intrinsic support

2025-04-04 Thread Tom Lane
John Naylor writes: > On Sat, Mar 15, 2025 at 6:04 AM Tom Lane wrote: >> In short, I think we ought to apply and perhaps back-patch something >> like the attached. > Seems like reasonable defensive coding and consistency. Thanks for looking at it. > I'd be okay with keeping the original commen

Re: BTScanOpaqueData size slows down tests

2025-04-04 Thread Andres Freund
Hi, On 2025-04-03 11:26:08 +1300, David Rowley wrote: > On Thu, 3 Apr 2025 at 04:21, Andres Freund wrote: > > I was mildly > > surprised to see how expensive the new compact attribute checks are. > > Is this a fairly deform-heavy workload? It was our regression tests, although I was playing aro

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-04 Thread Jacob Champion
On Thu, Apr 3, 2025 at 12:50 PM Daniel Gustafsson wrote: > Thanks, both LGTM so pushed. Thank you! On Tue, Apr 1, 2025 at 3:40 PM Jacob Champion wrote: > Maybe a better idea would be to ship an SONAME of > `libpq-oauth.so.0.`, without any symlinks, so that there's > never any ambiguity about wh

Re: New criteria for autovacuum

2025-04-04 Thread Sami Imseih
> I think this enables us to update the VM > during on-access pruning. This is something I plan to work on in 19. > It seems like it would alleviate situations like this. IMO, index-only scans hitting the heap have always caught users off guard, especially because scan performance fluctuates betw

Re: New criteria for autovacuum

2025-04-04 Thread Konstantin Knizhnik
On 04/04/2025 10:41 pm, Melanie Plageman wrote: On Fri, Apr 4, 2025 at 3:27 PM Konstantin Knizhnik wrote: From logical point of view I agree with you: taken in account number of inserted tuples makes sense if it allows to mark page as all-visible. So `ins_since_vacuum` should be better rename

Re: RFC: Logging plan of the running query

2025-04-04 Thread Atsushi Torikoshi
On Thu, Apr 3, 2025 at 11:10 PM Robert Haas wrote: > Looking at ExplainAssembleLogOutput() is making me realize that > auto_explain is in serious need of some cleanup. That's not really the > fault of this patch, but the hack whereby we overwrite the [] that > would have surrounded the JSON output

Re: doc: Mention clock synchronization recommendation for hot_standby_feedback

2025-04-04 Thread Peter Eisentraut
On 24.03.25 10:24, Jakub Wartak wrote: On Fri, Mar 14, 2025 at 11:31 AM vignesh C wrote: On Wed, 5 Mar 2025 at 11:46, Amit Kapila wrote: On Tue, Mar 4, 2025 at 4:44 PM Jakub Wartak I can go with the last patch as you observed that in a real-world case, and we can look at others (if any) on

Re: Statistics Import and Export

2025-04-04 Thread Corey Huinker
On Fri, Apr 4, 2025 at 6:25 PM Nathan Bossart wrote: > On Fri, Apr 04, 2025 at 03:58:53PM -0500, Nathan Bossart wrote: > > I pushed commit 8ec0aae to fix this. > > And now I'm seeing cross-version test failures due to our use of WITH > ORDINALITY, which wasn't added until v9.4. Looking into it..

Re: Statistics Import and Export

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 07:32:48PM -0400, Corey Huinker wrote: > This patch shrinks the array size to 1 for versions < 9.4, which keeps the > modern code fairly elegant. Committed. -- nathan

Re: Change log level for notifying hot standby is waiting non-overflowed snapshot

2025-04-04 Thread torikoshia
On 2025-04-01 01:12, Fujii Masao wrote: On 2025/03/31 22:45, Yugo Nagata wrote: I prefer this approach clarifying that consistency and subtransaction overflow are separate concepts in the documentation. Here are minor comments on the patch: Thanks for the review! - case CAC_NOTC

Re: Proposal: Adding compression of temporary files

2025-04-04 Thread Tomas Vondra
On 3/15/25 11:40, Alexander Korotkov wrote: > On Sun, Jan 5, 2025 at 1:43 AM Filip Janus wrote: >> >> I apologize for multiple messages, but I found a small bug in the previous >> version. >> >> -Filip- > > Great, thank you for your work. > > I think the patches could use a pgindent run. >

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

2025-04-04 Thread Mahendra Singh Thalor
On Thu, 30 Jan 2025 at 16:47, Srinath Reddy wrote: > > > > On Wed, Jan 29, 2025 at 9:55 PM Mahendra Singh Thalor wrote: >> >> Hi, >> While doing some testing with pg_dumpall, I noticed one weird behaviour. >> >> While we create the database, we are allowing the database name with a new line (if n

Re: Making sslrootcert=system work on Windows psql

2025-04-04 Thread Jacob Champion
On Wed, Apr 2, 2025 at 7:15 AM George MacKerron wrote: > > But happily, I don’t think we need to choose. Can’t we just use the Windows > > system store if neither of the relevant environment variables is set? > > Thinking about this a little more, I guess the remaining concern is about > people

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Alvaro Herrera
On 2025-Apr-02, Ashutosh Bapat wrote: > I have closed the CF entry > https://commitfest.postgresql.org/patch/4564/ committed. I will > create another CF entry to park --no-statistics reversal change. That > way, we will know when statistics dump/restore has become stable. No commitfest entry ple

Re: Use XLOG_CONTROL_FILE macro everywhere?

2025-04-04 Thread Fujii Masao
On 2025/04/05 8:27, Anton A. Melnikov wrote: Hi! On 04.04.2025 19:32, Fujii Masao wrote: I'm not sure there's clear consensus yet on the changes in the 0001 and 0002 patches, and it might not be worth rushing them in right before the feature freeze. So for now, I reviewed and updated only th

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-04-04 Thread Rushabh Lathia
Hi Alvaro, Thank you for the offline discussion. As we all agree, changing the attnotnull datatype would not be a good idea since it is a commonly used catalog column, and many applications and extensions depend on it. Attached is another version of the patch (WIP), where I have introduced a new

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-04 Thread David Rowley
On Sat, 5 Apr 2025 at 04:05, Tom Lane wrote: > This patchset has a distinct whiff of unseemly haste. hmm, yes. I would like to give this patch as good a chance at making v18 as I can, and I admit to having optimised for that. Seemingly, we've got a few other good partitioning performance patches

Re: AIO v2.5

2025-04-04 Thread Andres Freund
Hi, On 2025-04-04 14:18:02 -0700, Noah Misch wrote: > On Fri, Apr 04, 2025 at 03:16:18PM -0400, Andres Freund wrote: > > > - Make DEFINED in completor before verifying page. It might be cleaner > > > to > > > do this when the completor first retrieves a return value from > > > io_uring, >

Re: general purpose array_sort

2025-04-04 Thread Junwang Zhao
On Tue, Apr 1, 2025 at 1:11 AM Tom Lane wrote: > > Junwang Zhao writes: > > On Mon, Mar 31, 2025 at 5:58 AM Tom Lane wrote: > >> In v18, it's somewhat annoying that the typcache doesn't cache > >> the typarray field; we would not need a separate get_array_type() > >> lookup if it did. I doubt t

Re: Using read stream in autoprewarm

2025-04-04 Thread Nazir Bilal Yavuz
Hi, On Fri, 4 Apr 2025 at 10:59, Daniel Gustafsson wrote: > > This patch is already marked Ready for Committer and I concur with that. Same on my end, v14 LGTM. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Improve documentation regarding custom settings, placeholders, and the administrative functions

2025-04-04 Thread Heikki Linnakangas
On 19/10/2024 23:11, David G. Johnston wrote: diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 934ef5e469..4478d0aa91 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -23,7 +23,7 @@             All parameter names are case-insensitive. Every paramet

Re: Changing the state of data checksums in a running cluster

2025-04-04 Thread Alexander Korotkov
Hi! On Sat, Mar 15, 2025 at 7:33 PM Tomas Vondra wrote: > On 3/15/25 17:26, Andres Freund wrote: > > Jo. > > > > On 2025-03-15 16:50:02 +0100, Tomas Vondra wrote: > >> Thanks, here's an updated patch version > > > > FWIW, this fails in CI; > > > > https://cirrus-ci.com/build/4678473324691456 > >

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
On Thu, Apr 3, 2025 at 10:44 PM Alvaro Herrera wrote: > > On 2025-Apr-03, Andres Freund wrote: > > > I've increased the timeout even further, but I can't say that I am happy > > about > > the slowest test getting even slower. Adding test time in the serially > > slowest > > test is way worse tha

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 4:41 PM Ashutosh Bapat wrote: > > On Thu, Apr 3, 2025 at 10:44 PM Alvaro Herrera > wrote: > > > > On 2025-Apr-03, Andres Freund wrote: > > > > > I've increased the timeout even further, but I can't say that I am happy > > > about > > > the slowest test getting even slower

002_pg_upgrade is broken for custom install

2025-04-04 Thread Ashutosh Bapat
Hi All, When oldinstall and olddump environment variables are set, 002_pg_upgrade.pl test does not finish and fails. # initializing database system by running initdb ok 1 - check locales in original cluster ok 2 - loaded old dump file # initializing database system by running initdb ok 3 # skip sou

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-04-04 Thread Alvaro Herrera
On 2025-Apr-01, Antonin Houska wrote: > Besides that, it occurred to me that 0005 ("Preserve visibility > information of the concurrent data changes.") will probably introduce > significant overhead. The problem is that the table we're repacking is > treated like a catalog, for reorderbuffer.c to

Re: Draft for basic NUMA observability

2025-04-04 Thread Jakub Wartak
On Fri, Apr 4, 2025 at 8:50 AM Bertrand Drouvot wrote: > > Hi, > > On Thu, Apr 03, 2025 at 08:53:57PM +0200, Tomas Vondra wrote: > > On 4/3/25 15:12, Jakub Wartak wrote: > > > On Thu, Apr 3, 2025 at 1:52 PM Tomas Vondra wrote: > > > > > >> ... > > >> > > >> So unless someone can demonstrate a use

Re: Some problems regarding the self-join elimination code

2025-04-04 Thread Richard Guo
On Wed, Apr 2, 2025 at 10:26 PM Richard Guo wrote: > This is not a thorough review of the code; I just randomly looked at > ChangeVarNodes(). It's possible that there are other issues that > haven't been discovered, but I think we should at least fix these. > I'll provide a patch for the fixes la

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-04-04 Thread Amit Langote
On Fri, Apr 4, 2025 at 5:48 PM Ashutosh Bapat wrote: > On Fri, Apr 4, 2025 at 2:04 PM Amit Langote wrote: > > I’ve now marked this as committed after pushing the patches earlier today. > > Thanks a lot. Thank you too for working on it. > > I realize the CF entry was originally about the project

Re: 002_pg_upgrade is broken for custom install

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 5:02 PM Álvaro Herrera wrote: > > On 2025-Apr-04, Ashutosh Bapat wrote: > > > # test failed > > stderr: > > # Tests were run but no plan was declared and done_testing() was not seen. > > # Looks like your test exited with 29 just after 4. > > > > Doesn't look like it's expec

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-04-04 Thread Rushabh Lathia
Hi Alvaro, I’ve consolidated all the changes and attached the latest version of the patch, which includes the updates submitted by Jian for pg_dump as well. Patch 0001 contains changes to MergeWithExistingConstraint to fix the marking on local constraints. Patch 0002 includes support for NOT NULL

Re: Using read stream in autoprewarm

2025-04-04 Thread Daniel Gustafsson
> On 3 Apr 2025, at 22:54, Melanie Plageman wrote: > On Thu, Apr 3, 2025 at 4:22 PM Daniel Gustafsson wrote: >> + while (p->pos < apw_state->prewarm_stop_idx) >> + { >> + BlockInfoRecord blk = p->block_info[p->pos]; >> + >> + CHECK_FOR_INTERRUPTS(); >> Isn't checking inside this

Re: Extend ALTER DEFAULT PRIVILEGES for large objects

2025-04-04 Thread Fujii Masao
On 2025/04/04 0:21, Fujii Masao wrote: Thanks for updating the patch! If there are no objections, I'll proceed with committing it using the following commit log. I've pushed the patch. Thanks! While testing the feature, I noticed that psql doesn't complete "ALTER DEFAULT PRIVILEGES GRANT/R

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-04-04 Thread Antonin Houska
Alvaro Herrera wrote: > On 2025-Apr-01, Antonin Houska wrote: > > > Besides that, it occurred to me that 0005 ("Preserve visibility > > information of the concurrent data changes.") will probably introduce > > significant overhead. The problem is that the table we're repacking is > > treated lik

Re: rename pg_log_standby_snapshot

2025-04-04 Thread Amit Kapila
On Thu, Apr 3, 2025 at 9:30 PM Sami Imseih wrote: > > While looking at [1] which introduces a new function called pg_log_query_plan > to > write an explain plan to the log file, I noticed that we currently > have overloaded > the meaning of the "pg_log_" prefix. > > Currently there is pg_log_back

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-04-04 Thread Amit Langote
On Thu, Apr 3, 2025 at 12:28 PM Amit Langote wrote: > On Wed, Apr 2, 2025 at 9:52 PM Ashutosh Bapat > wrote: > > On Wed, Apr 2, 2025 at 1:00 PM Amit Langote wrote: > > > I'm feeling good about this version, but let me know if you have any > > > further thoughts / comments. > > > > Thanks for inc

Re: Removing unneeded self joins

2025-04-04 Thread Andrei Lepikhov
On 4/4/25 04:53, Richard Guo wrote: On Fri, Apr 4, 2025 at 1:02 AM Alexander Korotkov wrote: I've got an off-list bug report from Alexander Lakhin involving a placeholder variable. Alena and Andrei proposed a fix. It is fairly simple: we just shouldn't remove PHVs during self-join elimination

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-04-04 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 31 Mar 2025 12:35:23 -0700, Masahiko Sawada wrote: > Most of the queries under test_copy_format/sql verifies the input > patterns of the FORMAT option. I find that the regression tests > included in th

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-04 Thread Amit Langote
Hi David, Impressive results! On Fri, Apr 4, 2025 at 3:05 PM David Rowley wrote: > I've done some further work on this, mostly relating to the code > comments. It looks to me like the following hunks in 0002 probably belong in 0001, unless you’re planning to commit the patches together anyway:

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-04-04 Thread Peter Smith
Hi Nisha, I saw this patch was already pushed [1], but there was one thing I never quite understood about this feature, and I didn't find the answer in the thread posts above. My question: Why is there only a single new conflict type being added here? e.g. Conflict due to INSERT - single confli

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Alena Rybakina
On 02.04.2025 22:00, Alexander Korotkov wrote: Hi, Alena! On Wed, Apr 2, 2025 at 5:33 PM Alena Rybakina wrote: I prepared a patch according to my suggestions, it just checks that the transformation is not carried out if there is a var element, there are changes only in one test,

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-04-04 Thread jian he
On Thu, Mar 20, 2025 at 11:53 PM Alvaro Herrera wrote: > > On 2025-Mar-20, jian he wrote: > > > > Is it expected that a child may have VALID constraint but parent has > > > not valid constraint? > > > > but the MergeConstraintsIntoExisting logic is when > > ALTER TABLE ATTACH PARTITION, > > it exp

Re: 002_pg_upgrade is broken for custom install

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 5:23 PM Álvaro Herrera wrote: > > On 2025-Apr-04, Ashutosh Bapat wrote: > > > connection error: 'psql: error: connection to server on socket > > "/tmp/LiPa_UJpSb/.s.PGSQL.13779" failed: FATAL: database "regression" > > does not exist' > > while running '/home/ashutosh/work/u

Re: Changing shared_buffers without restart

2025-04-04 Thread Ni Ku
Thanks for your insights and confirmation, Dmitry. Right, I think the anonymous fd approach would work to keep the memory contents intact in between munmap and mmap with the new size, so bufferpool expansion would work. But it seems shrinking would still be problematic, since that approach requires

Re: speedup COPY TO for partitioned table.

2025-04-04 Thread Kirill Reshke
Hi! First of all, a commit message does not need to contain SQL examples of what it does. We should provide human-readable explanations and that's it. Next, about changes to src/test/regress/sql/copy2.sql. I find the sql you used to test really unintuitive. How about CREATE TABLE ... PARTITION OF

Re: Improvement of var_eq_non_const()

2025-04-04 Thread Ilia Evdokimov
On 20.02.2025 21:21, Tom Lane wrote: Teodor Sigaev writes: I'd like to suggest to improve var_eq_non_const() by using knowledge of MCV and estimate the selectivity as quadratic mean of non-null fraction divided by number of distinct values (as it was before) and set of MCV selectivities. Wha

autoprewarm_dump_now

2025-04-04 Thread Дарья Шанина
Hello everyone! I have a question. What would be better for the function autoprewarm_dump_now in case when we need to allocate memory that exceeds 1 GB: 1) allocate enough memory for the entire shared_buffer array (1..NBuffers) using palloc_extended; 2) allocate the maximum of currently possible m

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-04-04 Thread Ashutosh Bapat
On Fri, Apr 4, 2025 at 2:04 PM Amit Langote wrote: > > On Thu, Apr 3, 2025 at 12:28 PM Amit Langote wrote: > > On Wed, Apr 2, 2025 at 9:52 PM Ashutosh Bapat > > wrote: > > > On Wed, Apr 2, 2025 at 1:00 PM Amit Langote > > > wrote: > > > > I'm feeling good about this version, but let me know if

Re: Memoize ANTI and SEMI JOIN inner

2025-04-04 Thread Andrei Lepikhov
On 4/1/25 09:18, Richard Guo wrote: On Mon, Mar 31, 2025 at 7:33 PM Andrei Lepikhov wrote: and I don't get the case. As I see, ANTI/SEMI join just transforms to the regular join and it is still not the case. May you be more specific? Upthread, you said that a qual contained in ppi_clauses wil

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-04 Thread David Rowley
Thank you for having a look at this. On Fri, 4 Apr 2025 at 21:47, Amit Langote wrote: > It looks to me like the following hunks in 0002 probably belong in > 0001, unless you’re planning to commit the patches together anyway: Ah, yeah. Unsure about that as yet, but I've moved it over. > The comm

Re: pg_upgrade: Support for upgrading to checksums enabled

2025-04-04 Thread Robert Treat
On Thu, Apr 3, 2025 at 3:25 AM Peter Eisentraut wrote: > > On 11.03.25 11:42, Peter Eisentraut wrote: > > Here is an updated patch that works more along those lines. It adds a > > pg_upgrade option --update-checksums, which activates the code to > > rewrite the checksums. You must specify this o

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Andres Freund
Hi, On 2025-04-03 19:14:02 +0200, Alvaro Herrera wrote: > On 2025-Apr-03, Andres Freund wrote: > > > I've increased the timeout even further, but I can't say that I am happy > > about > > the slowest test getting even slower. Adding test time in the serially > > slowest > > test is way worse tha

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Tom Lane
Peter Geoghegan writes: > Is somebody going to commit this soon? Alexander? Done. I thought the point of the test case was to check VALUES folding on both sides of the join. I'm not sure why that's actually a useful expenditure of test cycles, but Alena's patch undid that without adjusting the

Re: Network failure may prevent promotion

2025-04-04 Thread Heikki Linnakangas
On 21/03/2025 15:24, Yura Sokolov wrote: I've just rebased patches and merged last two fix-commits (0003 and 0004) into 0002. Thanks for reviving this! I looked over this again and committed. On May 14, 2024 at 14:16 Robert Haas wrote: I think it would be helpful to make it more clear exactl

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Peter Geoghegan
On Fri, Apr 4, 2025 at 12:00 PM Tom Lane wrote: > Peter Geoghegan writes: > > Is somebody going to commit this soon? Alexander? > > Done. Thanks! > If it's still not stable I think the next step is to nuke both > test queries, since I remain of the opinion that they're likely > a waste of cycle

Re: Fwd: [BUG]: the walsender does not update its IO statistics until it exits

2025-04-04 Thread vignesh C
On Fri, 4 Apr 2025 at 10:31, Bertrand Drouvot wrote: > > Hi, > > On Thu, Apr 03, 2025 at 03:23:31PM +0530, vignesh C wrote: > > Can we add it to one of the subscription tests, such as 001_rep_changes.pl? > > Yeah that sounds like a good place for it. Done in the attached. The new test added curre

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Alexander Korotkov
On Fri, Apr 4, 2025 at 7:00 PM Tom Lane wrote: > Peter Geoghegan writes: > > Is somebody going to commit this soon? Alexander? > > Done. > > I thought the point of the test case was to check VALUES folding > on both sides of the join. I'm not sure why that's actually > a useful expenditure of te

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Andres Freund
Hi, On 2025-04-04 12:01:16 -0400, Andres Freund wrote: > FWIW, for me 027 is actually considerably faster. In an cassert -O0 build (my > normal development env, I find even -Og too problematic for debugging): > > pg_upgrade/002_pg_upgrade96.61s > recovery/027_stream_regress 66.04s > > After

Re: Index AM API cleanup

2025-04-04 Thread Andres Freund
Hi,, skink/valgrind just started to die during the main regression tests: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2025-04-04%2011%3A01%3A56 The set of commits seem to point to the changes made as part of this thread. ==4163809== VALGRINDERROR-BEGIN ==4163809== Invalid re

Re: New criteria for autovacuum

2025-04-04 Thread Melanie Plageman
On Fri, Apr 4, 2025 at 1:53 AM Konstantin Knizhnik wrote: > > What is needed to reproduce the problem? > 1. Table with populated data > 2. Presence of transaction with assigned XID which prevents vacuum from > marking pages of this table as all visible > 3. Vacuum or autovacuum processed this tabl

Re: Quote-less file names in error messages

2025-04-04 Thread Daniel Gustafsson
> On 4 Apr 2025, at 05:03, Kyotaro Horiguchi wrote: > > Hello, > > The recent commit 2da74d8d640 added the following new messages: > > + libpq_append_conn_error(conn, "could not open ssl keylog file %s: %s", > + libpq_append_conn_error(conn, "could not write to ssl keylog file %s: %s > > Howev

Re: Improve error reporting for few options in pg_createsubscriber

2025-04-04 Thread Amit Kapila
On Fri, Apr 4, 2025 at 10:11 AM vignesh C wrote: > > On Fri, 4 Apr 2025 at 09:36, Amit Kapila wrote: > > > > The attached v2 version patch has the changes for the same. > Pushed. -- With Regards, Amit Kapila.

Re: Better HINT message for "unexpected data beyond EOF"

2025-04-04 Thread Jakub Wartak
On Tue, Apr 1, 2025 at 3:59 PM Andres Freund wrote: Hi Robert, Andres, Christoph, > On 2025-04-01 09:49:12 -0400, Robert Haas wrote: > > On Tue, Apr 1, 2025 at 7:13 AM Jakub Wartak > > wrote: > > > Thread bump. So we have the following candidates: > > > > > > 1. remove it as Andres stated: > >

Re: rename pg_log_standby_snapshot

2025-04-04 Thread Bertrand Drouvot
Hi, On Fri, Apr 04, 2025 at 04:10:19PM +0530, Amit Kapila wrote: > On Thu, Apr 3, 2025 at 9:30 PM Sami Imseih wrote: > > > > While looking at [1] which introduces a new function called > > pg_log_query_plan to > > write an explain plan to the log file, I noticed that we currently > > have overlo

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2025-04-04 Thread jian he
On Tue, Mar 25, 2025 at 2:31 PM vignesh C wrote: > > 2) Here in error we say column c1 violates not-null constraint and in > the context we show column c2, should the context also display c2 > column: > postgres=# create table t3(c1 int not null, c2 int, check (c1 > 10)); > CREATE TABLE > postgres

Re: Thread-safe nl_langinfo() and localeconv()

2025-04-04 Thread Peter Eisentraut
On 31.03.25 15:52, Tom Lane wrote: Peter Eisentraut writes: I'm not sure what to do with this. If setlocale() and newlocale() indeed behave differently in what set of locale names they accept, then technically we ought to test both of them, since we do use both of them later on. Or maybe we p

Re: Index AM API cleanup

2025-04-04 Thread Peter Eisentraut
On 04.04.25 14:17, Andres Freund wrote: skink/valgrind just started to die during the main regression tests: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2025-04-04%2011%3A01%3A56 Thanks, fix is on the way. The set of commits seem to point to the changes made as part of th

RE: doc patch: wrong descriptions for dropping replication slots

2025-04-04 Thread Hayato Kuroda (Fujitsu)
Dear Fujii-san, > Unless there are any objections, I plan to push your patch with > the following commit message and back-patch it to all supported versions. ... Thanks for updating the commit message. LGTM. Best regards, Hayato Kuroda FUJITSU LIMITED

Re: Reduce "Var IS [NOT] NULL" quals during constant folding

2025-04-04 Thread Robert Haas
On Tue, Apr 1, 2025 at 2:34 AM Richard Guo wrote: > However, I gave up this idea because I realized it would require > retrieving a whole bundle of catalog information that isn't needed > until after the RelOptInfos are built, such as max_attr, pages, > tuples, reltablespace, parallel_workers, ext

Re: AIO v2.5

2025-04-04 Thread Andres Freund
Hi, On 2025-04-01 09:07:27 -0700, Noah Misch wrote: > On Tue, Apr 01, 2025 at 11:55:20AM -0400, Andres Freund wrote: > > WRT the locking issues, I've been wondering whether we could make > > LWLockWaitForVar() work that purpose, but I doubt it's the right approach. > > Probably better to get rid o

Re: in BeginCopyTo make materialized view using COPY TO instead of COPY (query).

2025-04-04 Thread Fujii Masao
On 2025/04/03 20:46, Fujii Masao wrote: Thanks for updating the patch! I made some minor cosmetic changes and updated the commit log. The revised patch is attached. Unless there are any objections, I'll proceed with committing it. I've pushed the patch. Thanks! As a follow-up, it might be

Re: Test to dump and restore objects left behind by regression

2025-04-04 Thread Ashutosh Bapat
On Thu, Apr 3, 2025 at 1:50 PM Alvaro Herrera wrote: > > On 2025-Apr-03, Ashutosh Bapat wrote: > > > Looks like the problem is in the test itself as pointed out by Jeff in > > [1]. PFA patch fixing the test and enabling statistics back. > > Thanks, pushed. Thanks. -- Best Wishes, Ashutosh Bapat

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Alena Rybakina
On 04.04.2025 19:00, Tom Lane wrote: Peter Geoghegan writes: Is somebody going to commit this soon? Alexander? Done. I thought the point of the test case was to check VALUES folding on both sides of the join. I'm not sure why that's actually a useful expenditure of test cycles, but Alena's p

Re: Proposal: Progressive explain

2025-04-04 Thread torikoshia
On Fri, Mar 7, 2025 at 6:43 AM Rafael Thofehrn Castro wrote: The wrapper code was implemented by torikoshia (torikoshia(at)oss(dot)nttdata(dot)com), so adding the credits here. On Thu, Mar 20, 2025 at 5:35 AM Robert Haas wrote: Without having the prior discussion near to hand, I *think* th

Re: Using read stream in autoprewarm

2025-04-04 Thread Nazir Bilal Yavuz
Hi, On Tue, 1 Apr 2025 at 05:14, Melanie Plageman wrote: > > On Mon, Mar 31, 2025 at 3:45 PM Melanie Plageman > wrote: > > > > Whoops, this isn't right. It does work. I'm going to draft a version > > suggesting slightly different variable naming and a couple comments to > > make this more clear.

Re: Draft for basic NUMA observability

2025-04-04 Thread Tomas Vondra
On 4/4/25 09:35, Jakub Wartak wrote: > On Fri, Apr 4, 2025 at 8:50 AM Bertrand Drouvot > wrote: >> >> Hi, >> >> On Thu, Apr 03, 2025 at 08:53:57PM +0200, Tomas Vondra wrote: >>> On 4/3/25 15:12, Jakub Wartak wrote: On Thu, Apr 3, 2025 at 1:52 PM Tomas Vondra wrote: > ... > >>

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Andres Freund
On 2025-04-03 18:26:48 +0300, Alexander Korotkov wrote: > On Thu, Apr 3, 2025 at 5:18 PM Alena Rybakina > wrote: > > Okay, I agree with you. > > Good. I've reflected this limitation in comments and the commit > message. Also, I've adjust regression tests by removing excessive > ones and adding

Re: Extend ALTER DEFAULT PRIVILEGES for large objects

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 07:18:11PM +0900, Fujii Masao wrote: > I've pushed the patch. Thanks! Just a heads up, I fixed a pgindent issue in this commit (see commits e1a8b1ad58 and 742317a80f). I'd ordinarily just report it, but since we're nearing feature freeze, I just fixed it because my workflo

Re: Draft for basic NUMA observability

2025-04-04 Thread Tomas Vondra
On 4/4/25 08:50, Bertrand Drouvot wrote: > Hi, > > On Thu, Apr 03, 2025 at 08:53:57PM +0200, Tomas Vondra wrote: >> On 4/3/25 15:12, Jakub Wartak wrote: >>> On Thu, Apr 3, 2025 at 1:52 PM Tomas Vondra wrote: >>> ... So unless someone can demonstrate a use case where this would matt

Re: Add pg_buffercache_evict_all() and pg_buffercache_mark_dirty[_all]() functions

2025-04-04 Thread Nazir Bilal Yavuz
Hi, Thanks for the review! On Wed, 2 Apr 2025 at 20:06, Andres Freund wrote: > > Hi, > > On 2025-03-31 19:49:25 +0300, Nazir Bilal Yavuz wrote: > > > After this discussion, I think it would be helpful if one of the more > > > experienced > > > hackers could take a look at the overall picture (pe

Re: Improve monitoring of shared memory allocations

2025-04-04 Thread Rahila Syed
Hi, Analysis of the Bug in 0002 reported by David Rowley : The 0001* patch allocates memory for the hash header, directory, segments, and elements collectively for both shared and non-shared hash tables. While

Re: 002_pg_upgrade is broken for custom install

2025-04-04 Thread Álvaro Herrera
On 2025-Apr-04, Ashutosh Bapat wrote: > connection error: 'psql: error: connection to server on socket > "/tmp/LiPa_UJpSb/.s.PGSQL.13779" failed: FATAL: database "regression" > does not exist' > while running '/home/ashutosh/work/units/pghead/build/dev/bin/psql > --no-psqlrc --no-align --tuples-on

Exponential notation bug

2025-04-04 Thread Marcos Pegoraro
I was deleting thousands of records each time with \gexec, then ... This one works select format('delete from table where ID = any(%L::integer[]);', array_agg(ID)), (ord-1)/1 from ( select * from generate_series(15e2,65e5) with ordinality) x(ID, ord) group by 2 order by 2; But it's easier to

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Alena Rybakina
On 04.04.2025 17:39, Andres Freund wrote: On 2025-04-03 18:26:48 +0300, Alexander Korotkov wrote: On Thu, Apr 3, 2025 at 5:18 PM Alena Rybakina wrote: Okay, I agree with you. Good. I've reflected this limitation in comments and the commit message. Also, I've adjust regression tests by remov

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Tom Lane
Andres Freund writes: > This just failed on both buildfarm and CI: Yeah, the plan for that join seems fairly unstable. Is there actually any need for the join to pg_am? regards, tom lane

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-04-04 Thread Shlok Kyal
On Thu, 20 Mar 2025 at 18:09, Ajin Cherian wrote: > > On Thu, Mar 13, 2025 at 5:49 PM Ajin Cherian wrote: > > > > Moving this patch to the next CF as this patch needs more design level > > inputs which may not be feasible in this CF but do continue to review > > the patch. > > > > regards, > > Aj

Re: psql suggestion "select " offers nothing, can we get functions like "\df "

2025-04-04 Thread Kirill Reshke
On Thu, 3 Apr 2025 at 20:07, Kirk Wolak wrote: > Worse, "select pg_stat_st" has no clue. I was looking for ... _reset > > It's not that difficult to add, I am suggesting that we use the same logic > as \df at that point? Is not so simple here, because we have to try to complete this query w

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-04 Thread David Rowley
On Sat, 5 Apr 2025 at 02:54, Ashutosh Bapat wrote: > I haven't measured if the patches improve performance of simple scans > with thousands of partitions. Have you tried measuring that? I just tried 10k partitions on my Zen4 laptop. create table lp (a int) partition by list(a); select 'create ta

  1   2   3   >