Re: making relfilenodes 56 bits

2022-07-25 Thread Dilip Kumar
On Mon, Jul 25, 2022 at 9:51 PM Ashutosh Sharma wrote: > > Hi, > > As oid and relfilenumber are linked with each other, I still see that if the > oid value reaches the threshold limit, we are unable to create a table with > storage. For example I set FirstNormalObjectId to 4294967294 (one value

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-25 Thread Amit Kapila
On Tue, Jul 26, 2022 at 7:00 AM Masahiko Sawada wrote: > > On Mon, Jul 25, 2022 at 7:57 PM shiy.f...@fujitsu.com > wrote: > > > > Hi, > > > > I did some performance test for the master branch patch (based on v6 patch) > > to > > see if the bsearch() added by this patch will cause any overhead.

Re: Cygwin cleanup

2022-07-25 Thread Thomas Munro
On Tue, Jul 26, 2022 at 4:34 PM Tom Lane wrote: > Thomas Munro writes: > > 3. You can't really run PostgreSQL on Cygwin for real, because its > > implementation of signals does not have reliable signal masking, so > > unsubtle and probably also subtle breakage occurs. That was reported > >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-25 Thread Michael Paquier
On Tue, Jul 26, 2022 at 01:04:02PM +0800, Julien Rouhaud wrote: > It doesn't have much impact most of the time. The filename is reported if > there's an IO error while reading the already opened correct file. The real > problem is if the hba_file and ident_file are stored in different directory,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-25 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 04:22:32PM +0900, Michael Paquier wrote: > On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > > See the attached, for reference, but it would fail with EXEC_BACKEND > > on WIN32. > > Ditto. While working on the full regression test coverage for the

Re: Introduce wait_for_subscription_sync for TAP tests

2022-07-25 Thread Amit Kapila
On Tue, Jul 26, 2022 at 7:07 AM Masahiko Sawada wrote: > > Hi, > > In tap tests for logical replication, we have the following code in many > places: > > $node_publisher->wait_for_catchup('tap_sub'); > my $synced_query = > "SELECT count(1) = 0 FROM pg_subscription_rel WHERE srsubstate NOT > IN

Re: fairywren hung in pg_basebackup tests

2022-07-25 Thread Noah Misch
On Mon, Jul 25, 2022 at 11:35:12AM -0400, Tom Lane wrote: > Noah Misch writes: > > On Mon, Jul 25, 2022 at 09:44:21AM -0400, Andrew Dunstan wrote: > >> Perhaps we should have a guard in system_or_bail() and/or system_log() > >> which bails if some element of @_ is undefined. > > +1, seeing how

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Peter Smith
On Tue, Jul 26, 2022 at 2:09 PM Amit Kapila wrote: > > On Tue, Jul 26, 2022 at 5:04 AM Peter Smith wrote: > > > > On Mon, Jul 25, 2022 at 7:33 PM vignesh C wrote: > > > > > > On Mon, Jul 25, 2022 at 12:58 PM Peter Smith > > > wrote: > > > > > > > > Firstly, I have some (case-sensitivity)

Re: making relfilenodes 56 bits

2022-07-25 Thread Amul Sul
On Fri, Jul 22, 2022 at 4:21 PM vignesh C wrote: > > On Wed, Jul 20, 2022 at 4:57 PM Dilip Kumar wrote: > > > > On Mon, Jul 18, 2022 at 4:51 PM Dilip Kumar wrote: > > > > > > I was doing some more testing by setting the FirstNormalRelFileNumber > > > to a high value(more than 32 bits) I have

Re: Cygwin cleanup

2022-07-25 Thread Tom Lane
Thomas Munro writes: > 3. You can't really run PostgreSQL on Cygwin for real, because its > implementation of signals does not have reliable signal masking, so > unsubtle and probably also subtle breakage occurs. That was reported > upstream by Noah years ago, but they aren't working on a fix.

Cygwin cleanup

2022-07-25 Thread Thomas Munro
Hi, Continuing a discussion started over at [1]. Moving this to a new thread so that other thread can focus on Unix cleanup, and both threads can get CI coverage... 1. In a few places, it is alleged that both __CYGWIN__ and WIN32 might be defined at the same time. Do you think we should try

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Amit Kapila
On Tue, Jul 26, 2022 at 5:04 AM Peter Smith wrote: > > On Mon, Jul 25, 2022 at 7:33 PM vignesh C wrote: > > > > On Mon, Jul 25, 2022 at 12:58 PM Peter Smith wrote: > > > > > > Firstly, I have some (case-sensitivity) questions about the previous > > > patch which was already pushed [1]. > > > >

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Amit Kapila
On Tue, Jul 26, 2022 at 7:48 AM Peter Smith wrote: > > On Tue, Jul 26, 2022 at 11:43 AM Jonathan S. Katz > wrote: > > > ... > > That said, this introduces a new restriction for this particular > > scenario that doesn't exist on other scenarios. Instead, I would > > advocate we document how to

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Amit Kapila
On Tue, Jul 26, 2022 at 7:13 AM Jonathan S. Katz wrote: > > On 7/25/22 4:54 AM, vignesh C wrote: > > > > Let's take a simple case to understand why copy_data = force is > > required to replicate between two primaries for table t1 which has > > data as given below: > > > step4 - Node-2: > > Create

Re: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?

2022-07-25 Thread Kyotaro Horiguchi
At Mon, 25 Jul 2022 14:21:38 +0530, Bharath Rupireddy wrote in > Hm. I think we must take this opportunity to clean it up. You are > right, we don't need to parse the label file contents (just like we > used to do previously after reading it from the file) in > do_pg_backup_stop(), instead we

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Peter Smith
On Tue, Jul 26, 2022 at 11:43 AM Jonathan S. Katz wrote: > ... > That said, this introduces a new restriction for this particular > scenario that doesn't exist on other scenarios. Instead, I would > advocate we document how to correctly set up the two-way replication > scenario (which we have a

Re: Fix annotations nextFullXid

2022-07-25 Thread Fujii Masao
On 2022/07/23 14:01, Zhang Mingli wrote: Hi, VariableCacheData.nextFullXid is renamed to nextXid in commit https://github.com/postgres/postgres//commit/fea10a64340e529805609126740a540c8f9daab4 Fix the 

Re: Remove useless arguments in ReadCheckpointRecord().

2022-07-25 Thread Fujii Masao
On 2022/07/26 9:42, Kyotaro Horiguchi wrote: At Sun, 24 Jul 2022 22:40:16 -0400, Tom Lane wrote in Fujii Masao writes: On 2022/07/22 17:31, Kyotaro Horiguchi wrote: I believed that it is recommended to move to the style not having the outmost parens. That style has been introduced by

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-25 Thread Kyotaro Horiguchi
At Tue, 26 Jul 2022 10:47:12 +0900 (JST), Kyotaro Horiguchi wrote in > WARNING: skipping "pg_statistic" --- only superusers, roles with privileges > of pg_vacuum_analyze, or the database owner can vacuum it > WARNING: skipping "pg_type" --- only superusers, roles with privileges of >

Re: optimize lookups in snapshot [sub]xip arrays

2022-07-25 Thread Zhang Mingli
> On Jul 26, 2022, at 03:04, Nathan Bossart wrote: >> > From the discussion thus far, it seems there is interest in optimizing > [sub]xip lookups, so I'd like to spend some time moving it forward. I > think the biggest open question is which approach to take. Both the SIMD > and hash table

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-25 Thread Kyotaro Horiguchi
At Mon, 25 Jul 2022 09:40:49 -0700, Nathan Bossart wrote in > On Mon, Jul 25, 2022 at 12:58:36PM +0530, Bharath Rupireddy wrote: > > Thanks. I'm personally happy with more granular levels of control (as > > we don't have to give full superuser access to just run a few commands > > or

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Peter Smith
Here are some review comments for the patch v38-0002: == - terminology There seemed to be an inconsistent alternation of the terms "primaries" and "nodes"... For example "Setting replication between two primaries" versus "Adding a new node..." (instead of "Adding a new primary..."?). I

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Jonathan S. Katz
On 7/25/22 4:54 AM, vignesh C wrote: On Sun, Jul 24, 2022 at 10:21 PM Jonathan S. Katz wrote: On 7/22/22 12:47 AM, Amit Kapila wrote: On Fri, Jul 22, 2022 at 1:39 AM Jonathan S. Katz wrote: BTW, do you have any opinion on the idea of the first remaining patch where we accomplish two

Introduce wait_for_subscription_sync for TAP tests

2022-07-25 Thread Masahiko Sawada
Hi, In tap tests for logical replication, we have the following code in many places: $node_publisher->wait_for_catchup('tap_sub'); my $synced_query = "SELECT count(1) = 0 FROM pg_subscription_rel WHERE srsubstate NOT IN ('r', 's');"; $node_subscriber->poll_query_until('postgres',

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-25 Thread Masahiko Sawada
On Mon, Jul 25, 2022 at 7:57 PM shiy.f...@fujitsu.com wrote: > > Hi, > > I did some performance test for the master branch patch (based on v6 patch) to > see if the bsearch() added by this patch will cause any overhead. Thank you for doing performance tests! > > I tested them three times and

Re: Make name optional in CREATE STATISTICS

2022-07-25 Thread Michael Paquier
On Mon, Jul 25, 2022 at 12:55:54PM +0200, Alvaro Herrera wrote: > Agreed. I think you already have the query for that elsewhere in the > test, so it's just a matter of copying it from there. I actually already wrote most of it in 2cbc3c1, and I just needed to extend things a bit to detect the

Re: log_line_prefix: make it possible to add the search_path

2022-07-25 Thread Lukas Fittl
On Mon, Jul 25, 2022 at 12:38 AM Pierre Ducroquet wrote: > usecase by not showing the schema, one of them being log_line_prefix. > It is possible to work around this using the application_name, but a > mistake > on the application side would be fatal, while the search_path would still > indicate

Re: Remove useless arguments in ReadCheckpointRecord().

2022-07-25 Thread Kyotaro Horiguchi
At Sun, 24 Jul 2022 22:40:16 -0400, Tom Lane wrote in > Fujii Masao writes: > > On 2022/07/22 17:31, Kyotaro Horiguchi wrote: > >> I believed that it is recommended to move to the style not having the > >> outmost parens. That style has been introduced by e3a87b4991. > > > I read the commit

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-07-25 Thread Zhihong Yu
On Mon, Jul 25, 2022 at 4:39 PM David Rowley wrote: > On Fri, 22 Jul 2022 at 21:33, Richard Guo wrote: > > I can see this problem with > > the query below: > > > > select max(b order by b), max(a order by a) from t group by a; > > > > When processing the first aggregate, we compose the

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-07-25 Thread David Rowley
On Fri, 22 Jul 2022 at 21:33, Richard Guo wrote: > I can see this problem with > the query below: > > select max(b order by b), max(a order by a) from t group by a; > > When processing the first aggregate, we compose the 'currpathkeys' as > {a, b} and mark this aggregate in 'aggindexes'. When

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Peter Smith
On Mon, Jul 25, 2022 at 7:33 PM vignesh C wrote: > > On Mon, Jul 25, 2022 at 12:58 PM Peter Smith wrote: > > > > Firstly, I have some (case-sensitivity) questions about the previous > > patch which was already pushed [1]. > > > > Q1. create_subscription docs > > > > I did not understand why the

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Peter Smith
On Mon, Jul 25, 2022 at 6:54 PM Amit Kapila wrote: > > On Mon, Jul 25, 2022 at 12:58 PM Peter Smith wrote: > > ... > > > > Q2. parse_subscription_options > > > > Similarly, in the code (parse_subscription_options), I did not > > understand why the checks for special name values are implemented >

Re: Proposal to provide the facility to set binary format output for specific OID's per session

2022-07-25 Thread Dave Cramer
Hi Sehrope, On Mon, 25 Jul 2022 at 17:22, Sehrope Sarkuni wrote: > Idea here makes sense and I've seen this brought up repeatedly on the JDBC > lists. > > Does the driver need to be aware that this SET command was executed? I'm > wondering what happens if an end user executes this with an OID

Re: Proposal to provide the facility to set binary format output for specific OID's per session

2022-07-25 Thread Sehrope Sarkuni
Idea here makes sense and I've seen this brought up repeatedly on the JDBC lists. Does the driver need to be aware that this SET command was executed? I'm wondering what happens if an end user executes this with an OID the driver does not actually know how to handle. > + Oid *tmpOids =

Re: Pluggable toaster

2022-07-25 Thread Nikita Malakhov
Hi hackers! Here is the patch set rebased to master from 22.07. I've got some trouble rebasing it due to conflicts, so it took some time. I've made some corrections according to Matthias and Aleksander comments, though not all of them, because some require refactoring of very old code and would

Re: log_line_prefix: make it possible to add the search_path

2022-07-25 Thread Tom Lane
Alvaro Herrera writes: > Maybe your idea of allowing arbitrary GUCs is not a bad one, something > like > %{search_path}G > (where we add a letter at the end just so we can add other things in the > future that aren't GUCs.) I'm pretty uncomfortable about the amount of code that could

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-25 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > Tom Lane writes: >> I wonder if it'd be a good idea to convert >> auto_explain's TAP test to load auto_explain via session_preload_libraries >> instead of shared_preload_libraries, and then pass in the settings for >> each test via PGOPTIONS

Re: Expose Parallelism counters planned/execute in pg_stat_statements

2022-07-25 Thread Anthony Sotolongo
On 23-07-22 00:03, Julien Rouhaud wrote: Hi, On Fri, Jul 22, 2022 at 02:11:35PM -0400, Anthony Sotolongo wrote: On 22-07-22 12:08, Julien Rouhaud wrote: With your current patch it only says if the plan and execution had parallelism enabled, but not if it could actually use with parallelism at

Re: optimize lookups in snapshot [sub]xip arrays

2022-07-25 Thread Nathan Bossart
On Sat, Jul 16, 2022 at 08:59:57PM -0700, Nathan Bossart wrote: > On Fri, Jul 15, 2022 at 01:08:57PM -0700, Andres Freund wrote: >> I wonder if we additionally / alternatively could use a faster method of >> searching the array linearly, e.g. using SIMD. > > I looked into using SIMD. The patch

Re: very long record lines in expanded psql output

2022-07-25 Thread Andrew Dunstan
On 2021-10-03 Su 16:03, Platon Pronko wrote: >> Hi, >> >> +   pg_log_error("\\pset: allowed xheader_width values >> are full >> (default), column, page, or an number specifying exact width."); >> >>   an number specifying -> a number specifying >> >> Cheers >> > > Fixed, attaching

Re: Proposal to provide the facility to set binary format output for specific OID's per session

2022-07-25 Thread Joe Conway
On 7/25/22 10:07, Jack Christensen wrote: The advantage is to be able to use the binary format with only a single network round trip in cases where prepared statements are not possible. e.g. when using PgBouncer. Using the simple protocol with this patch lets users of pgx (the Go driver

Re: Allow pageinspect's bt_page_stats function to return a set of rows instead of a single row

2022-07-25 Thread Hamid Akhtar
On Fri, 1 Jul 2022 at 13:01, Drouvot, Bertrand wrote: > Hi, > On 6/30/22 10:24 AM, Hamid Akhtar wrote: > > On Mon, 27 Jun 2022 at 15:52, Bharath Rupireddy < > bharath.rupireddyforpostg...@gmail.com> wrote: > >> On Mon, Jun 27, 2022 at 1:40 PM Drouvot, Bertrand >> wrote: >> > >> > Hi, >> > >> >

Re: explain analyze rows=%.0f

2022-07-25 Thread Hamid Akhtar
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: not tested Documentation:not tested LGTM

Re: predefined role(s) for VACUUM and ANALYZE

2022-07-25 Thread Nathan Bossart
On Mon, Jul 25, 2022 at 12:58:36PM +0530, Bharath Rupireddy wrote: > Thanks. I'm personally happy with more granular levels of control (as > we don't have to give full superuser access to just run a few commands > or maintenance operations) for various postgres commands. The only > concern is that

Re: making relfilenodes 56 bits

2022-07-25 Thread Ashutosh Sharma
Hi, As oid and relfilenumber are linked with each other, I still see that if the oid value reaches the threshold limit, we are unable to create a table with storage. For example I set FirstNormalObjectId to 4294967294 (one value less than the range limit of 2^32 -1 = 4294967295). Now when I try

Re: Pre-allocating WAL files

2022-07-25 Thread Nathan Bossart
On Thu, Jul 14, 2022 at 11:34:07AM -0700, Nathan Bossart wrote: > It's now been over a year since I first posted a patch in this thread, and > I still sense very little interest for this feature. I intend to mark it > as Withdrawn at the end of this commitfest. Done. -- Nathan Bossart Amazon

Re: allow specifying action when standby encounters incompatible parameter settings

2022-07-25 Thread Nathan Bossart
On Mon, Jul 18, 2022 at 03:17:10PM -0700, Nathan Bossart wrote: > Given this feedback, I intend to mark the associated commitfest entry as > Withdrawn at the conclusion of the current commitfest. Done. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: log_line_prefix: make it possible to add the search_path

2022-07-25 Thread Alvaro Herrera
On 2022-Jul-25, Pierre wrote: > On Monday, July 25, 2022 11:52:41 AM CEST Alvaro Herrera wrote: > > It seems that this would be too noisy to be truly usable. What if we > > emitted a log line when the variable changed, and the value that's in > > use when the connection starts? > > Then the

Re: fairywren hung in pg_basebackup tests

2022-07-25 Thread Andrew Dunstan
On 2022-07-25 Mo 11:24, Thomas Munro wrote: > On Tue, Jul 26, 2022 at 3:08 AM Tom Lane wrote: >> I wrote: >>> Right, so the "glob" failed to find anything. Seeing that this test >>> is new as of 534472375, which postdates fairywren's last successful >>> run, I'd guess that the "glob" needs

Refactoring postgres_fdw/connection.c

2022-07-25 Thread Fujii Masao
Hi, When reviewing the postgres_fdw parallel-abort patch [1], I found that there are several duplicate codes in postgres_fdw/connection.c. Which seems to make it harder to review the patch changing connection.c. So I'd like to remove such duplicate codes and refactor the functions in

Re: fairywren hung in pg_basebackup tests

2022-07-25 Thread Tom Lane
Noah Misch writes: > On Mon, Jul 25, 2022 at 09:44:21AM -0400, Andrew Dunstan wrote: >> Perhaps we should have a guard in system_or_bail() and/or system_log() >> which bails if some element of @_ is undefined. +1, seeing how hard this is to diagnose. > That would be reasonable. Also reasonable

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-25 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > I wonder if it'd be a good idea to convert > auto_explain's TAP test to load auto_explain via session_preload_libraries > instead of shared_preload_libraries, and then pass in the settings for > each test via PGOPTIONS instead of constantly rewriting postgresql.conf. That

Re: fairywren hung in pg_basebackup tests

2022-07-25 Thread Thomas Munro
On Tue, Jul 26, 2022 at 3:08 AM Tom Lane wrote: > I wrote: > > Right, so the "glob" failed to find anything. Seeing that this test > > is new as of 534472375, which postdates fairywren's last successful > > run, I'd guess that the "glob" needs adjustment for msys path names. The test added by

Re: fairywren hung in pg_basebackup tests

2022-07-25 Thread Noah Misch
On Mon, Jul 25, 2022 at 09:44:21AM -0400, Andrew Dunstan wrote: > On 2022-07-24 Su 15:10, Noah Misch wrote: > > On Sun, Jul 24, 2022 at 12:55:56PM -0400, Tom Lane wrote: > >> Andrew Dunstan writes: > >>> fairywren (msys2 animal) is currently hung in the pg_basebackup tests. > >>> Here's the

Re: [PATCH v1] strengthen backup history filename check

2022-07-25 Thread Junwang Zhao
On Mon, Jul 25, 2022 at 7:39 PM Bharath Rupireddy wrote: > > On Mon, Jul 25, 2022 at 5:01 PM Junwang Zhao wrote: > > > > This patch makes the backup history filename check more tight. > > Can you please elaborate a bit on the issue with existing > IsBackupHistoryFileName(), if there's any? >

Re: Cleaning up historical portability baggage

2022-07-25 Thread Andrew Dunstan
On 2022-07-25 Mo 10:35, Thomas Munro wrote: > On Sun, Jul 24, 2022 at 12:23 PM Tom Lane wrote: >> After looking through these briefly, I'm pretty concerned about >> whether this won't break our Cygwin build in significant ways. >> For example, lorikeet reports "HAVE_SETSID 1", a condition that

Re: fairywren hung in pg_basebackup tests

2022-07-25 Thread Tom Lane
I wrote: > Right, so the "glob" failed to find anything. Seeing that this test > is new as of 534472375, which postdates fairywren's last successful > run, I'd guess that the "glob" needs adjustment for msys path names. Hmm ... an alternative theory is that the test is fine, and what it's

Re: fairywren hung in pg_basebackup tests

2022-07-25 Thread Tom Lane
Andrew Dunstan writes: > On 2022-07-24 Su 15:10, Noah Misch wrote: >> On Sun, Jul 24, 2022 at 12:55:56PM -0400, Tom Lane wrote: >>> Andrew Dunstan writes: fairywren (msys2 animal) is currently hung in the pg_basebackup tests. Here's the bottom of the regress log. I don't have further

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread vignesh C
On Mon, Jul 25, 2022 at 2:24 PM vignesh C wrote: > > On Sun, Jul 24, 2022 at 10:21 PM Jonathan S. Katz > wrote: > > > > On 7/22/22 12:47 AM, Amit Kapila wrote: > > > On Fri, Jul 22, 2022 at 1:39 AM Jonathan S. Katz > > > wrote: > > > > >> 1. I'm concerned by calling this "Bidirectional

[Patch] Fix bounds check in trim_array()

2022-07-25 Thread Martin Kalcher
Hi, while working on something else i encountered a bug in the trim_array() function. The bounds check fails for empty arrays without any dimensions. It reads the size of the non existing first dimension to determine the arrays length. select trim_array('{}'::int[], 10);

Re: Cleaning up historical portability baggage

2022-07-25 Thread Thomas Munro
On Sun, Jul 24, 2022 at 12:23 PM Tom Lane wrote: > After looking through these briefly, I'm pretty concerned about > whether this won't break our Cygwin build in significant ways. > For example, lorikeet reports "HAVE_SETSID 1", a condition that > you want to replace with !WIN32. The question

Re: Unprivileged user can induce crash by using an SUSET param in PGOPTIONS

2022-07-25 Thread Tom Lane
Nathan Bossart writes: > Given all this, I think I'm inclined for the new argument. Pushed like that then (after a bit more fooling with the comments). I haven't done anything about a test case. We can't rely on plperl getting built, and even if we could, it doesn't have any TAP-style tests so

Re: Proposal to provide the facility to set binary format output for specific OID's per session

2022-07-25 Thread Jack Christensen
On Mon, Jul 25, 2022 at 4:57 AM Dave Cramer wrote: > > Dave Cramer > > > On Sun, 24 Jul 2022 at 23:02, Kyotaro Horiguchi > wrote: > >> At Fri, 22 Jul 2022 11:00:18 -0400, Dave Cramer >> wrote in >> > As a proof of concept I provide the attached patch which implements the >> > ability to

Re: Perform streaming logical transactions by background workers and parallel apply

2022-07-25 Thread Amit Kapila
On Fri, Jul 22, 2022 at 8:26 AM wangw.f...@fujitsu.com wrote: > > On Tues, Jul 19, 2022 at 10:29 AM I wrote: > > Attach the news patches. > > Not able to apply patches cleanly because the change in HEAD (366283961a). > Therefore, I rebased the patch based on the changes in HEAD. > > Attach the

Re: fairywren hung in pg_basebackup tests

2022-07-25 Thread Andrew Dunstan
On 2022-07-24 Su 15:10, Noah Misch wrote: > On Sun, Jul 24, 2022 at 12:55:56PM -0400, Tom Lane wrote: >> Andrew Dunstan writes: >>> fairywren (msys2 animal) is currently hung in the pg_basebackup tests. >>> Here's the bottom of the regress log. I don't have further info as yet, >>> but can dig

Re: log_line_prefix: make it possible to add the search_path

2022-07-25 Thread Pierre
On Monday, July 25, 2022 11:52:41 AM CEST Alvaro Herrera wrote: > On 2022-Jul-25, Pierre Ducroquet wrote: > > This is great for performance, but several tools are lacking around this > > usecase by not showing the schema, one of them being log_line_prefix. > > > > The attached patch implements

Re: COPY FROM FORMAT CSV FORCE_NULL(*) ?

2022-07-25 Thread Andrew Dunstan
On 2022-07-25 Mo 00:18, jian he wrote: > Hi, there. > > copy force null git commit > > didn't attach a discussion link. So I don't know if it's already been > discussed. > > Current seem you

Re: add log messages when replication slots become active and inactive (was Re: Is it worth adding ReplicationSlot active_pid to ReplicationSlotPersistentData?)

2022-07-25 Thread Bharath Rupireddy
On Thu, May 5, 2022 at 2:57 PM Bharath Rupireddy wrote: > > On Fri, Apr 29, 2022 at 4:02 PM Alvaro Herrera > wrote: > > > > If I read the code right, this patch emits logs when > > pg_logical_slot_get_changes and pg_replication_slot_advance SQL > > functions are called. Is this

Re: Custom tuplesorts for extensions

2022-07-25 Thread Pavel Borisov
> > Thank you for caching this. Fixed in the revision attached. > > Testing subsets of patchsets in cfbot looks like a good idea to me. > However, I'm not sure if we always require subsets to be consistent. > Hi, hackers! I've looked through a new v6 of a patchset and find it ok. When applied

Re: Typo in misc_sanity.sql?

2022-07-25 Thread Japin Li
On Mon, 25 Jul 2022 at 19:39, Tom Lane wrote: > Japin Li writes: >> I found the misc_sanity has a SQL to check system catalogs that >> do not have primary keys, however, in current exceptions it says >> pg_depend, pg_shdepend don't have a unique key. > > As indeed they do not: > > regression=#

Re: [PATCH v1] strengthen backup history filename check

2022-07-25 Thread Bharath Rupireddy
On Mon, Jul 25, 2022 at 5:01 PM Junwang Zhao wrote: > > This patch makes the backup history filename check more tight. Can you please elaborate a bit on the issue with existing IsBackupHistoryFileName(), if there's any? Also, the patch does have hard coded numbers [1] which isn't good from a

Re: Typo in misc_sanity.sql?

2022-07-25 Thread Tom Lane
Japin Li writes: > I found the misc_sanity has a SQL to check system catalogs that > do not have primary keys, however, in current exceptions it says > pg_depend, pg_shdepend don't have a unique key. As indeed they do not: regression=# \d pg_depend Table "pg_catalog.pg_depend"

[PATCH v1] strengthen backup history filename check

2022-07-25 Thread Junwang Zhao
This patch makes the backup history filename check more tight. -- Regards Junwang Zhao 0001-strengthen-backup-history-filename-check.patch Description: Binary data

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-07-25 Thread Matthias van de Meent
On Wed, 13 Jul 2022 at 07:54, Michael Paquier wrote: > > On Mon, Jul 11, 2022 at 02:26:46PM +0200, Matthias van de Meent wrote: > > Thanks for reviewing. > > I think that v6 is over-engineered because there should be no need to > add a check in xlogreader.c as long as the origin of the problem is

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-07-25 Thread Bharath Rupireddy
On Mon, Apr 25, 2022 at 5:17 PM Bharath Rupireddy wrote: > > On Mon, Apr 25, 2022 at 6:38 AM Michael Paquier wrote: > > > > On Fri, Apr 22, 2022 at 07:17:37PM +0530, Bharath Rupireddy wrote: > > > Right. We find enough disk space and go to write and suddenly the > > > write operations fail for

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-07-25 Thread Heikki Linnakangas
On 13/07/2022 08:54, Michael Paquier wrote: I think that v6 is over-engineered because there should be no need to add a check in xlogreader.c as long as the origin of the problem is blocked, no? And the origin here is when the record is assembled. At least this is the cleanest solution for

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-25 Thread shiy.f...@fujitsu.com
Hi, I did some performance test for the master branch patch (based on v6 patch) to see if the bsearch() added by this patch will cause any overhead. I tested them three times and took the average. The results are as follows, and attach the bar chart. case 1 - No catalog modifying

Re: Make name optional in CREATE STATISTICS

2022-07-25 Thread Alvaro Herrera
On 2022-Jul-25, Michael Paquier wrote: > On Mon, Jul 25, 2022 at 11:49:50AM +0200, Alvaro Herrera wrote: > > On 2022-Jul-23, Michael Paquier wrote: > >> By the way, it seems that 83011ce also broke the case of "REINDEX > >> DATABASE CONCURRENTLY", where the parser missed the addition of a > >>

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-07-25 Thread Andrey Borodin
> 25 июля 2022 г., в 14:29, Bharath Rupireddy > написал(а): > > Hm, after thinking for a while, I tend to agree with the above > approach - meaning, query cancel interrupt processing can completely > be disabled in SyncRepWaitForLSN() and process proc die interrupt > immediately, this

Re: Make name optional in CREATE STATISTICS

2022-07-25 Thread Michael Paquier
On Mon, Jul 25, 2022 at 11:49:50AM +0200, Alvaro Herrera wrote: > On 2022-Jul-23, Michael Paquier wrote: >> As the problem comes down to the fact that INDEX/TABLE, SCHEMA and >> DATABASE/SYSTEM need to handle names for different object types each, >> I think that we could do something like the

Re: make -C libpq check fails obscurely if tap tests are disabled

2022-07-25 Thread Alvaro Herrera
On 2022-Jul-25, Alvaro Herrera wrote: > Anyway, I tried a revert of 1bd201214965 -- I ended up with the > attached. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ >From 387835c04436bc59b9c82e32d9259c74d68ceabf Mon Sep 17 00:00:00 2001 From: Alvaro Herrera

Re: make -C libpq check fails obscurely if tap tests are disabled

2022-07-25 Thread Alvaro Herrera
On 2022-Jul-22, Tom Lane wrote: > Barring objections, I'll push the attached patch. I suppose we > could undo whatever dumbing-down was done in _create_recursive_target, > but is it worth troubling with? Excellent, many thanks. I tried to get Make 3.80 built here, to no avail. I have updated

Re: Collect ObjectAddress for ATTACH DETACH PARTITION to use in event trigger

2022-07-25 Thread Michael Paquier
On Mon, Jul 25, 2022 at 09:25:07AM +, houzj.f...@fujitsu.com wrote: > BTW, while reviewing it, I found there are some more subcommands that the > get_altertable_subcmdtypes() doesn't handle(e.g., ADD/DROP/SET IDENTITY and > re ADD > STAT). Shall we fix them all while on it ? > > Attach a

Re: Proposal to provide the facility to set binary format output for specific OID's per session

2022-07-25 Thread Dave Cramer
Dave Cramer On Sun, 24 Jul 2022 at 23:02, Kyotaro Horiguchi wrote: > At Fri, 22 Jul 2022 11:00:18 -0400, Dave Cramer > wrote in > > As a proof of concept I provide the attached patch which implements the > > ability to specify which oids will be returned in binary format per > > session. >

Re: log_line_prefix: make it possible to add the search_path

2022-07-25 Thread Alvaro Herrera
On 2022-Jul-25, Pierre Ducroquet wrote: > This is great for performance, but several tools are lacking around this > usecase by not showing the schema, one of them being log_line_prefix. > The attached patch implements this, using %S. I've not written the > documentation yet, since I'm not

Re: Make name optional in CREATE STATISTICS

2022-07-25 Thread Alvaro Herrera
On 2022-Jul-23, Michael Paquier wrote: > As the problem comes down to the fact that INDEX/TABLE, SCHEMA and > DATABASE/SYSTEM need to handle names for different object types each, > I think that we could do something like the attached, removing one > block on the way at the cost of an extra

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread vignesh C
On Mon, Jul 25, 2022 at 12:58 PM Peter Smith wrote: > > Firstly, I have some (case-sensitivity) questions about the previous > patch which was already pushed [1]. > > Q1. create_subscription docs > > I did not understand why the docs refer to slot_name = NONE, yet the > newly added option says

Re: Collect ObjectAddress for ATTACH DETACH PARTITION to use in event trigger

2022-07-25 Thread Alvaro Herrera
On 2022-Jul-25, houzj.f...@fujitsu.com wrote: > BTW, while reviewing it, I found there are some more subcommands that the > get_altertable_subcmdtypes() doesn't handle(e.g., ADD/DROP/SET IDENTITY and > re ADD > STAT). Shall we fix them all while on it ? > > Attach a minor patch to fix those

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-07-25 Thread Bharath Rupireddy
On Tue, May 10, 2022 at 5:55 PM Andrey Borodin wrote: > > > 10 мая 2022 г., в 12:59, Bharath Rupireddy > > написал(а): > > > > If okay, I can make the GUC behave this way - value 0 existing > > behaviour i.e. no wait for sync repl ack, just process query cancels > > and proc die interrupts

RE: Collect ObjectAddress for ATTACH DETACH PARTITION to use in event trigger

2022-07-25 Thread houzj.f...@fujitsu.com
On Saturday, July 23, 2022 6:58 PM Michael Paquier wrote: > > On Sat, Jul 23, 2022 at 05:44:28PM +0900, Michael Paquier wrote: > > Changing get_altertable_subcmdtypes() to return a set of rows made of > > (subcommand, object description) is what I actually meant upthread as > > it feels natural

Re: Add last failed connection error message to pg_stat_wal_receiver

2022-07-25 Thread Michael Paquier
On Mon, Jul 25, 2022 at 12:19:40PM +0530, Bharath Rupireddy wrote: > Thanks a lot Cary for reviewing. It will be great if you can add > yourself as a reviewer and set the status accordingly in the CF entry > here - https://commitfest.postgresql.org/38/3666/. Hmm. This stands for the connection

Re: Collect ObjectAddress for ATTACH DETACH PARTITION to use in event trigger

2022-07-25 Thread Michael Paquier
On Mon, Jul 25, 2022 at 08:42:18AM +0530, Amit Kapila wrote: > What I intended to say is similar to what you have done in the patch > but in a new function. OTOH, your point that it is okay to change > function signature/name in the test module seems reasonable to me. Thanks. Let's do with the

Password reset link / 'less' does not exit in psql version 13.4

2022-07-25 Thread Michael J. Baars
Hello, I have two very simple questions: 1) I have an account at postgresql.org, but a link to a 'forgot password' seems to be missing on the login page. I have my password stored only on an old Fedora 32 computer. To change the password when logged in, you need to supply the old password. In

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread Amit Kapila
On Mon, Jul 25, 2022 at 12:58 PM Peter Smith wrote: > > Firstly, I have some (case-sensitivity) questions about the previous > patch which was already pushed [1]. > > Q1. create_subscription docs > > I did not understand why the docs refer to slot_name = NONE, yet the > newly added option says

Re: SLRUs in the main buffer pool, redux

2022-07-25 Thread Heikki Linnakangas
On 25/07/2022 09:54, Heikki Linnakangas wrote: In RecordTransactionCommit(), we enter a critical section, and then call TransactionIdCommitTree() to update the CLOG pages. That now involves a call to ReadBuffer_common(), which in turn calls ResourceOwnerEnlargeBuffers(). That can fail, because

Re: Handle infinite recursion in logical replication setup

2022-07-25 Thread vignesh C
On Sun, Jul 24, 2022 at 10:21 PM Jonathan S. Katz wrote: > > On 7/22/22 12:47 AM, Amit Kapila wrote: > > On Fri, Jul 22, 2022 at 1:39 AM Jonathan S. Katz > > wrote: > > >> 1. I'm concerned by calling this "Bidirectional replication" in the docs > >> that we are overstating the current

Re: Is it correct to say, "invalid data in file \"%s\"", BACKUP_LABEL_FILE in do_pg_backup_stop?

2022-07-25 Thread Bharath Rupireddy
On Thu, Jul 21, 2022 at 2:33 PM Kyotaro Horiguchi wrote: > > At Wed, 20 Jul 2022 17:09:09 +0530, Bharath Rupireddy > wrote in > > Hi, > > > > After the commit [1], is it correct to say errmsg("invalid data in file > > \"%s\"", BACKUP_LABEL_FILE))); in do_pg_backup_stop() when we hold the > >

Re: Typo in misc_sanity.sql?

2022-07-25 Thread Bharath Rupireddy
On Mon, Jul 25, 2022 at 1:24 PM Japin Li wrote: > > > Hi, hackers > > I found the misc_sanity has a SQL to check system catalogs that > do not have primary keys, however, in current exceptions it says > pg_depend, pg_shdepend don't have a unique key. > > Should we fix it? Indeed. There's a clear

Re: Column Filtering in Logical Replication

2022-07-25 Thread Peter Smith
On Sat, Dec 11, 2021 at 12:24 AM Alvaro Herrera wrote: > > On 2021-Dec-10, Peter Eisentraut wrote: > ... > > > There was no documentation, so I wrote a bit (patch 0001). It only touches > > the CREATE PUBLICATION and ALTER PUBLICATION pages at the moment. There was > > no mention in the Logical

Typo in misc_sanity.sql?

2022-07-25 Thread Japin Li
Hi, hackers I found the misc_sanity has a SQL to check system catalogs that do not have primary keys, however, in current exceptions it says pg_depend, pg_shdepend don't have a unique key. Should we fix it? -- Regrads, Japin Li. ChengDu WenWu Information Technology Co.,Ltd. diff --git

  1   2   >