Re: How to check for in-progress transactions

2023-03-15 Thread Bharath Rupireddy
On Thu, Mar 16, 2023 at 1:18 AM Tejasvi Kashi wrote: > > For my use case, I'm trying to ascertain if there are any in-flight > transactions that are yet to be replicated to synchronous standbys (in a > synchronous streaming replication setting) > > I've been looking at sent_lsn, write_lsn,

Re: Get rid of PgStat_BackendFunctionEntry

2023-03-15 Thread Michael Paquier
On Wed, Mar 08, 2023 at 01:38:38PM -0800, Nathan Bossart wrote: > Looks reasonable to me. I have been catching up with this thread and the other thread, and indeed it looks like this is going to help in refactoring pgstatfuncs.c to have more macros for all these mostly-duplicated functions. So,

Re: Add macros for ReorderBufferTXN toptxn

2023-03-15 Thread Amit Kapila
On Thu, Mar 16, 2023 at 10:40 AM Bharath Rupireddy wrote: > > On Thu, Mar 16, 2023 at 7:20 AM Peter Smith wrote: > > > > PSA v4 which addresses both of your review comments. > > Looks like a reasonable change to me. > > A nitpick: how about using rbtxn_get_toptxn instead of an explicit >

Re: Bug in jsonb_in function (14 & 15 version are affected)

2023-03-15 Thread Yura Sokolov
В Пн, 13/03/2023 в 13:58 -0400, Tom Lane пишет: > Nikolay Shaplov writes: > > I found a bug in jsonb_in function (it converts json from sting > > representation > >  into jsonb internal representation). > > Yeah.  Looks like json_lex_string is failing to honor the invariant > that it needs to

Re: Add macros for ReorderBufferTXN toptxn

2023-03-15 Thread Bharath Rupireddy
On Thu, Mar 16, 2023 at 7:20 AM Peter Smith wrote: > > PSA v4 which addresses both of your review comments. Looks like a reasonable change to me. A nitpick: how about using rbtxn_get_toptxn instead of an explicit variable toptxn for single use? 1. Change ReorderBufferTXN *toptxn =

Re: zstd compression for pg_dump

2023-03-15 Thread Justin Pryzby
On Fri, Mar 10, 2023 at 12:48:13PM -0800, Jacob Champion wrote: > On Wed, Mar 8, 2023 at 10:59 AM Jacob Champion > wrote: > > I did some smoke testing against zstd's GitHub release on Windows. To > > build against it, I had to construct an import library, and put that > > and the DLL into the

Re: Simplify some codes in pgoutput

2023-03-15 Thread Amit Kapila
On Wed, Mar 15, 2023 at 2:00 PM houzj.f...@fujitsu.com wrote: > > I noticed that there are some duplicated codes in pgoutput_change() function > which can be simplified, and here is an attempt to do that. > For REORDER_BUFFER_CHANGE_DELETE, when the old tuple is missing, after this patch, we

Re: Initial Schema Sync for Logical Replication

2023-03-15 Thread Peter Smith
Hi, I have a couple of questions. Q1. What happens if the subscriber already has some tables present? For example, I did not see the post saying anything like "Only if the table does not already exist then it will be created". On the contrary, the post seemed to say SUBREL_STATE_CREATE 'c'

Re: "current directory" in a server error message

2023-03-15 Thread Bharath Rupireddy
On Thu, Mar 16, 2023 at 7:47 AM Kyotaro Horiguchi wrote: > > Hello. > > When I ran pg_ls_dir('..'), the error message I received was somewhat > difficult to understand. > > postgres=> select * from pg_ls_dir('..'); > ERROR: path must be in or below the current directory > > As far as I know the

Re: meson documentation build open issues

2023-03-15 Thread Andres Freund
Hi, On 2023-03-15 08:14:09 +0100, Peter Eisentraut wrote: > I have identified several open issues with the documentation build under > Meson (approximately in priority order): > > 1. Image files are not handled at all, so they don't show up in the final > product. Hm. Somehow I thought I'd

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Amit Kapila
On Thu, Mar 16, 2023 at 8:27 AM Euler Taveira wrote: > > On Wed, Mar 8, 2023, at 11:50 PM, Amit Kapila wrote: > > It is not clear to me which version check you wanted to add because we > seem to have a binary option in COPY from the time prior to logical > replication. I feel we need a publisher

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-15 Thread Alexander Lakhin
16.03.2023 00:13, Thomas Munro wrote: Thanks. We were wondering if the retry mechanism might somehow be hiding this in non-assert builds, but, looking more closely, that is tied specifically to the memory reservation operation. As to hiding, when analyzing the Assert issue, I was wondered if

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Euler Taveira
On Wed, Mar 8, 2023, at 11:50 PM, Amit Kapila wrote: > On Wed, Mar 8, 2023 at 6:17 PM Melih Mutlu wrote: > > > > On 7 Mar 2023 Tue at 04:10 Amit Kapila wrote: > >> > >> As per what I could read in this thread, most people prefer to use the > >> existing binary option rather than inventing a new

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Amit Kapila
On Wed, Mar 15, 2023 at 3:33 PM Melih Mutlu wrote: > > Amit Kapila , 15 Mar 2023 Çar, 12:31 tarihinde şunu > yazdı: >> >> On Tue, Mar 14, 2023 at 4:32 PM Melih Mutlu wrote: > > >> >> What purpose does this test serve w.r.t this patch? Before checking >> the sync for different column orders, the

RE: Allow logical replication to copy tables in binary format

2023-03-15 Thread houzj.f...@fujitsu.com
Hi, Thanks for updating the patch, I think it is a useful feature. I looked at the v15 patch and the patch looks mostly good to me. Here are few comments: 1. + { + appendStringInfo(, " WITH (FORMAT binary)"); We could use appendStringInfoString here. 2. +# It should fail

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Amit Kapila
On Thu, Mar 16, 2023 at 5:59 AM Peter Smith wrote: > > On Wed, Mar 15, 2023 at 5:31 PM Amit Kapila wrote: > > > > On Wed, Mar 15, 2023 at 11:52 AM Peter Smith wrote: > > > > > > == > > > src/backend/replication/logical/tablesync.c > > > > > > 5. > > > + > > > + /* > > > + * If the publisher

RE: Support logical replication of DDLs

2023-03-15 Thread wangw.f...@fujitsu.com
On Tues, Mar 14, 2023 12:17 PM Ajin Cherian wrote: > On Mon, Mar 13, 2023 at 2:24 AM Zheng Li wrote: > > > > Thanks for working on the test coverage for CREATE and ALTER TABLE. > > I've made fixes for some of the failures in the v79 patch set (0002, > > 0003 and 0004 are updated). The changes

RE: Allow logical replication to copy tables in binary format

2023-03-15 Thread shiy.f...@fujitsu.com
On Thu, Mar 16, 2023 2:26 AM Melih Mutlu wrote: > > Right, it needs to be ordered. Fixed. > Hi, Thanks for updating the patch. I tested some cases like toast data, combination of row filter and column lists, and it works well. Here is a comment: +# Ensure the COPY command is executed in

"current directory" in a server error message

2023-03-15 Thread Kyotaro Horiguchi
Hello. When I ran pg_ls_dir('..'), the error message I received was somewhat difficult to understand. postgres=> select * from pg_ls_dir('..'); ERROR: path must be in or below the current directory As far as I know the concept of a "current directory" doesn't apply to the server side. In fact,

Re: Add macros for ReorderBufferTXN toptxn

2023-03-15 Thread Peter Smith
On Wed, Mar 15, 2023 at 4:55 PM Masahiko Sawada wrote: > > Hi, > ... > +1 to the idea. Here are some minor comments: > > @@ -1667,7 +1658,7 @@ ReorderBufferTruncateTXN(ReorderBuffer *rb, > ReorderBufferTXN *txn, bool txn_prep > * about the toplevel xact (we send the XID in all messages), but we

Re: Making empty Bitmapsets always be NULL

2023-03-15 Thread Yuya Watari
Hello, On Tue, Mar 7, 2023 at 1:07 PM David Rowley wrote: > I adjusted the patch to remove the invariant checks and fixed up a > couple of things I'd missed. The 0002 patch changes the for loops > into do while loops. I wanted to see if we could see any performance > gains from doing this. I

Re: recovery modules

2023-03-15 Thread Michael Paquier
On Tue, Mar 14, 2023 at 09:13:09PM -0700, Nathan Bossart wrote: > However, that's not what happens when hot_standby is off. In that case, > the postmaster.pid file is updated with PM_STATUS_STANDBY once recovery > starts, which wait_for_postmaster_start() interprets as "ready." I see > this was

Re: Fix fseek() detection of unseekable files on WIN32

2023-03-15 Thread Michael Paquier
On Wed, Mar 15, 2023 at 12:18:25PM +0100, Juan José Santamaría Flecha wrote: > PFA a new version of the patch. +_pgftello64(FILE *stream) +{ + DWORD fileType; + + fileType = GetFileType((HANDLE) _get_osfhandle(_fileno(stream))); Hmm. I am a bit surprised here.. It seems to me

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-15 Thread Melanie Plageman
Thanks for the reviews and for trying the patch! On Wed, Mar 15, 2023 at 4:31 AM Masahiko Sawada wrote: > > On Sat, Mar 11, 2023 at 11:55 PM Melanie Plageman > wrote: > > > > > On Tue, Feb 28, 2023 at 3:16 AM Bharath Rupireddy > > > wrote: > > > > > > > On Thu, Jan 12, 2023 at 6:06 AM Andres

Re: psql \watch 2nd argument: iteration count

2023-03-15 Thread Michael Paquier
On Wed, Mar 15, 2023 at 04:58:49PM +0900, Michael Paquier wrote: > Yep. You are right. Fixed that and applied 0001. +valptr++; +if (strncmp("i", opt, strlen("i")) == 0 || +strncmp("interval", opt, strlen("interval")) == 0) +{ Did you look at process_command_g_options()

Re: Add a hook to allow modification of the ldapbindpasswd

2023-03-15 Thread Michael Paquier
On Wed, Mar 15, 2023 at 06:18:28PM -0400, Andrew Dunstan wrote: > Ugh. Not batting 1000 today. Will investigate. I have noticed that you forgot a .gitignore in this new path, as well, so I have taken the liberty to add one ;) FWIW, I use git-sh-prompt prompt to detect such things quickly. --

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Peter Smith
On Wed, Mar 15, 2023 at 5:31 PM Amit Kapila wrote: > > On Wed, Mar 15, 2023 at 11:52 AM Peter Smith wrote: > > > > == > > src/backend/replication/logical/tablesync.c > > > > 5. > > + > > + /* > > + * If the publisher version is earlier than v14, it COPY command doesn't > > + * support the

Re: Add LZ4 compression in pg_dump

2023-03-15 Thread Justin Pryzby
On Mon, Mar 13, 2023 at 10:47:12PM +0100, Tomas Vondra wrote: > > Rearrange functions to their original order allowing a cleaner diff to the > > prior code; > > OK. I wasn't very enthusiastic about this initially, but after thinking > about it a bit I think it's meaningful to make diffs clearer.

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-15 Thread Justin Pryzby
On Tue, Mar 14, 2023 at 08:56:58PM -0400, Melanie Plageman wrote: > On Sat, Mar 11, 2023 at 2:16 PM Justin Pryzby wrote: > > > @@ -586,6 +587,45 @@ GetAccessStrategy(BufferAccessStrategyType btype) > > > +BufferAccessStrategy > > > +GetAccessStrategyWithSize(BufferAccessStrategyType btype, int

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-15 Thread Justin Pryzby
On Tue, Mar 14, 2023 at 06:58:14PM +0400, Ilya Gladyshev wrote: > > The only change from the current patch is (3). (1) still calls > > count_leaf_partitions(), but only once. I'd prefer that to rearranging > > the progress reporting to set the TOTAL in ProcessUtilitySlow(). > > As for reusing

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Peter Smith
Here are some review comments for v15-0001 == doc/src/sgml/logical-replication.sgml 1. + target table. However, logical replication in binary format is more restrictive, + see binary option of + CREATE SUBSCRIPTION + for more details. IMO (and Chat-GPT agrees) the new text should

Re: Add a hook to allow modification of the ldapbindpasswd

2023-03-15 Thread Andrew Dunstan
On 2023-03-15 We 17:50, Tom Lane wrote: Andrew Dunstan writes: pushed. drongo is not happy with this, but I'm kind of baffled as to why: "c:\\prog\\bf\\root\\HEAD\\pgsql.build\\pgsql.sln" (default target) (1) -> "c:\\prog\\bf\\root\\HEAD\\pgsql.build\\ldap_password_func.vcxproj" (default

Re: ICU locale validation / canonicalization

2023-03-15 Thread Jeff Davis
On Tue, 2023-03-14 at 23:47 -0700, Jeff Davis wrote: > On Tue, 2023-03-14 at 10:10 -0700, Jeff Davis wrote: > > One loose end is that we really should support language tags like > > "und" > > in those older versions (54 and earlier). Your commit d72900bded > > avoided the problem, but perhaps we

Re: Memory leak in libxml2 (was Re: [PATCH] Add pretty-printed XML output option)

2023-03-15 Thread Daniel Gustafsson
> On 15 Mar 2023, at 22:38, Tom Lane wrote: > After a bit of further testing: the leak is present in libxml2 2.9.7 > which is what I have on this RHEL8 box, but it seems not to occur > in libxml2 2.10.3 (tested on Fedora 37, and I verified that Fedora > isn't carrying any relevant local patch).

Re: Speed-up shared buffers prewarming

2023-03-15 Thread Melanie Plageman
On Wed, Mar 15, 2023 at 4:38 PM Konstantin Knizhnik wrote: > It is well known fact that queries using sequential scan can not be used to > prewarm cache, because them are using ring buffer > even if shared buffers are almost empty. > I have searched hackers archive but failed to find any

Re: Add a hook to allow modification of the ldapbindpasswd

2023-03-15 Thread Tom Lane
Andrew Dunstan writes: > pushed. drongo is not happy with this, but I'm kind of baffled as to why: "c:\\prog\\bf\\root\\HEAD\\pgsql.build\\pgsql.sln" (default target) (1) -> "c:\\prog\\bf\\root\\HEAD\\pgsql.build\\ldap_password_func.vcxproj" (default target) (60) -> (Link target) ->

Re: Speed-up shared buffers prewarming

2023-03-15 Thread Matthias van de Meent
On Wed, 15 Mar 2023 at 21:38, Konstantin Knizhnik wrote: > > Hi hackers, > > It is well known fact that queries using sequential scan can not be used to > prewarm cache, because them are using ring buffer > even if shared buffers are almost empty. > I have searched hackers archive but failed to

Memory leak in libxml2 (was Re: [PATCH] Add pretty-printed XML output option)

2023-03-15 Thread Tom Lane
I wrote: > BTW, the libxml leak problem seems to extend to other cases too. > I tested with code like > do $$ > declare x xml; t text; > begin > x := ' encoding="utf8"?>73'; > for i in 1..1000 loop > t := xmlserialize(document x as text); > end loop; > raise notice 't = %', t; > end; > $$;

Re: Add support for DEFAULT specification in COPY FROM

2023-03-15 Thread Andrew Dunstan
On 2023-03-15 We 13:00, Alexander Lakhin wrote: Hello, 13.03.2023 17:15, Andrew Dunstan wrote: On 2022-12-02 Fr 09:11, Israel Barth Rubio wrote: Hello all, I'm submitting a new version of the patch. Instead of changing signature of several functions in order to use the defaults parameter,

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-15 Thread Thomas Munro
On Thu, Mar 16, 2023 at 2:00 AM Alexander Lakhin wrote: > 15.03.2023 11:43, Thomas Munro wrote: > > Do you know how it fails in non-assert builds, without the fix? > So at least with my test script that doesn't lead to a crash or something. Thanks. We were wondering if the retry mechanism

Re: [PATCH] Add pretty-printed XML output option

2023-03-15 Thread Tom Lane
I wrote: > Huh, interesting. That is a legitimate pretty-fication of the input, > I suppose, but some people might think it goes beyond the charter of > "indentation". I'm okay with it personally; anyone want to object? Hearing no objections to that, I moved ahead with this. It occurred to me

Re: Add support for DEFAULT specification in COPY FROM

2023-03-15 Thread Andrew Dunstan
On 2023-03-15 We 13:00, Alexander Lakhin wrote: Hello, 13.03.2023 17:15, Andrew Dunstan wrote: On 2022-12-02 Fr 09:11, Israel Barth Rubio wrote: Hello all, I'm submitting a new version of the patch. Instead of changing signature of several functions in order to use the defaults parameter,

Re: Add a hook to allow modification of the ldapbindpasswd

2023-03-15 Thread Andrew Dunstan
On 2023-01-23 Mo 14:11, Andrew Dunstan wrote: On 2022-12-19 Mo 11:29, Andrew Dunstan wrote: This patch, mostly the work of John Naylor, provides a hook whereby a module can modify the ldapbindpasswd before it is handed to the ldap server. This is similar in concept to the

Speed-up shared buffers prewarming

2023-03-15 Thread Konstantin Knizhnik
Hi hackers, It is well known fact that queries using sequential scan can not be used to prewarm cache, because them are using ring buffer even if shared buffers are almost empty. I have searched hackers archive but failed to find any discussion about it. What are the drawbacks of using free

Re: On login trigger: take three

2023-03-15 Thread Mikhail Gribkov
Hi Gregory, Thanks for the note. The problem was that the patch was not aware of yesterday Tom Lane's changes in the test. It's fixed now: the attached v39 patch contains the updated version along with the freshest rebase on master branch. -- best regards, Mikhail A. Gribkov e-mail:

How to check for in-progress transactions

2023-03-15 Thread Tejasvi Kashi
Hi everyone, I'm Tej, a grad student poking around postgres for a project. For my use case, I'm trying to ascertain if there are any in-flight transactions that are yet to be replicated to synchronous standbys (in a synchronous streaming replication setting) The first way to do this would be to

Re: Commitfest 2023-03 starting tomorrow!

2023-03-15 Thread Gregory Stark (as CFM)
So, sorry I've been a bit under the weather but can concentrate on the commitfest now. I tried to recapitulate the history but the activity log only goes back a certain distance on the web. If I can log in to the database I guess I could construct the history from sql queries if it was important.

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Melih Mutlu
Hi, vignesh C , 15 Mar 2023 Çar, 18:12 tarihinde şunu yazdı: > One comment: > 1) There might be a chance the result order of select may vary as > "ORDER BY" is not specified, Should we add "ORDER BY" as the table > has multiple rows: > +# Check the synced data on the subscriber > +$result =

Re: On login trigger: take three

2023-03-15 Thread Gregory Stark (as CFM)
It looks like the patch is failing a test by not dumping login_event_trigger_func? I'm guessing there's a race condition in the test but I don't know. I also don't see the tmp_test_jI6t output file being preserved in the artifacts anywhere :(

Initial Schema Sync for Logical Replication

2023-03-15 Thread Kumar, Sachin
Hi Everyone, I am working on the initial schema sync for Logical replication. Currently, user have to manually create a schema on subscriber side. Aim of this feature is to add an option in create subscription, so that schema sync can be automatic. I am sharing Design Doc below, but there are

Re: Add support for DEFAULT specification in COPY FROM

2023-03-15 Thread Alexander Lakhin
Hello, 13.03.2023 17:15, Andrew Dunstan wrote: On 2022-12-02 Fr 09:11, Israel Barth Rubio wrote: Hello all, I'm submitting a new version of the patch. Instead of changing signature of several functions in order to use the defaults parameter, it is now storing that in the cstate structure,

Re: optimize several list functions with SIMD intrinsics

2023-03-15 Thread Ankit Kumar Pandey
On 15/03/23 21:53, Nathan Bossart wrote: Did you try building without SIMD support? This is what I see: list.c: In function ‘list_member_ptr’: list.c:697:2: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement] 697 | const

Re: optimize several list functions with SIMD intrinsics

2023-03-15 Thread Nathan Bossart
On Wed, Mar 15, 2023 at 07:31:46PM +0530, Ankit Kumar Pandey wrote: >> On 14/03/23 03:10, Nathan Bossart wrote: >> On Sat, Mar 11, 2023 at 09:41:18AM +, Ankit Kumar Pandey wrote: >> > 1. In list_member_ptr, will it be okay to bring `const ListCell >> > *cell` from #ifdef USE_NO_SIMD >> >

Re: pkg-config Requires.private entries should be comma-separated

2023-03-15 Thread Andres Freund
Hi, On 2023-03-15 08:51:04 +0100, Peter Eisentraut wrote: > While comparing the .pc (pkg-config) files generated by the make and meson > builds, I noticed that the Requires.private entries use different > delimiters. The make build uses spaces, the meson build uses commas. The > pkg-config

Re: CI and test improvements

2023-03-15 Thread Peter Eisentraut
On 15.03.23 15:56, Justin Pryzby wrote: I'm surprised if there's any question about the merits of making documentation easily available for review. Several people have agreed; one person mailed me privately specifically to ask how to show HTML docs on cirrusci. Anyway, all this stuff is best

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread vignesh C
On Wed, 15 Mar 2023 at 15:30, Melih Mutlu wrote: > > Hi, > > Please see the attached patch. One comment: 1) There might be a chance the result order of select may vary as "ORDER BY" is not specified, Should we add "ORDER BY" as the table has multiple rows: +# Check the synced data on the

Re: CI and test improvements

2023-03-15 Thread Justin Pryzby
On Wed, Mar 15, 2023 at 10:58:41AM +0100, Peter Eisentraut wrote: > On 14.03.23 05:56, Justin Pryzby wrote: > > I'm soliticing feedback on those patches that I've sent recently - I've > > elided patches if they have some unresolved issue. > > > [PATCH 1/8] cirrus/windows: add

Re: Making background psql nicer to use in tap tests

2023-03-15 Thread Andrew Dunstan
On 2023-01-30 Mo 19:00, Andres Freund wrote: Hi, On 2023-01-30 15:06:46 -0500, Tom Lane wrote: Andres Freund writes: It's annoyingly hard to wait for the result of a query in a generic way with background_psql(), and more generally for psql. background_psql() uses -XAtq, which means that

Re: optimize several list functions with SIMD intrinsics

2023-03-15 Thread Ankit Kumar Pandey
Agree with your points Nathan. Just a headup. On 14/03/23 03:10, Nathan Bossart wrote: On Sat, Mar 11, 2023 at 09:41:18AM +, Ankit Kumar Pandey wrote: 1. In list_member_ptr, will it be okay to bring `const ListCell *cell` from #ifdef USE_NO_SIMD const ListCell *cell; #endif to

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-15 Thread Alexander Lakhin
Hi, 15.03.2023 11:43, Thomas Munro wrote: On Wed, Mar 15, 2023 at 9:00 PM Alexander Lakhin wrote: The result depends on some OS conditions (it reproduced pretty well immediately after VM reboot), but it's enough to test the patch proposed. And I can confirm that the Assert is not observed

Re: logical decoding and replication of sequences, take 2

2023-03-15 Thread Tomas Vondra
On 3/10/23 11:03, John Naylor wrote: > > On Wed, Mar 1, 2023 at 1:02 AM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: >> here's a rebased patch to make cfbot happy, dropping the first part that >> is now unnecessary thanks to 7fe1aa991b. > > Hi Tomas, > > I'm looking into

Re: Documentation for building with meson

2023-03-15 Thread Peter Eisentraut
> [PATCH v8 1/5] Make minor additions and corrections to meson docs The last hunk revealed that there is some mixing up between meson setup and meson configure. This goes a bit further. For example, earlier it says that to get a list of meson setup options, call meson configure --help and

Re: Fix fseek() detection of unseekable files on WIN32

2023-03-15 Thread Juan José Santamaría Flecha
On Wed, Mar 15, 2023 at 5:57 AM Michael Paquier wrote: > On Tue, Mar 14, 2023 at 01:26:27PM +0100, Juan José Santamaría Flecha > wrote: > > As highlighted in [1] fseek() might fail to error even when accessing > > unseekable streams. > > > > PFA a patch that checks the file type before the

Re: postgres_fdw: Useless if-test in GetConnection()

2023-03-15 Thread Daniel Gustafsson
> On 15 Mar 2023, at 11:18, Etsuro Fujita wrote: > While working on something else, I noticed that the “if (entry->conn > == NULL)” test after doing disconnect_pg_server() when re-establishing > a given connection in GetConnection() is pointless, because the former > function ensures that

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-15 Thread Ants Aasma
On Wed, 15 Mar 2023 at 02:57, Melanie Plageman wrote: > > > Subject: [PATCH v3 3/3] add vacuum option to specify ring_size and guc > > > > > #define INT_ACCESS_ONCE(var) ((int)(*((volatile int *)&(var > > > +#define bufsize_limit_to_nbuffers(bufsize) (bufsize * 1024 / BLCKSZ) > > > > Macros

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-15 Thread Ants Aasma
On Wed, 15 Mar 2023 at 02:29, Melanie Plageman wrote: > As for routine vacuuming and the other buffer access strategies, I think > there is an argument for configurability based on operator knowledge -- > perhaps your workload will use the data you are COPYing as soon as the > COPY finishes, so

Re: postgres_fdw: Useless if-test in GetConnection()

2023-03-15 Thread Richard Guo
On Wed, Mar 15, 2023 at 6:18 PM Etsuro Fujita wrote: > While working on something else, I noticed that the “if (entry->conn > == NULL)” test after doing disconnect_pg_server() when re-establishing > a given connection in GetConnection() is pointless, because the former > function ensures that

Re: TAP output format in pg_regress

2023-03-15 Thread Peter Eisentraut
On 24.02.23 10:49, Daniel Gustafsson wrote: Another rebase on top of 337903a16f. Unless there are conflicting reviews, I consider this patch to be done and ready for going in during the next CF. I think this is just about as good as it's going to get, so I think we can consider committing

postgres_fdw: Useless if-test in GetConnection()

2023-03-15 Thread Etsuro Fujita
Hi, While working on something else, I noticed that the “if (entry->conn == NULL)” test after doing disconnect_pg_server() when re-establishing a given connection in GetConnection() is pointless, because the former function ensures that entry->conn is NULL. So I removed the if-test. Attached is

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Melih Mutlu
Amit Kapila , 15 Mar 2023 Çar, 12:31 tarihinde şunu yazdı: > On Tue, Mar 14, 2023 at 4:32 PM Melih Mutlu > wrote: > > What purpose does this test serve w.r.t this patch? Before checking > the sync for different column orders, the patch has already changed > binary to false, so it doesn't seem

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Melih Mutlu
Hi, Please see v14 [1]. Peter Smith , 15 Mar 2023 Çar, 09:22 tarihinde şunu yazdı: > Here are some review comments for v13-0001 > > == > doc/src/sgml/logical-replication.sgml > > 1. > That's why in the previous v12 review [1] (comment #3) I suggested > that this page should just say

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Melih Mutlu
Hi, Please see the attached patch. Takamichi Osumi (Fujitsu) , 14 Mar 2023 Sal, 18:20 tarihinde şunu yazdı: > (1) create_subscription.sgml > > + column types as opposed to text format. Even when this option > is enabled, > + only data types having binary send and receive

Re: CI and test improvements

2023-03-15 Thread Peter Eisentraut
On 14.03.23 05:56, Justin Pryzby wrote: I'm soliticing feedback on those patches that I've sent recently - I've elided patches if they have some unresolved issue. > [PATCH 1/8] cirrus/windows: add compiler_warnings_script Needs a better description of what it actually does. (And fewer "I'm

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-15 Thread Michael Paquier
On Wed, Mar 15, 2023 at 12:40:17PM +0530, Bharath Rupireddy wrote: > -1 for removing \dx+ for pg_walinspect version 1.0, because we wanted > to show the diff of functions along with testing the upgrade path in > the oldextversions.sql. Therefore, I prefer something like [1]: This is a duplicate

Re: [PATCH] Extend the length of BackgroundWorker.bgw_library_name

2023-03-15 Thread Daniel Gustafsson
> On 13 Mar 2023, at 18:35, Nathan Bossart wrote: > > On Mon, Mar 13, 2023 at 07:57:47AM -0700, Yurii Rashkovskii wrote: >> However, there are use cases where [potentially] longer names are >> expected/desired; for example, test benches (where library files may not >> [or can not] be copied to

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Amit Kapila
On Tue, Mar 14, 2023 at 4:32 PM Melih Mutlu wrote: > > Attached v13. > I have a question related to the below test in the patch: +# Setting binary to false should allow syncing +$node_subscriber->safe_psql( +'postgres', qq( +ALTER SUBSCRIPTION tsub SET (binary = false);)); + +# Ensure

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2023-03-15 Thread Jelte Fennema
The rebase was indeed trivial (git handled everything automatically), because my first patch was doing a superset of the changes that were committed in b6dfee28f. Attached are the new patches. On Tue, 14 Mar 2023 at 19:04, Greg Stark wrote: > > On Tue, 14 Mar 2023 at 13:59, Tom Lane wrote: > >

Re: [EXTERNAL] Re: Support load balancing in libpq

2023-03-15 Thread Jelte Fennema
Rebased On Tue, 14 Mar 2023 at 19:05, Gregory Stark (as CFM) wrote: > > The pgindent run in b6dfee28f is causing this patch to need a rebase > for the cfbot to apply it. v12-0002-Refactor-libpq-to-store-addrinfo-in-a-libpq-owne.patch Description: Binary data

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-15 Thread Thomas Munro
On Wed, Mar 15, 2023 at 9:00 PM Alexander Lakhin wrote: > The result depends on some OS conditions (it reproduced pretty well > immediately after VM reboot), but it's enough to test the patch proposed. > And I can confirm that the Assert is not observed anymore (with the sleep > added after

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-03-15 Thread Richard Guo
On Wed, Mar 15, 2023 at 2:25 AM Gregory Stark (as CFM) wrote: > So what is the status of this patch? > > It looks like you received some feedback from Emre, Tom, Ronan, and > Alvaro but it also looks like you responded to most or all of that. > Are you still blocked waiting for feedback?

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-15 Thread Masahiko Sawada
On Sat, Mar 11, 2023 at 11:55 PM Melanie Plageman wrote: > > > On Tue, Feb 28, 2023 at 3:16 AM Bharath Rupireddy > > wrote: > > > > > On Thu, Jan 12, 2023 at 6:06 AM Andres Freund wrote: > > > > > > > > On 2023-01-11 17:26:19 -0700, David G. Johnston wrote: > > > > > Should we just add

Simplify some codes in pgoutput

2023-03-15 Thread houzj.f...@fujitsu.com
Hi, I noticed that there are some duplicated codes in pgoutput_change() function which can be simplified, and here is an attempt to do that. Best Regards, Hou Zhijie 0001-simplify-the-code-in-pgoutput_change.patch Description: 0001-simplify-the-code-in-pgoutput_change.patch

Re: meson: Non-feature feature options

2023-03-15 Thread Peter Eisentraut
On 14.03.23 15:07, Nazir Bilal Yavuz wrote: I think the uuid side of this is making this way too complicated. I'm content leaving this as a manual option for now. There is much more value in making the ssl option work automatically. So I would welcome a patch that just makes -Dssl=auto work

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-15 Thread Alexander Lakhin
Hi, 14.03.2023 01:20, Andres Freund wrote: I am yet to construct a reproduction of the case, but it seems to me that the race condition is not impossible here. I suspect the issue could be made much more likely by adding a sleep before the pg_queue_signal(SIGCHLD) in

Re: psql \watch 2nd argument: iteration count

2023-03-15 Thread Michael Paquier
On Tue, Mar 14, 2023 at 09:23:48PM -0700, Nathan Bossart wrote: > + sleep = strtod(opt, _end); > + if (sleep < 0 || *opt_end || errno == ERANGE) > > Should we set errno to 0 before calling strtod()? Yep. You are right. -- Michael signature.asc

pkg-config Requires.private entries should be comma-separated

2023-03-15 Thread Peter Eisentraut
While comparing the .pc (pkg-config) files generated by the make and meson builds, I noticed that the Requires.private entries use different delimiters. The make build uses spaces, the meson build uses commas. The pkg-config documentation says that it should be comma-separated, but apparently

Re: Privileges on PUBLICATION

2023-03-15 Thread Antonin Houska
Gregory Stark (as CFM) wrote: > FYI this looks like it needs a rebase due to a conflict in copy.c and > an offset in pgoutput.c. > > Is there anything specific that still needs review or do you think > you've handled all Peter's concerns? In particular, is there "a > comprehensive description

meson documentation build open issues

2023-03-15 Thread Peter Eisentraut
I have identified several open issues with the documentation build under Meson (approximately in priority order): 1. Image files are not handled at all, so they don't show up in the final product. 2. Defaults to website stylesheet, no way to configure. This should be adjusted to match the

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-15 Thread Bharath Rupireddy
On Wed, Mar 15, 2023 at 12:27 PM Michael Paquier wrote: > > On Tue, Mar 14, 2023 at 07:05:20PM -0700, Andres Freund wrote: > > It's using ICU, but not a specific collation. The build I linked to is WIP > > hackery to add ICU support to windows CI. Here's the initdb output: > >

Re: Add pg_walinspect function with block info columns

2023-03-15 Thread Michael Paquier
On Wed, Mar 15, 2023 at 12:13:56PM +0530, Bharath Rupireddy wrote: > How about something like the attached? It adds the per-record columns > to pg_get_wal_block_info() avoiding "possibly expensive" joins with > pg_get_wal_records_info(). > > With this, pg_get_wal_records_info() too will be useful

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-15 Thread Michael Paquier
On Tue, Mar 14, 2023 at 07:05:20PM -0700, Andres Freund wrote: > It's using ICU, but not a specific collation. The build I linked to is WIP > hackery to add ICU support to windows CI. Here's the initdb output: >

Re: Add pg_walinspect function with block info columns

2023-03-15 Thread Michael Paquier
On Tue, Mar 14, 2023 at 06:50:15PM -0700, Peter Geoghegan wrote: > On Tue, Mar 14, 2023 at 5:34 PM Melanie Plageman > wrote: >> Well, I think if you only care about the WAL record-level information >> and not the block-level information, having the WAL record information >> denormalized like that

RE: Allow logical replication to copy tables in binary format

2023-03-15 Thread Takamichi Osumi (Fujitsu)
Hi, On Wednesday, March 15, 2023 2:34 PM Amit Kapila wrote: > On Tue, Mar 14, 2023 at 8:50 PM Takamichi Osumi (Fujitsu) > wrote: > > > > On Tuesday, March 14, 2023 8:02 PM Melih Mutlu > wrote: > > (3) copy_table() > > > > + /* > > +* If the publisher version is earlier than

Re: ICU locale validation / canonicalization

2023-03-15 Thread Jeff Davis
On Tue, 2023-03-14 at 10:10 -0700, Jeff Davis wrote: > One loose end is that we really should support language tags like > "und" > in those older versions (54 and earlier). Your commit d72900bded > avoided the problem, but perhaps we should fix it by looking for > "und" > and replacing it with

Re: Add pg_walinspect function with block info columns

2023-03-15 Thread Bharath Rupireddy
On Wed, Mar 15, 2023 at 7:20 AM Peter Geoghegan wrote: > > > But, perhaps you are suggesting a parameter to pg_get_wal_records_info() > > like "with_block_info" or something, which produces the full > > denormalized block + record output? > > I was thinking of something like that, yes -- though

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Amit Kapila
On Wed, Mar 15, 2023 at 11:52 AM Peter Smith wrote: > > == > src/backend/replication/logical/tablesync.c > > 5. > + > + /* > + * If the publisher version is earlier than v14, it COPY command doesn't > + * support the binary option. > + */ > + if (walrcv_server_version(LogRepWorkerWalRcvConn)

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Peter Smith
Here are some review comments for v13-0001 == doc/src/sgml/logical-replication.sgml 1. @@ -241,10 +241,13 @@ types of the columns do not need to match, as long as the text representation of the data can be converted to the target type. For example, you can replicate from a