Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jelte Fennema-Nio
On Fri, 29 Dec 2023 at 19:38, Tom Lane wrote: >> Jelte Fennema-Nio writes: > > 1. Protocol messages are much easier to inspect for connection poolers > > than queries > > Unless you somehow forbid clients from setting GUCs in the old way, > exactly how will that hel

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jelte Fennema-Nio
On Fri, 29 Dec 2023 at 19:32, Jeff Davis wrote: > On Fri, 2023-12-29 at 18:29 +0100, Jelte Fennema-Nio wrote: > > There's definitely still some more work that needs to be done > > (docs for new libpq APIs, protocol version bump, working protocol > > version negotiation). &

Add new protocol message to change GUCs for usage with future protocol-only GUCs

2023-12-29 Thread Jelte Fennema-Nio
Currently the only way to set GUCs from a client is by using SET commands or set them in the StartupMessage. I think it would be very useful to be able to change settings using a message at the protocol level. For the following reasons: 1. Protocol messages are much easier to inspect for

Re: libpq compression (part 3)

2023-12-29 Thread Jelte Fennema-Nio
On Fri, 29 Dec 2023 at 11:02, Andrey M. Borodin wrote: > This patchset allows sending CompressionMethod message, which allows to set > another codec\level picked from the set of negotiated codec sets (during > startup). Did you mean to attach a patchset? I don't see the CompressionMethod

Re: Add PQsendSyncMessage() to libpq

2023-12-29 Thread Jelte Fennema-Nio
On Mon, 13 Nov 2023 at 09:20, Anthonin Bonnefoy wrote: > \syncpipeline > tps = 2607.587877 (without initial connection time) > ... > \endpipeline > \startpipeline > tps = 2290.527462 (without initial connection time) Those are some nice improvements. And I think once this patch is in, it would

Re: Add PQsendSyncMessage() to libpq

2023-12-29 Thread Jelte Fennema-Nio
On Sun, 12 Nov 2023 at 14:37, Anton Kirilov wrote: > Since there haven't been any comments from the other people who have > chimed in on this e-mail thread, I will assume that there is consensus > (we are doing a U-turn with the implementation approach after all), so > here is the updated version

Re: Should we remove -Wdeclaration-after-statement?

2023-12-27 Thread Jelte Fennema-Nio
On Wed, 27 Dec 2023 at 16:05, Tom Lane wrote: > This has already been debated, and the conclusion was that we would > stick to the existing style for consistency reasons. I looked through the archives quite a bit, but I couldn't find any conclusive debate about the current declaration style.

Should we remove -Wdeclaration-after-statement?

2023-12-27 Thread Jelte Fennema-Nio
Postgres currently requires all variables to be declared at the top of the function, because it specifies -Wdeclaration-after-statement. One of the reasons that we had this warning was because C89 required this style of declaration. Requiring it everywhere made backporting easier, since some of

Re: A tiny improvement of psql

2023-12-26 Thread Jelte Fennema-Nio
On Tue, 26 Dec 2023 at 22:45, Vik Fearing wrote: > It is kind of something we control. Per the psql docs, setting > > HISTCONTROL=ignoredups > > will do the trick. Yeah, the easiest "fix" (that I know of) for a user is to set HISTCONTROL in ~/.psqlrc to ignoredups using: \set HISTCONTROL

Re: Add --check option to pgindent

2023-12-21 Thread Jelte Fennema-Nio
On Tue, 19 Dec 2023 at 17:54, Tristan Partin wrote: > I was envisioning something along the lines of: > > pgindent --check --diff > patches.txt > status=$? > patch with manual parsing > exit $status Okay, I got a working version. And I updated the pre-commit hook

Re: libpq compression (part 3)

2023-12-20 Thread Jelte Fennema-Nio
Thanks for working on this! One thing I'm wondering: should it be possible for the client to change the compression it wants mid-connection? I can think of some scenarios where that would be useful to connection poolers: if a pooler does plain forwarding of the compressed messages, then it would

Re: backtrace_on_internal_error

2023-12-20 Thread Jelte Fennema-Nio
On Wed, 20 Dec 2023 at 11:30, Andres Freund wrote: > Tom's patch imo doesn't really introduce anything really new - we already deal > with EOF that way in other places. And it's how the standard APIs deal with > the issue. I'd not design it this way on a green field, but given the current > state

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2023-12-20 Thread Jelte Fennema-Nio
On Thu, 14 Dec 2023 at 13:57, Jelte Fennema-Nio wrote: > I changed all the places that were not adhering to those spellings. It seems I forgot a /g on my sed command to do this so it turned out I missed one that caused the test to fail to compile... Attached is a fixed version. I also upda

Support a wildcard in backtrace_functions

2023-12-20 Thread Jelte Fennema-Nio
a value in the list, then a > backtrace is written to the server log together with the error > message. This can be used to debug specific areas of the > source code. From dacb18f62e7794d8165de80b811c994d384dc060 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio Date: Wed, 20 Dec 2023 11:41

Re: backtrace_on_internal_error

2023-12-20 Thread Jelte Fennema-Nio
On Tue, 19 Dec 2023 at 17:12, Robert Haas wrote: > On Fri, Dec 8, 2023 at 1:34 PM Andres Freund wrote: > > Oh, very much agreed. But I suspect we won't quickly do the same for > > out-of-core extensions... > > I feel like this is a problem that will sort itself out just fine. The > rules about

Re: backtrace_on_internal_error

2023-12-20 Thread Jelte Fennema-Nio
On Sun, 10 Dec 2023 at 00:14, Tom Lane wrote: > I'm not actually sure that the fe-secure.c part of v3-0002 is > necessary, because it's guarding plain recv(2) which really shouldn't > return -1 without setting errno. Still, it's a pretty harmless > addition. v3-0002 seems have a very similar

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-12-19 Thread Jelte Fennema-Nio
On Tue, 19 Dec 2023 at 16:52, Nathan Bossart wrote: > I'm not sure we should proceed with rewriting most/all eligible foreach > loops. I think it's fine to use the new macros in new code or to update > existing loops in passing when changing nearby code, but rewriting > everything likely just

Re: Add --check option to pgindent

2023-12-19 Thread Jelte Fennema-Nio
On Mon, 18 Dec 2023 at 22:18, Tristan Partin wrote: > Here is an additional patch which implements the behavior you described. > The first patch is just Daniel's squashed version of my patches. I think we'd still want the early exit behaviour when only --check is provided. No need to spend time

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-12-19 Thread Jelte Fennema-Nio
On Tue, 19 Dec 2023 at 11:59, vignesh C wrote: > I noticed that this change can be done in several other places too. My guess would be that ~90% of all existing foreach loops in the codebase can be easily rewritten (and simplified) using these new macros. So converting all of those would likely

Re: Add --check option to pgindent

2023-12-18 Thread Jelte Fennema-Nio
On Mon, 18 Dec 2023 at 17:50, Tristan Partin wrote: > I could propose something. It would help if I had an example of such > a tool that already exists. Basically the same behaviour as what you're trying to add now for --check, only instead of printing the diff it would actually change the files

Re: psql JSON output format

2023-12-18 Thread Jelte Fennema-Nio
On Mon, 18 Dec 2023 at 16:38, Christoph Berg wrote: > We'd want both patches even if they do the same thing on two different > levels, I'd say. Makes sense. One thing I was still wondering is if it wouldn't be easier to wrap all queries in "copy (select whatever) to stdout (format json)"

Re: Add --check option to pgindent

2023-12-18 Thread Jelte Fennema-Nio
On Mon, 18 Dec 2023 at 17:14, Jelte Fennema-Nio wrote: > One thing I'm wondering: When both --check and --diff are passed, > should pgindent still early exit with 2 on the first incorrectly > formatted file? Or should it show diffs for all failing files? I'm > leaning towards the l

Re: Add --check option to pgindent

2023-12-18 Thread Jelte Fennema-Nio
On Mon, 18 Dec 2023 at 13:42, Daniel Gustafsson wrote: > I think this is pretty much ready to go, the attached v4 squashes the changes > and fixes the man-page which also needed an update. The referenced Wiki page > will need an edit or two after this goes in, but that's easy enough. One thing

Re: psql JSON output format

2023-12-18 Thread Jelte Fennema-Nio
On Mon, 18 Dec 2023 at 15:56, Christoph Berg wrote: > I noticed psql was lacking JSON formatting of query results which I > need for a follow-up patch. This seems useful to me too, but my usecases would also be solved (and possibly better solved) by adding JSON support to COPY as proposed here:

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-12-18 Thread Jelte Fennema-Nio
and moved modifying foreach_delete_current and foreach_current_index to their own commit. On Thu, 14 Dec 2023 at 16:54, Jelte Fennema-Nio wrote: > > On Fri, 1 Dec 2023 at 05:20, Nathan Bossart wrote: > > Could we simplify it with something like the following? > > Great s

Re: Add --check option to pgindent

2023-12-15 Thread Jelte Fennema-Nio
This part of the first patch seems incorrect, i.e. same condition in the if as elsif - if ($silent_diff) + if ($check) + { + print show_diff($source, $source_filename); + exit 2; + } + elsif ($check) { exit 2; } On

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-12-14 Thread Jelte Fennema-Nio
On Fri, 1 Dec 2023 at 05:20, Nathan Bossart wrote: > Could we simplify it with something like the following? Great suggestion! Updated the patchset accordingly. This made it also easy to change the final patch to include the automatic scoped declaration logic for all of the new macros. I quite

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2023-12-14 Thread Jelte Fennema-Nio
On Mon, 13 Nov 2023 at 03:39, Thomas Munro wrote: > We follow the common en-US usage: "canceled", "canceling" but > "cancellation". Blame Webstah et al. I changed all the places that were not adhering to those spellings. There were also a few of such places in parts of the codebase that these

Call pqPipelineFlush from PQsendFlushRequest

2023-11-07 Thread Jelte Fennema-Nio
In pipeline mode after queuing a message to be sent we would flush the buffer if the size of the buffer passed some threshold. The only message type that we didn't do that for was the Flush message. This addresses that oversight. I noticed this discrepancy while reviewing the

Re: Add PQsendSyncMessage() to libpq

2023-11-07 Thread Jelte Fennema-Nio
On Fri, 28 Apr 2023 at 14:07, Robert Haas wrote: > I wonder whether this is the naming that we want. The two names are > significantly different. Something like PQpipelineSendSync() would be > more similar. > > I also wonder, really even more, whether it would be better to do > something like

Re: Report search_path value back to the client.

2023-11-03 Thread Jelte Fennema-Nio
For completeness attached is a tiny patch implementing this, so this thread can be added to the commit fest. v1-0001-Mark-search_path-as-GUC_REPORT.patch Description: Binary data

Re: Report search_path value back to the client.

2023-11-03 Thread Jelte Fennema-Nio
I wanted to revive this thread, since it's by far one of the most common foot guns that people run into with PgBouncer. Almost all session level SET commands leak across transactions, but SET search_path is by far the one with the biggest impact when it is not the setting that you expect. As well

Re: libpq async connection and multiple hosts

2023-10-26 Thread Jelte Fennema
On Thu, 26 Oct 2023 at 03:31, Daniele Varrazzo wrote: > The goal here was only non-blocking name resolution. Ahaini understand we > should do is to split on the hosts for sync connections too, shuffle if > requested, and make separate connection attempts. If you pack the resolved addresses in

Re: libpq async connection and multiple hosts

2023-10-25 Thread Jelte Fennema
On Wed, 25 Oct 2023 at 18:54, Daniele Varrazzo wrote: > - connect_timeout > - multiple host, hostaddr, port > - load_balance_hosts=random > > Does this list sound complete? I think you'd also want to resolve the hostnames to IPs yourself and iterate over those one-by-one. Otherwise if the first

Re: libpq async connection and multiple hosts

2023-10-25 Thread Jelte Fennema
On Wed, 25 Oct 2023 at 17:03, Daniele Varrazzo wrote: > However, ISTM that connecting to multiple hosts is not supported > either. I have a couple of issues I am looking into in psycopg 3: > > - https://github.com/psycopg/psycopg/issues/602 > - https://github.com/psycopg/psycopg/issues/674

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-25 Thread Jelte Fennema
On Wed, 25 Oct 2023 at 13:52, Alvaro Herrera wrote: > Looking at for_each_ptr() I think it may be cleaner to follow > palloc_object()'s precedent and make it foreach_object() instead (I have > no love for the extra underscore, but I won't object to it either). And > like foreach_node, have it

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-25 Thread Jelte Fennema
From: Jelte Fennema-Nio Date: Tue, 24 Oct 2023 17:13:20 +0200 Subject: [PATCH v4 2/2] Use new for_each_xyz macros in a few places This starts using each of the newly added for_each_xyz macros in at least one place. This shows how they improve readability by reducing the amount of boilerplate code

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-25 Thread Jelte Fennema
On Wed, 25 Oct 2023 at 04:55, David Rowley wrote: > With foreach(), we commonly do "if (lc == NULL)" at the end of loops > as a way of checking if we did "break" to terminate the loop early. Afaict it's done pretty infrequently. The following crude attempt at an estimate estimates it's only done

Re: Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-24 Thread Jelte Fennema
On Tue, 24 Oct 2023 at 18:47, Nathan Bossart wrote: > BTW after applying your patches, initdb began failing with the following > for me: > > TRAP: failed Assert("n >= 0 && n < list->length"), File: "list.c", > Line: 770, PID: 902807 Oh oops... That was an off by one error in the

Add new for_each macros for iterating over a List that do not require ListCell pointer

2023-10-24 Thread Jelte Fennema
Many usages of the foreach macro in the Postgres codebase only use the ListCell variable to then get its value. This adds macros that simplify iteration code for that very common use case. Instead of passing a ListCell you can pass a variable of the type of its contents. This IMHO improves

Re: prevent non-superuser terminate bgworker running as superuser

2023-10-19 Thread Jelte Fennema
This seems like it should even be considered a security honestly. On Thu, 19 Oct 2023, 19:49 Hemanth Sandrana, wrote: > Hi All, > > Currently, BackgroundWorker connected to a database by calling > BackgroundWorkerInitializeConnection with username as NULL can be > terminated by non-superuser

Re: run pgindent on a regular basis / scripted manner

2023-10-18 Thread Jelte Fennema
On Wed, 18 Oct 2023 at 06:40, David Rowley wrote: > How many of the committers who have broken koel are repeat offenders? I just checked the commits and there don't seem to be real repeat offenders. The maximum number of times someone broke koel since its inception is two. That was the case for

Re: run pgindent on a regular basis / scripted manner

2023-10-18 Thread Jelte Fennema
On Tue, 17 Oct 2023 at 23:01, Magnus Hagander wrote: > And unless we're only enforcing it on master, we'd also need to make > provisions for different versions of it on different branches, I > think? Only enforcing on master sounds fine to me, that's what koel is doing too afaik. In practice

Re: run pgindent on a regular basis / scripted manner

2023-10-17 Thread Jelte Fennema
On Tue, 17 Oct 2023 at 18:53, Maciek Sakrejda wrote: > Git push does have an --atomic flag to treat the entire push as a single > operation. I decided to play around a bit with server hooks. Attached is a git "update" hook that rejects pushes to the master branch when the new HEAD of master

Re: run pgindent on a regular basis / scripted manner

2023-10-17 Thread Jelte Fennema
On Tue, 17 Oct 2023 at 17:47, Peter Geoghegan wrote: > Once you figure all that out, you're still obligated to hand-polish > typedefs.list to be consistent with whatever Bruce's machine's copy of > objdump does (or is it Tom's?). You need to sort the entries so they > kinda look like they

Re: run pgindent on a regular basis / scripted manner

2023-10-17 Thread Jelte Fennema
On Tue, 17 Oct 2023 at 16:04, Robert Haas wrote: > What I really dislike about the current situation is that > it's doubling down on the idea that committers have to be perfect and > get everything right every time. Turns out, that's hard to do. If not, > why do people keep screwing things up?

Re: run pgindent on a regular basis / scripted manner

2023-10-17 Thread Jelte Fennema
On Tue, 17 Oct 2023 at 16:23, Tom Lane wrote: > > Or we could have a server-side hook that will refuse > > the misindented commit, with some kind of override for emergency > > situations. > > Even though I'm in the camp that would like the tree correctly > indented at all times, I remain very

Re: run pgindent on a regular basis / scripted manner

2023-10-17 Thread Jelte Fennema
On Tue, 17 Oct 2023 at 04:57, Michael Paquier wrote: > I see an extra reason with not doing that: this increases the > difficulty when it comes to send and maintain patches to the lists and > newcomers would need to learn more tooling. I don't think that we > should make that more complicated

Re: run pgindent on a regular basis / scripted manner

2023-10-17 Thread Jelte Fennema
On Tue, 17 Oct 2023 at 03:23, Peter Geoghegan wrote: > My main objection to the new policy is that it's not quite clear what > process I should go through in order to be 100% confident that koel > won't start whining (short of waiting around for koel to whine). I > know how to run pgindent, of

Re: Request for comment on setting binary format output per session

2023-10-09 Thread Jelte Fennema
On Mon, 9 Oct 2023 at 21:08, Dave Cramer wrote: > So if we use . would it be possible to have something like > which represents a set of well known types? > My goal here is to reduce the overhead of naming all the types the client > wants in binary. The list of well known types is pretty

Re: Request for comment on setting binary format output per session

2023-10-09 Thread Jelte Fennema
On Mon, 9 Oct 2023 at 22:25, Jeff Davis wrote: > We absolutely would > need to update the documentation, and clients (like psql) really should > be updated. +1 > > I think one > > could conclude on these facts either that (a) client_encoding is fine > > and the problems with controlling

Re: Request for comment on setting binary format output per session

2023-10-09 Thread Jelte Fennema
On Mon, 9 Oct 2023 at 21:00, Robert Haas wrote: > ...but then... > > > With Citus the same user defined type can have > > different OIDs on each of the servers in the cluster. > > I realize that your intention here may be to say that this is not an > *additional* problem but one we have already.

Re: UUID v7

2023-10-09 Thread Jelte Fennema
On Mon, 9 Oct 2023 at 18:46, Nick Babadzhanian wrote: > > On Thu, 31 Aug 2023 at 23:10, Andrey M. Borodin wrote: > > Well, as far as I know, RFC discourages extracting timestamps from UUIDs. > > But we still can have such functions...maybe as an extension? > > Do you know of any reason for

Re: Request for comment on setting binary format output per session

2023-10-09 Thread Jelte Fennema
On Fri, 6 Oct 2023 at 13:11, Peter Eisentraut wrote: > > On 04.10.23 20:30, Dave Cramer wrote: > > We need to > > figure out what is the right way to globally identify types, like > > either > > by fully-qualified name, by base name, some combination, how does it > > work with

Re: Request for comment on setting binary format output per session

2023-10-09 Thread Jelte Fennema
On Wed, 4 Oct 2023 at 21:10, Robert Haas wrote: > > On Wed, Oct 4, 2023 at 10:17 AM Peter Eisentraut wrote: > > I think intuitively, this facility ought to work like client_encoding. > > I hadn't really considered client_encoding as a precedent for this > setting. A lot of my discomfort with the

Re: Support prepared statement invalidation when result types change

2023-09-18 Thread Jelte Fennema-Nio
@Euler thanks for the review. I addressed the feedback. On Fri, 15 Sept 2023 at 01:41, Andy Fan wrote: > What if a client has *cached* an old version of RowDescription > and the server changed it to something new and sent resultdata > with the new RowDescription. Will the client still be able

Re: Support prepared statement invalidation when result types change

2023-09-12 Thread Jelte Fennema
When running the Postgres JDBC tests with this patchset I found dumb mistake in my last patch where I didn't initialize the contents of orig_params correctly. This new patchset fixes that. v4-0001-Support-prepared-statement-invalidation-when-resu.patch Description: Binary data

Re: proposal: psql: show current user in prompt

2023-09-11 Thread Jelte Fennema
On Mon, 11 Sept 2023 at 13:59, Jelte Fennema wrote: > I think that would make the client code even simpler than it is now. To be clear, I'm not saying we should do this. There's benefits to using a dedicated new message type too (e.g. clearer errors if a proxy like pgbouncer does not understand it).

Re: proposal: psql: show current user in prompt

2023-09-11 Thread Jelte Fennema
@Tom and @Robert, since you originally suggested extending the protocol for this, I think some input from you on the protocol design would be quite helpful. BTW, this protocol extension is the main reason I personally care for this patch, because it would allow PgBouncer to ask for updates on

Re: proposal: psql: show current user in prompt

2023-09-11 Thread Jelte Fennema
On Fri, 8 Sept 2023 at 21:08, Pavel Stehule wrote: > ok changed - there is minor problem - almost all PQ functions are of int > type, but ProtocolVersion is uint Your current implementation requires using the PG_PROTOCOL macros to compare versions. But clients cannot currently use those macros

Re: PSQL error: total cell count of XXX exceeded

2023-09-11 Thread Jelte Fennema
On Mon, 11 Sept 2023 at 08:51, Hongxu Ma wrote: > > I created a patch to fix it. > Really appreciate to anyone can help to review it. > Thanks. I think "product" as a variable name isn't very descriptive. Let's call it total_cells (or something similar instead). Other than that I think it's a

Re: Support prepared statement invalidation when result types change

2023-09-08 Thread Jelte Fennema
that issue. [1]: https://github.com/pgbouncer/pgbouncer/pull/845#discussion_r1309454695 [2]: https://www.postgresql.org/message-id/flat/CA%2Bmi_8YAGf9qibDFTRNKgaTwaBa1OUcteKqLAxfMmKFbo3GHZg%40mail.gmail.com From 3bfc47cfa333158c2e0a2e0603e311d141fc473b Mon Sep 17 00:00:00 2001 From: Jelte Fennema Date

Re: proposal: psql: show current user in prompt

2023-09-05 Thread Jelte Fennema
On Tue, 5 Sept 2023 at 05:50, Pavel Stehule wrote: > I prefer to introduce a new function - it is ten lines of code. The form is > not important - it can be a full number or minor number. It doesn't matter I > think. But my opinion in this area is not strong, and I like to see feedback > from

Re: pg_basebackup: Always return valid temporary slot names

2023-09-05 Thread Jelte Fennema
I modified the PgBouncer PR to always set the sign bit to 0. But I still I think it makes sense to also address this in pg_basebackup. On Tue, 5 Sept 2023 at 12:21, Jelte Fennema wrote: > > On Tue, 5 Sept 2023 at 11:39, Daniel Gustafsson wrote: > > > > > On 5 Sep 2023, at

Re: pg_basebackup: Always return valid temporary slot names

2023-09-05 Thread Jelte Fennema
On Tue, 5 Sept 2023 at 11:39, Daniel Gustafsson wrote: > > > On 5 Sep 2023, at 09:09, Nishant Sharma > > wrote: > > > With this, I was thinking, isn't this a problem of pgbouncer filling > > be_pid with random bits? Maybe it should have filled the be_pid > > with a random positive integer

Re: proposal: psql: show current user in prompt

2023-09-04 Thread Jelte Fennema
On Sun, 3 Sept 2023 at 20:58, Pavel Stehule wrote: > here is an try Overall it does what I had in mind. Below a few suggestions: +int +PQprotocolSubversion(const PGconn *conn) Ugh, it's quite annoying that the original PQprotocolVersion only returns the major version and thus we need this new

Re: proposal: psql: show current user in prompt

2023-09-03 Thread Jelte Fennema
On Sun, 3 Sept 2023 at 08:24, Pavel Stehule wrote: > My personal feeling from this area is that the protocol design is done, but > it is not implemented on libpq level. My feelings can be wrong. The protocol > number is hardcoded in libpq, so I cannot change it from the client side. No, I

Re: Should we use MemSet or {0} for struct initialization?

2023-09-01 Thread Jelte Fennema
On Fri, 1 Sept 2023 at 15:25, John Naylor wrote: > On Fri, Sep 1, 2023 at 7:48 PM Jelte Fennema wrote: > > The C standard says: > > > When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that

Re: Should we use MemSet or {0} for struct initialization?

2023-09-01 Thread Jelte Fennema
On Thu, 31 Aug 2023 at 13:35, Junwang Zhao wrote: > > > If the struct has padding or aligned, {0} only guarantee the struct > > > members initialized to 0, while memset sets the alignment/padding > > > to 0 as well, but since we will not access the alignment/padding, so > > > they give the same

pg_basebackup: Always return valid temporary slot names

2023-08-31 Thread Jelte Fennema
With PgBouncer in the middle PQbackendPID can return negative values due to it filling all 32 bits of the be_pid with random bits. When this happens it results in pg_basebackup generating an invalid slot name (when no specific slot name is passed in) and thus throwing an error like this:

Re: Allow specifying a dbname in pg_basebackup connection string

2023-08-31 Thread Jelte Fennema
Attached is a new version with some slightly updated wording in the docs v4-0001-Allow-specifying-a-dbname-in-pg_basebackup-connec.patch Description: Binary data

Re: Allow specifying a dbname in pg_basebackup connection string

2023-08-30 Thread Jelte Fennema
On Wed, 30 Aug 2023 at 01:01, Jim Jones wrote: > However, pgbouncer closes with a segmentation fault, so I couldn't test the > result of pg_basebackup itself - but I guess it isn't the issue here. Oops it indeed seemed like I made an unintended change when handling database names that did not

Re: Allow specifying a dbname in pg_basebackup connection string

2023-08-29 Thread Jelte Fennema
On Mon, 28 Aug 2023 at 23:50, Tristen Raab wrote: > I've reviewed your patch and it applies and builds without error. When > testing this patch I was slightly confused as to what its purpose was, after > testing it I now understand. Initially, I thought this was a change to add >

Re: proposal: psql: show current user in prompt

2023-08-29 Thread Jelte Fennema
On Mon, 28 Aug 2023 at 15:00, Pavel Stehule wrote: + minServerMajor = 1600; + serverMajor = PQserverVersion(pset.db) / 100; Instead of using the server version, we should instead use the protocol version negotiation that's provided by the

Re: Support prepared statement invalidation when result types change

2023-08-29 Thread Jelte Fennema
On Tue, 29 Aug 2023 at 11:29, jian he wrote: > regression=# CREATE TEMP TABLE pcachetest AS SELECT * FROM int8_tbl; > SELECT 5 > regression=# PREPARE prepstmt2(bigint) AS SELECT * FROM pcachetest > WHERE q1 = $1;' > PREPARE > regression=# alter table pcachetest rename q1 to x; > ALTER TABLE >

Re: Support prepared statement invalidation when result types change

2023-08-28 Thread Jelte Fennema
On Mon, 28 Aug 2023 at 11:27, jian he wrote: > With parameters, it also works, only a tiny issue with error reporting. > > prepstmt2 | PREPARE prepstmt2(bigint) AS SELECT * FROM pcachetest > WHERE q1 = $1; | {bigint}| {bigint,bigint,bigint} > ERROR: column "q1" does not exist at

Re: Support prepared statement invalidation when result types change

2023-08-28 Thread Jelte Fennema
On Mon, 28 Aug 2023 at 15:05, Konstantin Knizhnik wrote: > The following assignment of format is not corrects: > > It has to be copied as below: > > portal->formats = (int16 *) > MemoryContextAlloc(portal->portalContext, > natts * sizeof(int16)); >

Support prepared statement invalidation when result types change

2023-08-25 Thread Jelte Fennema
/postgrex-errors-with-cached-plan-must-not-change-result-type-during-migration/51235/2 [8]: https://github.com/rails/rails/issues/12330 From b1a58082f2b226b37d237580e33b52438d480f48 Mon Sep 17 00:00:00 2001 From: Jelte Fennema Date: Fri, 25 Aug 2023 17:09:38 +0200 Subject: [PATCH v1 1/2] Support

Would it be possible to backpatch Close support in libpq (28b5726) to PG16?

2023-08-15 Thread Jelte Fennema
Hi, I know that this will probably get a staunch "No" as an answer, but... I'm still going to ask: Would it be possible to backport 28b5726 to the PG16 branch? Even though it's clearly a new feature? I'm working on named prepared statement support in PgBouncer:

Re: run pgindent on a regular basis / scripted manner

2023-08-11 Thread Jelte Fennema
On Fri, 11 Aug 2023 at 23:00, Peter Geoghegan wrote: > I'm starting to have doubts about this policy. There have now been > quite a few follow-up "fixes" to indentation issues that koel > complained about. I think one thing that would help a lot in reducing the is for committers to set up the

Re: proposal: psql: show current user in prompt

2023-08-10 Thread Jelte Fennema
On Thu, 10 Aug 2023 at 14:44, Pavel Stehule wrote: > čt 10. 8. 2023 v 14:05 odesílatel Jelte Fennema napsal: >> That it is not rolled-back >> in a case like this? >> >> BEGIN; >> \set PROMPT '%N' >> ROLLBACK; > > > surely not. > > \s

Re: proposal: psql: show current user in prompt

2023-08-10 Thread Jelte Fennema
On Tue, 8 Aug 2023 at 07:20, Pavel Stehule wrote: > The reason why I implemented separate flow is usage from psql and > independence of transaction state. It is used for the \set command, that is > non-transactional, not SQL. If I inject this message to some other flow, I > lose this

Re: proposal: psql: show current user in prompt

2023-07-31 Thread Jelte Fennema
On Mon, 24 Jul 2023 at 21:16, Pavel Stehule wrote: > I don't understand how it can be possible to do it without. I need to > process possible errors, and then I need to read and synchronize protocol. I > didn't inject > this feature to some oher flow, so I need to implement a complete process.

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2023-07-17 Thread Jelte Fennema
Rebased again to resolve some conflicts v22-0003-Add-non-blocking-version-of-PQcancel.patch Description: Binary data v22-0002-Return-2-from-pqReadData-on-EOF.patch Description: Binary data v22-0004-Start-using-new-libpq-cancel-APIs.patch Description: Binary data

Re: Allow specifying a dbname in pg_basebackup connection string

2023-07-05 Thread Jelte Fennema
On Wed, 5 Jul 2023 at 20:09, Thom Brown wrote: > Okay, understood. In that case, please remember to write changes to > the pg_basebackup docs page explaining how the dbname value is ignored I updated the wording in the docs for pg_basebackup and pg_receivewal. They now clarify that Postgres

Re: Allow specifying a dbname in pg_basebackup connection string

2023-07-05 Thread Jelte Fennema
On Wed, 5 Jul 2023 at 16:01, Euler Taveira wrote: > One of the PgBouncer's missions is to be a transparent proxy. > > Sometimes you cannot reach out the database directly due to a security policy. Indeed the transparent proxy use case is where replication through pgbouncer makes sense. There's

Re: Deleting prepared statements from libpq.

2023-07-03 Thread Jelte Fennema
@Michael is anything else needed from my side? If not, I'll mark the commitfest entry as "Ready For Committer".

Allow specifying a dbname in pg_basebackup connection string

2023-07-03 Thread Jelte Fennema
Normally it doesn't really matter which dbname is used in the connection string that pg_basebackup and other physical replication CLI tools use. The reason being, that physical replication does not work at the database level, but instead at the server level. So you will always get the data for all

Re: Deleting prepared statements from libpq.

2023-06-23 Thread Jelte Fennema
On Fri, 23 Jun 2023 at 05:59, Michael Paquier wrote: > [...] > res = PQgetResult(conn); > if (res == NULL) > - pg_fatal("expected NULL result"); > + pg_fatal("expected non-NULL result"); > > This should check for the NULL-ness of the result returned for > PQclosePrepared()

Re: Support logical replication of DDLs

2023-06-21 Thread Jelte Fennema
it might be useful to look at the patch with Citus its logic for some inspiration/copying things. I re-attached that patch here for ease of finding it. From ddb375afc74339bd0eaf0c272d06805637fd85cc Mon Sep 17 00:00:00 2001 From: Jelte Fennema Date: Mon, 5 Jun 2023 12:32:12 +0200 Subject: [PATCH v1

Re: Deleting prepared statements from libpq.

2023-06-20 Thread Jelte Fennema
On Tue, 20 Jun 2023 at 06:18, Michael Paquier wrote: > The amount of duplication between the describe and close paths > concerns me a bit. Should PQsendClose() and PQsendDescribe() be > merged into a single routine (say PQsendCommand), that uses a message > type for pqPutMsgStart and a

Re: Deleting prepared statements from libpq.

2023-06-19 Thread Jelte Fennema
On Mon, 19 Jun 2023 at 14:17, jian he wrote: > I am not sure the following two following function comments are right They were incorrect indeed. Attached is a patch with those two updated. On Mon, 19 Jun 2023 at 14:17, jian he wrote: > > On Mon, Jun 19, 2023 at 5:50 PM Jelte Fenne

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2023-06-19 Thread Jelte Fennema
I noticed that cfbot was unable to run tests due to some rebase conflict. It seems the pgindent changes from patch 1 have now been made. So adding the rebased patches without patch 1 now to unblock cfbot. v21-0002-Return-2-from-pqReadData-on-EOF.patch Description: Binary data

Re: Deleting prepared statements from libpq.

2023-06-19 Thread Jelte Fennema
On Mon, 19 Jun 2023 at 11:44, Jelte Fennema wrote: > Done Now with the actual attachment. PS. Another connection pooler (PgCat) now also supports prepared statements, but only using Close not DEALLOCATE: https://postgresml.org/blog/making-postgres-30-percent-faster-in-production F

Re: Deleting prepared statements from libpq.

2023-06-19 Thread Jelte Fennema
On Mon, 19 Jun 2023 at 04:52, jian he wrote: > > /* Now that it's closed we should get an error when describing */ > > res = PQdescribePortal(conn, "cursor_one"); > > if (PQresultStatus(res) != PGRES_FATAL_ERROR) > > pg_fatal("expected COMMAND_OK, got %s", PQresStatus(PQresultStatus(res))); >

Re: Deleting prepared statements from libpq.

2023-06-19 Thread Jelte Fennema
On Mon, 19 Jun 2023 at 01:57, Michael Paquier wrote: > +static int > +PQsendClose(PGconn *conn, char close_type, const char *close_target) > > Could it be better for this code path to issue an error if using a > non-supported close_type rather than sending it? Okay, you are > consistent with

Re: Deleting prepared statements from libpq.

2023-06-18 Thread Jelte Fennema
On Sat, 17 Jun 2023 at 15:34, jian he wrote: > I failed to link it. I don't know why. Sorry about that. I attached a new patch that allows linking to the new functions (I forgot to add the functions to exports.txt). This new patch also adds some basic tests for these new functions.

Re: Deleting prepared statements from libpq.

2023-06-16 Thread Jelte Fennema
On Fri, 16 Jun 2023 at 16:26, Craig Ringer wrote: > Nobody's implemented it. > > A patch to add PQclosePrepared and PQsendClosePrepared would be welcome. At > least, I think so... This might have been a pretty old thread. But I just took it upon me to implement these functions (or well I mostly

Re: run pgindent on a regular basis / scripted manner

2023-06-15 Thread Jelte Fennema
On Sat, 22 Apr 2023 at 13:42, Andrew Dunstan wrote: > Perhaps we should start with a buildfarm module, which would run pg_indent > --show-diff. That would only need to run on one animal, so a failure wouldn't > send the whole buildfarm red. This would be pretty easy to do. Just to be clear on

Re: Adding SHOW CREATE TABLE

2023-06-05 Thread Jelte Fennema
On Thu, 1 Jun 2023 at 18:57, Kirk Wolak wrote: > Can this get turned into a Patch? Were you offering this code up for others > (me?) to pull, and work into a patch? > [If I do the patch, I am not sure it gives you the value of reducing what > CITUS has to maintain. But it dawns on > me that

<    1   2   3   4   5   6   >