Re: libpq compression (part 3)

2024-05-21 Thread Jelte Fennema-Nio
On Mon, 20 May 2024 at 21:42, Jacob Champion wrote: > As Andrey points out, there was prior work done that started to take > this into account. I haven't reviewed it to see how good it is -- and > I think there are probably many use cases in which queries and tables > contain both private and

Re: Multiple startup messages over the same connection

2024-05-19 Thread Jelte Fennema-Nio
On Sat, 18 May 2024 at 23:10, Vladimir Churyukin wrote: > I guess the process of passing control from child processes to the parent > could be a bit tricky for that one, but doable? > Is there anything I'm missing that can be a no-go for this? One seriously difficult/possibly impossible thing

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 21:24, Robert Haas wrote: > I think the > fear that we're going to run into cases where such complex handshaking > is necessary is a major reason why I'm afraid of Jelte's ideas about > ParameterSet: it seems much more opinionated to me than he seems to > think it is. I

Re: libpq compression (part 3)

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 23:10, Jacob Champion wrote: > We're talking about a transport-level option, though -- I thought the > proposal enabled compression before authentication completed? Or has > that changed? I think it would make sense to only compress messages after authentication has

Re: libpq compression (part 3)

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 23:40, Robert Haas wrote: > To be clear, I am not arguing that it should be the receiver's choice. > I'm arguing it should be the client's choice, which means the client > decides what it sends and also tells the server what to send to it. > I'm open to counter-arguments,

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 17:59, Robert Haas wrote: > If they > get attention, it's much more likely to be because somebody saw their > email and wrote back than it is to be because somebody went through > the CommitFest and found their entry and was like "oh, I should review > this". I think this

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 14:19, Joe Conway wrote: > > On 5/16/24 22:26, Robert Haas wrote: > > For example, imagine that the CommitFest is FORCIBLY empty > > until a week before it starts. You can still register patches in the > > system generally, but that just means they get CI runs, not that > >

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 13:39, Robert Haas wrote: > > On Fri, May 17, 2024 at 7:11 AM Tomas Vondra > wrote: > > Yeah, I 100% agree with this. If a patch bitrots and no one cares enough > > to rebase it once in a while, then sure - it's probably fine to mark it > > RwF. But forcing all

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 11:02, Jelte Fennema-Nio wrote: > > On Fri, 17 May 2024 at 10:47, Daniel Gustafsson wrote: > > One way to ensure we capture detail could be if the system would send an > > automated email to the thread summarizing the entry when it's marked

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 10:47, Daniel Gustafsson wrote: > One way to ensure we capture detail could be if the system would send an > automated email to the thread summarizing the entry when it's marked as > "committed"? This sounds great! Especially if Going back from an archived thread to the

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Jelte Fennema-Nio
On Fri, 17 May 2024 at 06:58, Peter Eisentraut wrote: > Yeah, some fine-tuning might be required. But I would be wary of > over-designing too many new states at this point. I think the key idea > is that there ought to be a state that communicates "needs attention > from someone other than

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Jelte Fennema-Nio
On Thu, 16 May 2024 at 18:57, Robert Haas wrote: > Ugh, it's so hard to communicate clearly about this stuff. I didn't > really have any thought that we'd ever try to handle something as > complicated as compression using ParameterSet. Okay, then it's definitely very hard to communicate clearly

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Jelte Fennema-Nio
On Thu, 16 May 2024 at 17:29, Robert Haas wrote: > You're probably not going to like this answer, but I feel like this is > another sign that you're trying to use the protocol extensibility > facilities in the wrong way. In my first reply to the thread, I > proposed having the client send

Re: PostgreSQL 17 Beta 1 release announcement draft

2024-05-16 Thread Jelte Fennema-Nio
On Thu, 16 May 2024 at 03:45, Jonathan S. Katz wrote: > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement > draft. I think we can quickly mention c4ab7da6061 in the COPY paragraph, in some benchmarks it improved perf by close to 2x. Something like this: "has improved

Re: First draft of PG 17 release notes

2024-05-16 Thread Jelte Fennema-Nio
On Thu, 16 May 2024 at 05:48, Andres Freund wrote: > We're having this debate every release. I think the ongoing reticence to note > performance improvements in the release notes is hurting Postgres. > > For one, performance improvements are one of the prime reason users > upgrade. Without them

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-16 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:30, Dave Cramer wrote: > On Fri, 5 Apr 2024 at 12:09, Jelte Fennema-Nio wrote: >> I'll take a look at redesigning the protocol parameter stuff. To work >> with dedicated functions instead. > > +1 It's been a while, but I now actually

Re: Postgres and --config-file option

2024-05-15 Thread Jelte Fennema-Nio
On Wed, 15 May 2024 at 11:49, Peter Eisentraut wrote: > Yeah, some of this is becoming quite unwieldy, if we document and > mention each spelling variant of each option everywhere. > > Maybe if the original problem is that the option --config-file is not > explicitly in the --help output, let's

Re: First draft of PG 17 release notes

2024-05-14 Thread Jelte Fennema-Nio
On Tue, 14 May 2024 at 02:56, Bruce Momjian wrote: > > On Sat, May 11, 2024 at 10:24:39AM -0400, Joe Conway wrote: > > On 5/11/24 09:57, Jelte Fennema-Nio wrote: > > > The buffering change improved performance up to ~40% in some of the > > > benchmarks. The c

Re: JIT compilation per plan node

2024-05-14 Thread Jelte Fennema-Nio
On Tue, 14 May 2024 at 10:19, David Rowley wrote: > Here's a plan where the total cost of a subnode is greater than the > total cost of the top node: Ah I didn't realize it was possible for that to happen. **reads up on plan costs** This actually makes me think that using total_cost of the

Re: JIT compilation per plan node

2024-05-14 Thread Jelte Fennema-Nio
On Tue, 20 Feb 2024 at 06:38, David Rowley wrote: > > On Tue, 20 Feb 2024 at 18:31, Tom Lane wrote: > > FWIW, I seriously doubt that an extra walk of the plan tree is even > > measurable compared to the number of cycles JIT compilation will > > expend if it's called. So I don't buy your

Re: Direct SSL connection with ALPN and HBA rules

2024-05-13 Thread Jelte Fennema-Nio
On Mon, 13 May 2024 at 18:14, Robert Haas wrote: > I disagree with Jacob's assertion that sslmode=require has no security > benefits over sslmode=prefer. That seems like the kind of pessimism > that makes people hate security professionals. There have got to be > some attacks that are foreclosed

Re: Direct SSL connection with ALPN and HBA rules

2024-05-13 Thread Jelte Fennema-Nio
On Mon, 13 May 2024 at 13:07, Heikki Linnakangas wrote: > "channel_binding=require sslmode=require" also protects from MITM attacks. Cool, I didn't realize we had this connection option and it could be used like this. But I think there's a few security downsides of channel_binding=require over

Re: Direct SSL connection with ALPN and HBA rules

2024-05-13 Thread Jelte Fennema-Nio
On Mon, 13 May 2024 at 15:38, Heikki Linnakangas wrote: > Here's a patch to implement that. + if (conn->sslnegotiation[0] == 'd' && + conn->sslmode[0] != 'r' && conn->sslmode[0] != 'v') I think these checks should use strcmp instead of checking magic first characters. I see this

Re: Direct SSL connection with ALPN and HBA rules

2024-05-13 Thread Jelte Fennema-Nio
On Sun, 12 May 2024 at 23:39, Heikki Linnakangas wrote: > You might miss that by changing sslnegotiation to 'postgres', or by > removing it altogether, you not only made it compatible with older > server versions, but you also allowed falling back to a plaintext > connection. Maybe you're fine

Re: Direct SSL connection with ALPN and HBA rules

2024-05-11 Thread Jelte Fennema-Nio
On Fri, 10 May 2024 at 15:50, Heikki Linnakangas wrote: > New proposal: > > - Remove the "try both" mode completely, and rename "requiredirect" to > just "direct". So there would be just two modes: "postgres" and > "direct". On reflection, the automatic fallback mode doesn't seem very > useful.

Re: First draft of PG 17 release notes

2024-05-11 Thread Jelte Fennema-Nio
On Fri, 10 May 2024 at 23:31, Tom Lane wrote: > > Bruce Momjian writes: > > I looked at both of these. In both cases I didn't see why the user > > would need to know these changes were made: > > I agree that the buffering change is not likely interesting, but > the fact that you can now

Re: First draft of PG 17 release notes

2024-05-10 Thread Jelte Fennema-Nio
On Thu, 9 May 2024 at 06:04, Bruce Momjian wrote: > > I have committed the first draft of the PG 17 release notes; you can > see the results here: > > https://momjian.us/pgsql_docs/release-17.html Great work! There are two commits that I think would benefit from being listed (but maybe

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-06 Thread Jelte Fennema-Nio
On Tue, 23 Apr 2024 at 19:39, Jacob Champion wrote: > > On Mon, Apr 22, 2024 at 2:20 PM Jelte Fennema-Nio wrote: > > 1. I strongly believe minor protocol version bumps after the initial > > 3.1 one can be made painless for clients/poolers (so the ones to > > 3.2/3.3/etc)

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-05-06 Thread Jelte Fennema-Nio
On Tue, 23 Apr 2024 at 17:03, Robert Haas wrote: > If a client doesn't know about encrypted columns and sets that > bit at random, that will break things, and formally I think that's a > risk, because I don't believe we document anywhere that you shouldn't > set unused bits in the format mask.

Re: Reducing the log spam

2024-05-02 Thread Jelte Fennema-Nio
On Thu, 2 May 2024 at 13:08, Jelte Fennema-Nio wrote: > 2. Change the newly added check in errcode() to only set > output_to_server to false when IsLogicalWorker() returns false. Actually a third, and probably even better solution would be to only apply this new GUC to non-background

Re: Reducing the log spam

2024-05-02 Thread Jelte Fennema-Nio
On Thu, 2 May 2024 at 12:47, Laurenz Albe wrote: > Yes. But I'd argue that that is a shortcoming of logical replication: > there should be a ways to get this information via SQL. Having to look into > the log file is not a very useful option. Definitely agreed that accessing the error details

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-22 Thread Jelte Fennema-Nio
able to change your mind. On Thu, 18 Apr 2024 at 21:34, Jelte Fennema-Nio wrote: > When the server supports a lower version than the client, the client > should disable certain features because it gets the > ProtocolVersionNegotiation message. This is also true if we don't bump > th

Re: Support a wildcard in backtrace_functions

2024-04-21 Thread Jelte Fennema-Nio
On Sat, 20 Apr 2024 at 01:19, Michael Paquier wrote: > Removing this GUC and making the backend react by default the same way > as when this GUC was enabled sounds like a sensible route here. This > stuff is useful. I definitely agree it's useful. But I feel like changing the default of the GUC

Re: Support a wildcard in backtrace_functions

2024-04-19 Thread Jelte Fennema-Nio
On Fri, 19 Apr 2024 at 10:58, Peter Eisentraut wrote: > This presupposes that there is consensus about how the future > functionality should look like. This topic has gone through half a > dozen designs over a few months, and I think it would be premature to > randomly freeze that discussion now

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-18 Thread Jelte Fennema-Nio
On Thu, 18 Apr 2024 at 23:36, Jelte Fennema-Nio wrote: > To clarify: My proposed approach is to use a protocol extension > parameter for to enable the new messages that the server can send > (like Peter is doing now). And **in addition to that** gate the new > Bind format type behi

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-18 Thread Jelte Fennema-Nio
On Thu, 18 Apr 2024 at 22:17, Robert Haas wrote: > If we go with Peter's approach, every driver that supports his feature > will work perfectly, and every driver that doesn't will work exactly > as it does today. The risk of breaking anything is as near to zero as > human developers can

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-18 Thread Jelte Fennema-Nio
en still bail out if some >> feature is not supported. > > I don't think a client should ever bail out. They may not support something > but IMO bailing out is not an option. On Thu, 18 Apr 2024 at 21:01, Robert Haas wrote: > On Thu, Apr 18, 2024 at 1:49 PM Jelte F

Re: Transparent column encryption

2024-04-18 Thread Jelte Fennema-Nio
On Thu, 18 Apr 2024 at 18:46, Robert Haas wrote: > With regard to the Bind message, I suggest that we regard the protocol > change as reserving a currently-unused bit in the message to indicate > whether the value is pre-encrypted, without reference to the protocol > extension. It could be legal

Re: Transparent column encryption

2024-04-18 Thread Jelte Fennema-Nio
On Thu, 18 Apr 2024 at 13:25, Peter Eisentraut wrote: > Hopefully, the reason for key rotation is mainly that policies require > key rotation, not that keys get compromised all the time. These key rotation policies are generally in place to reduce the impact of a key compromise by limiting the

Re: Support a wildcard in backtrace_functions

2024-04-18 Thread Jelte Fennema-Nio
On Thu, 18 Apr 2024 at 09:02, Michael Paquier wrote: > > On Fri, Apr 12, 2024 at 09:36:36AM +0900, Michael Paquier wrote: > > log_backtrace speaks a bit more to me as a name for this stuff because > > it logs a backtrace. Now, there is consistency on HEAD as well > > because these GUCs are all

Re: Support a wildcard in backtrace_functions

2024-04-18 Thread Jelte Fennema-Nio
On Thu, 18 Apr 2024 at 10:50, Peter Eisentraut wrote: > Why exactly is this an open item? Is there anything wrong with the > existing feature? The name of the GUC backtrace_on_internal_error is so specific that it's impossible to extend our backtrace behaviour in future releases without adding

Re: Speed up clean meson builds by ~25%

2024-04-17 Thread Jelte Fennema-Nio
On Wed, 17 Apr 2024 at 16:00, Tom Lane wrote: > Break gram.y (say, misspell some token in a production) and > see what happens. The behavior's likely to be timing sensitive > though. Thanks for clarifying. It took me a little while to break gram.y in such a way that I was able to consistently

Re: Speed up clean meson builds by ~25%

2024-04-17 Thread Jelte Fennema-Nio
On Wed, 17 Apr 2024 at 13:41, Peter Eisentraut wrote: > > On 10.04.24 17:33, Tom Lane wrote: > > The immediate question then is do we want to take Jelte's patch > > as a way to ameliorate the pain meanwhile. I'm kind of down on > > it, because AFAICS what would happen if you break the core > >

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Jelte Fennema-Nio
On Mon, 15 Apr 2024 at 19:43, Robert Haas wrote: > > On Sat, Apr 6, 2024 at 6:14 PM Jelte Fennema-Nio wrote: > > I think for clients/drivers, the work would generally be pretty > > minimal. For almost all proposed changes, clients can "support" the > > protocol

Re: psql: Greatly speed up "\d tablename" when not using regexes

2024-04-11 Thread Jelte Fennema-Nio
On Wed, 10 Apr 2024 at 23:51, Tom Lane wrote: > Is it really necessary for Citus' filter to be a security qual rather > than a plain ol' filter condition? No, it's not. I think using security quals simply required the least amount of code (and it worked just fine if you didn't have lots of

Re: psql: Greatly speed up "\d tablename" when not using regexes

2024-04-10 Thread Jelte Fennema-Nio
On Wed, 10 Apr 2024 at 22:11, Tom Lane wrote: > There may be an argument for psql to do what you suggest, > but so far it seems like duplicative complication. > > If there's a case you can demonstrate where "\d foo" doesn't optimize > into an indexscan, we should look into exactly why that's

Re: psql: Greatly speed up "\d tablename" when not using regexes

2024-04-10 Thread Jelte Fennema-Nio
On Wed, 10 Apr 2024 at 20:21, Kirill Reshke wrote: > Do we need to force Collaction here like in other branches? > if (PQserverVersion(conn) >= 12) >appendPQExpBufferStr(buf, " COLLATE pg_catalog.default"); According to the commit and codecomment that introduced the COLLATE, it was

Re: psql: Greatly speed up "\d tablename" when not using regexes

2024-04-10 Thread Jelte Fennema-Nio
On Wed, 10 Apr 2024 at 20:06, Tom Lane wrote: > Really? ISTM this argument is ignoring an optimization the backend > has understood for a long time. Interesting. I didn't know about that optimization. I can't check right now, but probably the COLLATE breaks that optimization.

psql: Greatly speed up "\d tablename" when not using regexes

2024-04-10 Thread Jelte Fennema-Nio
Running "\d tablename" from psql could take multiple seconds when running on a system with 100k+ tables. The reason for this was that a sequence scan on pg_class takes place, due to regex matching being used. Regex matching is obviously unnecessary when we're looking for an exact match. This

Re: Transparent column encryption

2024-04-10 Thread Jelte Fennema-Nio
On Wed, 10 Apr 2024 at 12:13, Peter Eisentraut wrote: > > To kick some things off for PG18, here is an updated version of the > patch for automatic client-side column-level encryption. I only read the docs and none of the code, but here is my feedback on the current design: > (The CEK can't be

Re: Support a wildcard in backtrace_functions

2024-04-10 Thread Jelte Fennema-Nio
On Fri, 22 Mar 2024 at 11:09, Jelte Fennema-Nio wrote: > On Thu, 21 Mar 2024 at 15:41, Jelte Fennema-Nio wrote: > > Attached is a patch that implements this. Since the more I think about > > it, the more I like this approach. > > I now added a 3rd patch to this p

Re: Add trim_trailing_whitespace to editorconfig file

2024-04-09 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 16:58, Jelte Fennema-Nio wrote: > > ISTM that with a small shell script, .editorconfig could be generated > > from .gitattributes? > > Honestly, I don't think building such automation is worth the effort. Okay, I spent the time to add a script to generat

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Jelte Fennema-Nio
On Mon, 8 Apr 2024 at 20:15, Tomas Vondra wrote: > I 100% understand how frustrating the lack of progress can be, and I > agree we need to do better. I tried to move a number of stuck patches > this CF, and I hope (and plan) to do more of this in the future. > > But I don't quite see how would

Re: Flushing large data immediately in pqcomm

2024-04-08 Thread Jelte Fennema-Nio
On Sun, 7 Apr 2024 at 11:34, David Rowley wrote: > That seems to require modifying the following function signatures: > secure_write(), be_tls_write(), be_gssapi_write(). That's not an area > I'm familiar with, however. Attached is a new patchset where 0003 does exactly that. The only place

Re: Flushing large data immediately in pqcomm

2024-04-08 Thread Jelte Fennema-Nio
On Sun, 7 Apr 2024 at 14:41, David Rowley wrote: > Looking at the code in socket_putmessage_noblock(), I don't understand > why it's ok for PqSendBufferSize to be int but "required" must be > size_t. There's a line that does "PqSendBufferSize = required;". It > kinda looks like they both should

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Jelte Fennema-Nio
On Mon, 8 Apr 2024 at 10:02, Peter Eisentraut wrote: > I have tested this with various compilers, and I can confirm that this > shaves off about 5 seconds from the build wall-clock time, which > represents about 10%-20% of the total time. I think this is a good patch. Great to hear. >

Re: Speed up clean meson builds by ~25%

2024-04-08 Thread Jelte Fennema-Nio
On Mon, 8 Apr 2024 at 10:00, Alexander Lakhin wrote: > As I wrote in [1], I didn't observe the issue with clang-18, so maybe it > is fixed already. > Perhaps it's worth rechecking... Using the attached script I got these timings. Clang is significantly slower in all of them. But especially with

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread Jelte Fennema-Nio
On Sun, 7 Apr 2024 at 03:39, Andres Freund wrote: > Changing the global vars to size_t seems mildly bogus to me. All it's > achieving is to use slightly more memory. It also just seems unrelated to the > change. I took a closer look at this. I agree that changing PqSendBufferSize to size_t is

Re: Flushing large data immediately in pqcomm

2024-04-06 Thread Jelte Fennema-Nio
On Sat, 6 Apr 2024 at 22:21, Andres Freund wrote: > The small regression for small results is still kinda visible, I haven't yet > tested the patch downthread. Thanks a lot for the faster test script, I'm also impatient. I still saw the small regression with David his patch. Here's a v6 where I

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-06 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:48, Robert Haas wrote: > Maybe we'd be better off adding a libpq connection > option that forces the use of a specific minor protocol version, but > then we'll need backward-compatibility code in libpq basically > forever. But maybe we need that anyway to avoid older and

Re: Flushing large data immediately in pqcomm

2024-04-06 Thread Jelte Fennema-Nio
On Sat, 6 Apr 2024 at 03:34, David Rowley wrote: > Does anyone else want to try the attached script on the v5 patch to > see if their numbers are better? On my machine (i9-10900X, in Ubuntu 22.04 on WSL on Windows) v5 consistently beats master by ~0.25 seconds: master: Run 100 100 500:

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:43, Tom Lane wrote: > I don't buy that argument, actually. libpq, and pretty much every > other client AFAIK, has provisions to let higher code levels insert > random options into the startup packet. So to make this work libpq > would have to filter or at least inspect

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 18:30, Dave Cramer wrote: >> > I really intended the _pq_ prefix as a way of taking something out of >> > the GUC namespace, not as a part of the GUC namespace that users would >> > see. And I'm reluctant to go back on that. If we want to make >> > pg_protocol.${NAME} mean a

Re: Speed up clean meson builds by ~25%

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 17:24, Andres Freund wrote: > I recommend opening a bug report for clang, best with an already preprocessed > input file. > We're going to need to do something about this from our side as well, I > suspect. The times aren't great with gcc either, even if not as bad as with

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 16:02, Robert Haas wrote: > Often? > > I kind of hope that the protocol starts to evolve a bit more than it > has, but I don't want a continuous stream of changes. That will be > very hard to test and verify correctness, and a hassle for drivers to > keep up with, and a mess

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 16:04, Robert Haas wrote: > > On Thu, Apr 4, 2024 at 1:10 PM Jelte Fennema-Nio wrote: > > Attached is a rebased patchset > > We should keep talking about this, but I think we're far too close to > the wire at this point to think about commit

Re: Speed up clean meson builds by ~25%

2024-04-05 Thread Jelte Fennema-Nio
On Fri, 5 Apr 2024 at 00:45, Jelte Fennema-Nio wrote: > It improved clean build times on my machine (10 cores/20 threads) from ~40 > seconds to ~30 seconds. After discussing this off-list with Bilal, I realized that this gain is only happening for clang builds on my system. Because thos

Speed up clean meson builds by ~25%

2024-04-04 Thread Jelte Fennema-Nio
Building the generated ecpg preproc file can take a long time. You can check how long using: ninja -C build src/interfaces/ecpg/preproc/ecpg.p/meson-generated_.._preproc.c.o This moves that file much closer to the top of our build order, so building it can be pipelined much better with other

Re: WIP Incremental JSON Parser

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 23:52, Jelte Fennema-Nio wrote: > Fair enough, I guess I'll change my invocation to include the ninja > "test" target too: > ninja -C build test install-quiet && meson test -C build Actually that doesn't do what I want either because that

Re: WIP Incremental JSON Parser

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 23:34, Jacob Champion wrote: > > On Thu, Apr 4, 2024 at 1:42 PM Jelte Fennema-Nio wrote: > > Maybe that's something worth addressing too. I expected that > > install/install-quiet was a strict superset of a plain ninja > > invocation. > > Maybe

Re: Security lessons from liblzma

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 23:06, Daniel Gustafsson wrote: > > > On 4 Apr 2024, at 23:02, Jelte Fennema-Nio wrote: > > > How about we make it meson/make targets, so they are simply cached > > just like any of our other build artefacts are cached. Then only clean > >

Re: Security lessons from liblzma

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 22:56, Daniel Gustafsson wrote: > > > On 4 Apr 2024, at 22:47, Tom Lane wrote: > > > > Robert Haas writes: > >> On Thu, Apr 4, 2024 at 4:25 PM Daniel Gustafsson wrote: > >>> I don't disagree, like I said that very email: it's non-trivial and I > >>> wish we > >>> could

Re: WIP Incremental JSON Parser

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 20:48, Jacob Champion wrote: > > On Thu, Apr 4, 2024 at 11:12 AM Jacob Champion > wrote: > > What's in the `...`? I wouldn't expect to find the test binary in your > > tmp_install. > > Oh, I wonder if this is just a build dependency thing? I typically run > a bare `ninja`

Re: WIP Incremental JSON Parser

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 18:13, Andrew Dunstan wrote: > Argh. You get out of the habit when you're running with meson :-( I'm having some issues with meson too actually. "meson test -C build" is now failing with this for me: Command 'test_json_parser_incremental' not found in

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 14:50, Peter Eisentraut wrote: > It appears there are several different perspectives about this. My > intuition was that a protocol version change indicates something that we > eventually want all client libraries to support. Whereas protocol > extensions are truly opt-in.

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-04 Thread Jelte Fennema-Nio
Ugh, I seem to have somehow missed this response completely. On Thu, 14 Mar 2024 at 21:33, Robert Haas wrote: > While I think that will be a common > pattern, I do not think it will be a universal one. I do agree, > however, that every possible variation of the protocol is either >

Re: Add trim_trailing_whitespace to editorconfig file

2024-04-04 Thread Jelte Fennema-Nio
nitely think it would be good if there was an official cli tool like that for editorconfig, but the Javascript one was the closest I could find. The Go one I haven't tried. On Thu, 4 Apr 2024 at 17:23, Peter Eisentraut wrote: > > On 04.04.24 16:58, Jelte Fennema-Nio wrote: > > On Thu, 4

Re: Add trim_trailing_whitespace to editorconfig file

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 15:25, Peter Eisentraut wrote: > Everybody has git. Everybody who edits .gitattributes can use git to > check what they did. What CLI command do you use to fix/ gitattributes on all existing files? Afaict there's no command to actually remove the trailing whitespace that

Re: Flushing large data immediately in pqcomm

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 13:08, Melih Mutlu wrote: > I changed internal_flush() to an inline function, results look better this > way. It seems you also change internal_flush_buffer to be inline (but only in the actual function definition, not declaration at the top). I don't think inlining

Re: psql not responding to SIGINT upon db reconnection

2024-04-04 Thread Jelte Fennema-Nio
on Sep 17 00:00:00 2001 From: Jelte Fennema-Nio Date: Wed, 3 Apr 2024 15:19:04 +0200 Subject: [PATCH v14 1/2] Fix actually reachable pg_unreachable call In cafe1056558fe07cdc52b95205588fcd80870362 a call to pg_unreachable was introduced that was actually reachable, because there were

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-04-04 Thread Jelte Fennema-Nio
On Thu, 4 Apr 2024 at 10:45, Alvaro Herrera wrote: > Yeah, this seems to work and I no longer get that complaint from > headerscheck. patch LGTM

Re: psql not responding to SIGINT upon db reconnection

2024-04-03 Thread Jelte Fennema-Nio
17: warning: enumeration value > > ‘PGRES_POLLING_ACTIVE’ not handled in switch [-Wswitch] > > 3803 | switch (PQconnectPoll(conn)) > > | ^~ Ofcourse... fixed now From 29100578b5f0b4cec68ee1b8c572c4f280335da3 Mon Sep 17 00:00:00 2001 From

Re: psql not responding to SIGINT upon db reconnection

2024-04-03 Thread Jelte Fennema-Nio
e variant. But remove it from the recently added switch/case. And also updated the comment to remove the "for awhile". From 29100578b5f0b4cec68ee1b8c572c4f280335da3 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio Date: Wed, 3 Apr 2024 15:19:04 +0200 Subject: [PATCH v12 1/2] Fix actually reachable

Re: psql not responding to SIGINT upon db reconnection

2024-04-03 Thread Jelte Fennema-Nio
00:00:00 2001 From: Jelte Fennema-Nio Date: Wed, 3 Apr 2024 15:21:52 +0200 Subject: [PATCH v11 2/2] Remove PGRES_POLLING_ACTIVE This enum variant has been unused for at least 21 years 44aba280207740d0956160c0288e61f28f024a71. It was left for backwards compatibility for "awhile". I think

Re: Support prepared statement invalidation when result types change

2024-04-03 Thread Jelte Fennema-Nio
On Sun, 7 Jan 2024 at 07:55, vignesh C wrote: > One of the test has aborted in CFBot at [1] with: Rebased the patchset and removed patch 0003 since it was causing the CI issue reported by vignesh and it seems much less useful and more controversial to me anyway (due to the extra required

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-29 Thread Jelte Fennema-Nio
On Thu, 28 Mar 2024 at 19:03, Tom Lane wrote: > > Alvaro Herrera writes: > > It doesn't fail when it's too fast -- it's just that it doesn't cover > > the case we want to cover. > > That's hardly better, because then you think you have test > coverage but maybe you don't. Honestly, that seems

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-28 Thread Jelte Fennema-Nio
On Thu, 28 Mar 2024 at 17:43, Alvaro Herrera wrote: > Hah, you're right, I can reproduce with a smaller timeout, and using SET > LOCAL works as a fix. If we're doing that, why not reduce the timeout > to 1ms? We don't need to wait extra 9ms ... I think we don't really want to make the timeout

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-28 Thread Jelte Fennema-Nio
On Thu, 28 Mar 2024 at 17:34, Alvaro Herrera wrote: > > Eh, kestrel has also failed[1], apparently every query after the large > JOIN that this commit added as test fails with a statement timeout error. > > [1] >

Re: Possibility to disable `ALTER SYSTEM`

2024-03-28 Thread Jelte Fennema-Nio
On Thu, 28 Mar 2024 at 12:57, Robert Haas wrote: > I disagree with a lot of these changes. I think the old version was > mostly better. But I can live with a lot of it if it makes other > people happy. I'd have been fine with many of the previous versions of the docs too. (I'm not a native

Re: Possibility to disable `ALTER SYSTEM`

2024-03-28 Thread Jelte Fennema-Nio
On Thu, 28 Mar 2024 at 10:24, Jelte Fennema-Nio wrote: > I addressed them all I think. Mostly the small changes that were > suggested, but I rewrote the sentence with "might be discarded". And I > added references to the new GUC in both places suggested by David. Changed th

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-28 Thread Jelte Fennema-Nio
On Wed, 27 Mar 2024 at 19:27, Alvaro Herrera wrote: > I ended up reducing the two PG_TRY blocks to a single one. I see no > reason to split them up, and this way it looks more legible. I definitely agree this looks better. Not sure why I hadn't done that, maybe it wasn't possible in one of the

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-03-28 Thread Jelte Fennema-Nio
On Wed, 27 Mar 2024 at 19:46, Alvaro Herrera wrote: > > On 2024-Mar-27, Alvaro Herrera wrote: > > > I changed it so that the error messages are returned as translated > > phrases, and was bothered by the fact that if errors happen repeatedly, > > the memory for them might be leaked. Maybe this

Re: Possibility to disable `ALTER SYSTEM`

2024-03-28 Thread Jelte Fennema-Nio
On Thu, 28 Mar 2024 at 01:43, David G. Johnston wrote: > > On Wed, Mar 27, 2024 at 5:17 PM Bruce Momjian wrote: >> >> I addressed them all I think. Mostly the small changes that were suggested, but I rewrote the sentence with "might be discarded". And I added references to the new GUC in both

Re: Possibility to disable `ALTER SYSTEM`

2024-03-27 Thread Jelte Fennema-Nio
On Wed, 27 Mar 2024 at 23:06, Bruce Momjian wrote: > > > > > +some outside mechanism. In such environments, using > > > > > ALTER > > > > > +SYSTEM to make configuration changes might appear > > > > > to work, > > > > > +but then may be discarded at some point in the

Re: Possibility to disable `ALTER SYSTEM`

2024-03-27 Thread Jelte Fennema-Nio
On Wed, 27 Mar 2024 at 20:10, Maciek Sakrejda wrote: > > On Wed, Mar 27, 2024, 11:46 Robert Haas wrote: >> >> On Wed, Mar 27, 2024 at 1:12 PM Isaac Morland >> wrote: >> > On Wed, 27 Mar 2024 at 13:05, Greg Sabino Mullane >> > wrote: >> >>> The purpose of the setting is to prevent accidental

Re: Possibility to disable `ALTER SYSTEM`

2024-03-27 Thread Jelte Fennema-Nio
On Wed, 27 Mar 2024 at 23:23, Bruce Momjian wrote: > > On Wed, Mar 27, 2024 at 11:10:31AM -0400, Robert Haas wrote: > > > Is this really a patch we think we can push into PG 17. I am having my > > > doubts. > > > > If the worst thing that happens in PG 17 is that we push a patch that > > needs a

Re: Possibility to disable `ALTER SYSTEM`

2024-03-27 Thread Jelte Fennema-Nio
On Wed, 27 Mar 2024 at 16:10, Robert Haas wrote: > > On Wed, Mar 27, 2024 at 11:01 AM Bruce Momjian wrote: > > Uh, the above is clearly wrong. I think you mean "off" on the second line. > > Woops. When the name changed from externally_managed_configuration to > allow_alter_system, the sense of

Re: Possibility to disable `ALTER SYSTEM`

2024-03-27 Thread Jelte Fennema-Nio
On Wed, 27 Mar 2024 at 02:24, Andrew Dunstan wrote: > Agree. I don’t think “_command” adds much clarity. Alright, changed the GUC name to "allow_alter_system" since that seems to have the most "votes". One other option would be to call it simply "alter_system", just like "jit" is not called

Re: Possibility to disable `ALTER SYSTEM`

2024-03-25 Thread Jelte Fennema-Nio
On Mon, 25 Mar 2024 at 20:16, Bruce Momjian wrote: > I am wondering if the fact that you would be able to do: > > ALTER SYSTEM SET externally_managed_configuration = false > > and then be unable to use ALTER SYSTEM to revert the change is > significant. This is not possible, due to the

Re: session username in default psql prompt?

2024-03-25 Thread Jelte Fennema-Nio
On Mon, 25 Mar 2024 at 14:06, Robert Haas wrote: > On Mon, Mar 25, 2024 at 4:30 AM Jelte Fennema-Nio wrote: > > That problem seems easy to address by adding a newline into the > > default prompt. > > Ugh. Please, no! I guess it's partially a matter of taste, but personally

  1   2   3   >