Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

2025-10-18 Thread Philipp Marek
Hi Filip, 7% is an interesting improvement for these cases. I see it as a nice feature, What about adding a GUC variable to have the ability to configure PQ_RECV_BUFFER_SIZE based on the user's needs? Well, currently this is just the array size as a preprocessor macro, so the executables .bss

Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

2025-10-18 Thread Philipp Marek
Here's the patch again, this time with a 128kB buffer size. This gives us nearly the same gains (~7%) for the blob INSERTs, and the additional memory usage (120kB) shouldn't really matter, with "temp_buffer"s 8MB and "work_mem" 4MB defaults. Making it configurable would give a much more complex

Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

2025-10-18 Thread Philipp Marek
Hi Andres, Have you tried to verify that this doesn't cause performance regressions in other workloads? pq_recvbuf() has this code: ... I do seem to recall that just increasing the buffer size substantially lead to more time being spent inside that memmove() (likely due to exceeding L1/L

Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

2025-10-17 Thread Andres Freund
Hi, On 2025-10-07 15:03:29 +0200, Philipp Marek wrote: > > Have you tried to verify that this doesn't cause performance regressions > > in > > other workloads? pq_recvbuf() has this code: > > > ... > > > > I do seem to recall that just increasing the buffer size substantially > > lead to > > mor

Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

2025-10-17 Thread Filip Janus
Hi, 7% is an interesting improvement for these cases. I see it as a nice feature, What about adding a GUC variable to have the ability to configure PQ_RECV_BUFFER_SIZE based on the user's needs? -Filip- Ășt 30. 9. 2025 v 10:42 odesĂ­latel Philipp Marek napsal: > Here's the patch again, this

Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

2025-10-17 Thread Andres Freund
Hi, On 2025-09-30 10:42:00 +0200, Philipp Marek wrote: > Here's the patch again, this time with a 128kB buffer size. > > This gives us nearly the same gains (~7%) for the blob INSERTs, > and the additional memory usage (120kB) shouldn't really matter, > with "temp_buffer"s 8MB and "work_mem" 4MB

Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

2025-09-15 Thread Philipp Marek
Hi Kirill, Changing this constant will result in an overwhelming increase of memory consumption for instances that work with a large number of connections ( max_connections ~ 1e4) for zero benefit. No, it's not that bad. 1) With this being allocated in the bss segment, the RAM will only be

[PATCH] Better Performance for PostgreSQL with large INSERTs

2025-09-15 Thread Philipp Marek
Sometimes, storing documents (eg. PDFs) in a database is much easier than using a separate storage (like S3, NFS, etc.). (Because of issues like backup integrity, availability, service dependencies, access rights, encryption of data, etc..) With this patch: ```diff diff --git i/src/backend/lib

Re: [PATCH] Better Performance for PostgreSQL with large INSERTs

2025-09-15 Thread Kirill Reshke
Hi! On Mon, 15 Sept 2025 at 18:16, Philipp Marek wrote: > > Sometimes, storing documents (eg. PDFs) in a database > is much easier than using a separate storage (like S3, NFS, etc.). > > (Because of issues like backup integrity, availability, > service dependencies, access rights, encryption of d