Re: Refactoring backend fork+exec code

2024-02-08 Thread Heikki Linnakangas
On 07/02/2024 20:25, Andres Freund wrote: On 2024-01-30 02:08:36 +0200, Heikki Linnakangas wrote: From 54f22231bb2540fc5957c14005956161e6fc9dac Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Wed, 24 Jan 2024 23:15:55 +0200 Subject: [PATCH v8 1/5] Remove superfluous 'pgprocno&#

Re: confusing / inefficient "need_transcoding" handling in copy

2024-02-08 Thread Heikki Linnakangas
ing so you already know its length. 2. You don't need to worry about 'encoding_embeds_ascii' when you operate on the server encoding. -- Heikki Linnakangas Neon (https://neon.tech)

Re: glibc qsort() vulnerability

2024-02-07 Thread Heikki Linnakangas
with user-defined comparison functions, and we cannot make any guarantees about what they might do. So we must ensure that our qsort() doesn't overflow, no matter what the comparison function does. Looking at our ST_SORT(), it seems safe to me. -- Heikki Linnakangas Neon (https://neon.tech)

Re: ResourceOwner refactoring

2024-02-02 Thread Heikki Linnakangas
t recently remembered resource does not. I've added a note to that in ResourceOwnerForget. I read through the other callers of ResourceOwnerRemember and PinBuffer, but didn't find any other unsafe uses. I'm not too happy with this subtlety, but at least it's documented now. -- Heikki Linnakangas Neon (https://neon.tech)

Re: XLog size reductions: smaller XLRec block header for PG17

2024-02-02 Thread Heikki Linnakangas
compression though. Might as well use LZ4 or Snappy or something to compress the whole WAL stream. It's a bit tricky to get the crash-safety right, but shouldn't be impossible. Has anyone seriously considered implementing wholesale compression of WAL? -- Heikki Linnakangas Neon (https://neon.tech)

Re: POC: Extension for adding distributed tracing - pg_tracing

2024-02-02 Thread Heikki Linnakangas
gsql-hackers thread on in-core changes that would benefit the extension. Write patches for them. I'm thinking of the things I listed in the Core changes section above, but maybe there are others. PS. Does any other DBMS implement this? Any lessons to be learned from them? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2024-02-01 Thread Heikki Linnakangas
On 30/01/2024 02:08, Heikki Linnakangas wrote: On 29/01/2024 17:54, reid.thomp...@crunchydata.com wrote: On Thu, 2024-01-25 at 01:51 +0200, Heikki Linnakangas wrote: And here we go. BackendID is now a 1-based index directly into the PGPROC array. Would it be worthwhile to also note in this

Re: Extending SMgrRelation lifetimes

2024-01-31 Thread Heikki Linnakangas
On 31/01/2024 10:54, Thomas Munro wrote: On Wed, Nov 29, 2023 at 1:42 PM Heikki Linnakangas wrote: I spent some more time digging into this, experimenting with different approaches. Came up with pretty significant changes; see below: Hi Heikki, I think this approach is good. As I wrote in

Re: Fix some ubsan/asan related issues

2024-01-30 Thread Heikki Linnakangas
ith sanitizer in the cirrus CI. What does this enable that we're not already doing? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2024-01-29 Thread Heikki Linnakangas
On 29/01/2024 17:54, reid.thomp...@crunchydata.com wrote: On Thu, 2024-01-25 at 01:51 +0200, Heikki Linnakangas wrote: And here we go. BackendID is now a 1-based index directly into the PGPROC array. Would it be worthwhile to also note in this comment FIRST_AUX_PROC's and IsAuxProces

Re: Should we remove -Wdeclaration-after-statement?

2024-01-29 Thread Heikki Linnakangas
n't mind reviewing patches that don't follow the usual guidelines to the letter, stylistic things like this are easy to fix before committing. I don't feel like we're forcing these rules upon others. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Flushing large data immediately in pqcomm

2024-01-29 Thread Heikki Linnakangas
100) AS name, NOW() AS time FROM generate_series(1, 1) AS i; I'm surprised by these results, because each row in that table is < 600 bytes. PqSendBufferSize is 8kB, so the optimization shouldn't kick in in that test. Am I missing something? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Relation bulk write facility

2024-01-25 Thread Heikki Linnakangas
was a CFbot test failure last time it was run [2]. Please have a look and post an updated version if necessary. Fixed the headerscheck failure by adding appropriate #includes. -- Heikki Linnakangas Neon (https://neon.tech) From c2e8cff9326fb874b2e1643f5c3c8a4952eaa3ac Mon Sep 17 00:00:00 2001 From:

Re: Refactoring backend fork+exec code

2024-01-24 Thread Heikki Linnakangas
On 23/01/2024 21:50, Andres Freund wrote: On 2024-01-23 21:07:08 +0200, Heikki Linnakangas wrote: On 22/01/2024 23:07, Andres Freund wrote: diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c index 1a1050c8da1..92f24db4e18 100644 --- a/src

Re: Refactoring backend fork+exec code

2024-01-23 Thread Heikki Linnakangas
On 22/01/2024 23:07, Andres Freund wrote: On 2024-01-10 14:35:52 +0200, Heikki Linnakangas wrote: @@ -5344,31 +5344,31 @@ StartChildProcess(AuxProcType type) errno = save_errno; switch (type) { - case StartupProcess

Re: make BuiltinTrancheNames less ugly

2024-01-23 Thread Heikki Linnakangas
top of the array). Now, having written this proposal, I'm leaning towards idea 2 myself, but since the patch here is less invasive, it seems worth having as evidence. Idea 2 seems pretty straightforward, +1 for that. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Network failure may prevent promotion

2024-01-23 Thread Heikki Linnakangas
utdownRequestPending only at specific places, and don't want to get terminated at the any random CHECK_FOR_INTERRUPTS()? -- Heikki Linnakangas Neon (https://neon.tech) From 27b9f8283b2caa7a4243fe57a8d14a127396e80f Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Tue, 23 Jan 2024 11:

Re: Network failure may prevent promotion

2024-01-23 Thread Heikki Linnakangas
On 23/01/2024 06:23, Kyotaro Horiguchi wrote: At Mon, 22 Jan 2024 13:29:10 -0800, Andres Freund wrote in Hi, On 2024-01-19 12:28:05 +0900, Michael Paquier wrote: On Thu, Jan 18, 2024 at 03:42:28PM +0200, Heikki Linnakangas wrote: Given that commit 728f86fec6 that introduced this issue was

Re: Multiple startup messages over the same connection

2024-01-22 Thread Heikki Linnakangas
t one backend operates on one database is pretty deeply ingrained in the code. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Adding facility for injection points (or probe points?) for more advanced tests

2024-01-22 Thread Heikki Linnakangas
On 22/01/2024 18:08, Heikki Linnakangas wrote: I wrote the attached patch to enable injection points in the Cirrus CI config, to run the injection tests I wrote for a GIN bug today [1]. But that led to a crash in the asan-enabled build [2]. I didn't investigate it yet. Pushed a fix fo

Re: Adding facility for injection points (or probe points?) for more advanced tests

2024-01-22 Thread Heikki Linnakangas
crash in the asan-enabled build [2]. I didn't investigate it yet. [1] https://www.postgresql.org/message-id/d8f0b068-0e6e-4b2c-8932-62507eb7e1c6%40iki.fi [2] https://cirrus-ci.com/task/5242888636858368 -- Heikki Linnakangas Neon (https://neon.tech) From d641364bccec051761a9360453162929ef

Re: Change GUC hashtable to use simplehash?

2024-01-19 Thread Heikki Linnakangas
final32(&hs, (uint64) value); I couldn't find any guidance on what properties the 'seed' and 'tweak' have, compared to just accumulating the values with accum. Anyone know? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Network failure may prevent promotion

2024-01-18 Thread Heikki Linnakangas
t with libpqsrv_exec and libpqsrv_get_result? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Change GUC hashtable to use simplehash?

2024-01-17 Thread Heikki Linnakangas
need stability. " typo: licencse Other than that, LGTM. -- Heikki Linnakangas Neon (https://neon.tech)

Re: ResourceOwner refactoring

2024-01-16 Thread Heikki Linnakangas
take a look on it. Fixed, thanks. mark_pgdllimport.pl also highlighted two new variables in walsummarizer.h, fixed those too. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Stack overflow issue

2024-01-12 Thread Heikki Linnakangas
On 11/01/2024 19:37, Robert Haas wrote: On Wed, Jan 10, 2024 at 4:25 PM Heikki Linnakangas wrote: The problem with CommitTransactionCommand (or rather AbortCurrentTransaction() which has the same problem) and ShowTransactionStateRec is that they get called in a state where aborting can lead to

Re: Streaming I/O, vectored I/O (WIP)

2024-01-11 Thread Heikki Linnakangas
On 11/01/2024 05:19, Thomas Munro wrote: On Thu, Jan 11, 2024 at 8:58 AM Heikki Linnakangas wrote: On 10/01/2024 06:13, Thomas Munro wrote: Bikeshedding call: I am open to better suggestions for the names PrepareReadBuffer() and CompleteReadBuffers(), they seem a little grammatically clumsy

Re: Stack overflow issue

2024-01-10 Thread Heikki Linnakangas
On 05/01/2024 19:23, Robert Haas wrote: On Fri, Nov 24, 2023 at 10:47 AM Heikki Linnakangas wrote: What do you think? At least for 0001 and 0002, I think we should just add the stack depth checks. With regard to 0001, CommitTransactionCommand() and friends are hard enough to understand as

Re: Streaming I/O, vectored I/O (WIP)

2024-01-10 Thread Heikki Linnakangas
eadBuffers() call. If WaitReadBuffer() does the batching, that's not really required. But does that make sense with async I/O? With async I/O, will you need a vectorized version of StartReadBuffer() too? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2024-01-10 Thread Heikki Linnakangas
On 08/12/2023 14:33, Heikki Linnakangas wrote: + [PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true}, + [PMC_AV_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true}, + [PMC_BGWORKER] = {"bgworker", BackgroundWorkerMain,

Re: Relation bulk write facility

2024-01-10 Thread Heikki Linnakangas
On 09/01/2024 08:50, vignesh C wrote: There are few compilation errors reported by CFBot at [1], patch needs to be rebased: Here you go. -- Heikki Linnakangas Neon (https://neon.tech) From b1791303c54da762ecf3d63f1a60d5b93732af57 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Wed

Re: Experiments with Postgres and SSL

2023-12-30 Thread Heikki Linnakangas
On 05/07/2023 02:33, Michael Paquier wrote: On Tue, Jul 04, 2023 at 05:15:49PM +0300, Heikki Linnakangas wrote: I don't see the point of the libpq 'sslalpn' option either. Let's send ALPN always. Admittedly having the options make testing different of combinations of old

Re: Avoid computing ORDER BY junk columns unnecessarily

2023-12-29 Thread Heikki Linnakangas
dy, by removing the junk columns needed for grouping but not sorting from sort_input_target. But the grouping path generation has some assumptions that the grouping output target list includes all the grouping columns. I don't remember the exact problem that made me give up on that, but it probably could be fixed. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Avoid computing ORDER BY junk columns unnecessarily

2023-12-22 Thread Heikki Linnakangas
On 22/12/2023 17:24, Tom Lane wrote: Heikki Linnakangas writes: It won't help in all cases though, the index might not store the original value in the first place. I'm a little skeptical that an index could produce an accurate ORDER BY result if it doesn't store the valu

Re: Avoid computing ORDER BY junk columns unnecessarily

2023-12-22 Thread Heikki Linnakangas
ata. Hmm, so return the computed column from the index instead of recomputing it? Yeah, that makes sense too and would help in this example. It won't help in all cases though, the index might not store the original value in the first place. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Simplify newNode()

2023-12-19 Thread Heikki Linnakangas
On 18/12/2023 16:28, Heikki Linnakangas wrote: I think we have consensus on patch v2. It's simpler and not less performant than what we have now, at least on modern compilers. Barring objections, I'll commit that. Committed that. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Simplify newNode()

2023-12-18 Thread Heikki Linnakangas
nating the extra argument. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Simplify newNode()

2023-12-14 Thread Heikki Linnakangas
Yngmh6DsLL83XRVdG%2Bt4GnXYxA2G6KOruY1WljU9y/l3rr2yscB2KoAAHAANgXOHyQE5kBeLW2cMwclcCEFpesE0vBRtaCHaQNrHX9CcB66QIrJWg%2BDeG415rOeusi%2BL31svlexs5/F8kZwkggA%3D%3D -- Heikki Linnakangas Neon (https://neon.tech) From 15e93b1b9fdaf19dd55f943ab4f567b5feed2961 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Thu, 14 Dec 2023 14:44:10 +01

Re: Relation bulk write facility

2023-12-14 Thread Heikki Linnakangas
th the new API that this introduces, it should be an isolated change to the implementation, with no changes required to the callers. -- Heikki Linnakangas Neon (https://neon.tech)

Simplify newNode()

2023-12-13 Thread Heikki Linnakangas
dismiss it with modern compilers. It does get inlined on gcc 12 and clang 14 that I have installed. -- Heikki Linnakangas Neon (https://neon.tech)From 6ad4c4cf49ef5b3f7ed22acc258a868f1a13f6f4 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Thu, 14 Dec 2023 01:08:16 +0100 Subject: [PATCH 1/

Re: Streaming I/O, vectored I/O (WIP)

2023-12-11 Thread Heikki Linnakangas
array, but maybe just remove the 'const' and document that it scribbles on it? I'm planning to commit these fairly soon. +1 -- Heikki Linnakangas Neon (https://neon.tech)

Re: Streaming I/O, vectored I/O (WIP)

2023-12-09 Thread Heikki Linnakangas
On 09/12/2023 02:41, Thomas Munro wrote: On Sat, Dec 9, 2023 at 7:25 AM Andres Freund wrote: On 2023-11-30 13:01:46 +1300, Thomas Munro wrote: On Thu, Nov 30, 2023 at 12:16 PM Heikki Linnakangas wrote: Maybe we should bite the bullet and always retry short writes in FileWriteV(). Is that

Re: Remove some unnecessary includes of "access/xlog_internal.h"

2023-12-08 Thread Heikki Linnakangas
headerscheck. +1 -- Heikki Linnakangas Neon (https://neon.tech)

Re: Rename ShmemVariableCache and initialize it in more standard way

2023-12-07 Thread Heikki Linnakangas
On 05/12/2023 05:40, Richard Guo wrote: On Tue, Dec 5, 2023 at 12:31 AM Tristan Partin wrote: On Mon Dec 4, 2023 at 6:49 AM CST, Heikki Linnakangas wrote: > Here's a patch to allocate and initialize it with a pair of ShmemSize > and ShmemInit functions, like all o

Re: Extending SMgrRelation lifetimes

2023-12-07 Thread Heikki Linnakangas
On 29/11/2023 14:41, Heikki Linnakangas wrote: 2. A funny case with foreign tables: ANALYZE on a foreign table calls visibilitymap_count(). A foreign table has no visibility map so it returns 0, but before doing so it calls RelationGetSmgr on the foreign table, which has 0/0/0 rellocator. That

Re: Add checks in pg_rewind to abort if backup_label file is present

2023-12-05 Thread Heikki Linnakangas
open_target_file("backup_label", true); /* BACKUP_LABEL_FILE */ write_target_range(buf, 0, len); close_target_file(); That TODO comment needs to go away now. And we probably should complain if the file already exists. With your patch, we already checked earlier that it doesn't exists, so if it exists when we reach that code, something's gone wrong. -- Heikki Linnakangas Neon (https://neon.tech)

Re: btree: downlink right separator/HIKEY optimization

2023-12-04 Thread Heikki Linnakangas
scribe the new argument. Perhaps the text from README should go there, this feels like a detail specific to _bt_search and _bt_moveright. -- Heikki Linnakangas Neon (https://neon.tech)

Rename ShmemVariableCache and initialize it in more standard way

2023-12-04 Thread Heikki Linnakangas
hmemVariableCache are the authoritative source. Attached patch renames it to "TransamVariables", but I'm all ears for other suggestions. [0] https://www.postgresql.org/message-id/20230710225043.svl7fqxecwshw...@awork3.anarazel.de -- Heikki Linnakangas Neon (https://neon.tech)From aa61

Re: Refactoring backend fork+exec code

2023-12-03 Thread Heikki Linnakangas
D: 1713734 I haven't looked deeper yet, but it seems that we see two issues here (and Assert is not directly caused by the patches set.) I have not been able to reproduce this one. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2023-12-01 Thread Heikki Linnakangas
;m not sure what our policy with backpatching this kind of issues is. This goes back to all supported versions, but given the lack of complaints, I chose to not backpatch. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2023-12-01 Thread Heikki Linnakangas
eview, I made the new refactorings compared in a new commit 0003. I will squash that before pushing, but this makes it easier to see what changed. Barring any new feedback or issues, I will commit these. -- Heikki Linnakangas Neon (https://neon.tech) From 6dd44d7057e535eb441c9ab8fda1bbdd8079

Re: pgsql: Clean up role created in new subscription test.

2023-12-01 Thread Heikki Linnakangas
;tablespace', spcname AS obj FROM pg_catalog.pg_tablespace UNION ALL SELECT 'subscription', subname AS obj FROM pg_catalog.pg_subscription ) ORDER BY 1, 2 Is it OK to leave behind extra databases? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2023-11-30 Thread Heikki Linnakangas
On 30/11/2023 22:26, Andres Freund wrote: On 2023-11-30 01:36:25 +0200, Heikki Linnakangas wrote: From a96b6e92fdeaa947bf32774c425419b8f987b8e2 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Thu, 30 Nov 2023 00:01:25 +0200 Subject: [PATCH v3 1/7] Refactor

Re: PostgreSql: Canceled on conflict out to old pivot

2023-11-30 Thread Heikki Linnakangas
544821/postgresql-canceled-on-conflict-out-to-old-pivot> Nice graphs! -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2023-11-30 Thread Heikki Linnakangas
o, on shutdown. Committed a fix for that to master. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Streaming I/O, vectored I/O (WIP)

2023-11-29 Thread Heikki Linnakangas
-short. Thoughts? Feels pretty ugly, but I don't see anything outright wrong with that. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Change GUC hashtable to use simplehash?

2023-11-29 Thread Heikki Linnakangas
a quick glance. Let's also replace the partial murmurhash implementations we have in hashfn.h with this. It's a very similar algorithm, and we don't need two. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Extending SMgrRelation lifetimes

2023-11-29 Thread Heikki Linnakangas
t calls smgrrelease() now, so the distinction is just pro forma. The idea is that if you call smgrclose(), you hint that you don't need that SMgrRelation pointer anymore, although there might be other pointers to the same object and they stay valid. -- Heikki Linnakangas Neon (https://neon.te

Re: Streaming I/O, vectored I/O (WIP)

2023-11-28 Thread Heikki Linnakangas
On 28/11/2023 14:17, Thomas Munro wrote: On Thu, Sep 28, 2023 at 7:33 AM Heikki Linnakangas wrote: in streaming_read.h: typedef bool (*PgStreamingReadBufferDetermineNextCB) (PgStreamingRead *pgsr, uintptr_t pgsr_private, void *per_io_private, BufferManagerRelation

Re: Streaming I/O, vectored I/O (WIP)

2023-11-28 Thread Heikki Linnakangas
On 28/11/2023 14:17, Thomas Munro wrote: On Thu, Sep 28, 2023 at 7:33 AM Heikki Linnakangas wrote: + /* Avoid a slightly more expensive kernel call if there is no benefit. */ + if (iovcnt == 1) + returnCode = pg_pread(vfdP->

Re: Fix a wrong comment in setrefs.c

2023-11-28 Thread Heikki Linnakangas
are added in the setop tree.  We might have to clean that up someday; but for now, just ignore any false matches." +1.  It explains the situation much more clearly and accurately. To make it easier to review, I've updated the patch to be so. Committed, th

Re: XID formatting and SLRU refactorings

2023-11-28 Thread Heikki Linnakangas
On 28/11/2023 12:14, Pavel Borisov wrote: On Tue, 28 Nov 2023 at 13:13, Heikki Linnakangas wrote: On 27/11/2023 01:43, Alexander Korotkov wrote: v61 looks good to me. I'm going to push it as long as there are no objections. This was discussed earlier, but is still present i

Re: XID formatting and SLRU refactorings

2023-11-28 Thread Heikki Linnakangas
f a "long_segment_names" boolean, how about an integer field, to specify the length. That means that we'll need pg_upgrade to copy pg_multixact/members files under the new names. That should be pretty straightforward. -- Heikki Linnakangas Neon (https://neon.tech)

Re: PostgreSql: Canceled on conflict out to old pivot

2023-11-28 Thread Heikki Linnakangas
es. The amount of shared memory reserved for tracking the dependencies is determined by max_pred_locks_per_transaction, so you could try increasing that to reduce those false positives, even if you never get the "out of shared memory" error. -- Heikki Linnakangas Neon (https://neon.tech)

Re: walwriter interacts quite badly with synchronous_commit=off

2023-11-27 Thread Heikki Linnakangas
On 25/10/2023 21:59, Andres Freund wrote: See attached. It's the same logic as in your patch, just formulatd more clearly IMHO. Yep, makes sense! Pushed this. Thanks for the investigation! -- Heikki Linnakangas Neon (https://neon.tech)

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2023-11-27 Thread Heikki Linnakangas
_bt_readpage, but it has the side-effect of advancing the array keys. Side-effects from an assertion seems problematic. Vague idea: refactor _bt_checkkeys() into something that doesn't have side-effects, and have a separate function or an argument to _bt_checkkeys() to advance to next array key. The prechecking optimization and the Assertion could both use the side-effect-free function. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Relation bulk write facility

2023-11-24 Thread Heikki Linnakangas
On 19/11/2023 02:04, Andres Freund wrote: On 2023-11-17 11:37:21 +0100, Heikki Linnakangas wrote: The new facility makes it easier to optimize bulk loading, as the logic for buffering, WAL-logging, and syncing the relation only needs to be implemented once. It's also less error-prone: We

Re: Stack overflow issue

2023-11-24 Thread Heikki Linnakangas
l, to trigger the summarization, we could traverse the tree without using stack. But a limit seems useful.) What do you think? -- Heikki Linnakangas Neon (https://neon.tech) From 543e6569a3f4730f3105379cc946cc6b54525179 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Fri, 24 Nov 2023 14:13

Re: Improving the comments in pqsignal()

2023-11-23 Thread Heikki Linnakangas
le of the differences Here is my attempt to improve it. Thanks! This is program 10.12 from Advanced Programming in the UNIX Environment, with minor changes. In the copy I found online (3rd edition), it's "Figure 10.18", not "program 10.12". Other than that, looks good. -- Heikki Linnakangas Neon (https://neon.tech)

Re: PoC: prefetching index leaf pages (for inserts)

2023-11-23 Thread Heikki Linnakangas
dy in cache. If you get a cache miss, it would start an async I/O read for the page, and yield to the other insertions until the I/O completes. We already support async execution of FDWs now, with the ForeignAsyncRequest() and ForeignAsyncConfigureWait() callbacks. Can we generalize t

Re: WaitEventSet resource leakage

2023-11-23 Thread Heikki Linnakangas
14-16. Thank you! (As a side note, this fix doesn't resolve the issue #17828 completely, because that large number of handles might be also consumed legally.) :-( -- Heikki Linnakangas Neon (https://neon.tech)

Re: psql not responding to SIGINT upon db reconnection

2023-11-23 Thread Heikki Linnakangas
ect() or ppoll(). They were created specifically to address this problem. Not fully portable, I think it's missing on Windows at least. Maybe use pselect() if it's available, and select() with a timeout if it's not. -- Heikki Linnakangas Neon (https://neon.tech)

Re: psql not responding to SIGINT upon db reconnection

2023-11-22 Thread Heikki Linnakangas
On 22/11/2023 19:29, Tristan Partin wrote: On Thu Nov 16, 2023 at 8:33 AM CST, Heikki Linnakangas wrote: On 06/11/2023 19:16, Tristan Partin wrote: That sounds like a much better solution. Attached you will find a v4 that implements your suggestion. Please let me know if there is something

Re: Relation bulk write facility

2023-11-17 Thread Heikki Linnakangas
On 19/09/2023 17:13, Heikki Linnakangas wrote: The attached patch centralizes that pattern to a new bulk writing facility, and changes all those AMs to use it. Here's a new rebased version of the patch. This includes fixes to the pageinspect regression test. They were explained in the c

Re: psql not responding to SIGINT upon db reconnection

2023-11-16 Thread Heikki Linnakangas
x27;t understand the difference, so that's a genuine question. There was an attempt at unifying them in the past but it was reverted in commit 5d43c3c54d. -- Heikki Linnakangas Neon (https://neon.tech)

Re: [HACKERS] Should logtape.c blocks be of type long?

2023-11-16 Thread Heikki Linnakangas
out since year 2000. Other than that, looks good to me. -- Heikki Linnakangas Neon (https://neon.tech)

Re: WaitEventSet resource leakage

2023-11-16 Thread Heikki Linnakangas
On 16/11/2023 01:08, Tom Lane wrote: Heikki Linnakangas writes: On 09/03/2023 20:51, Tom Lane wrote: After further thought that seems like a pretty ad-hoc solution. We probably can do no better in the back branches, but shouldn't we start treating WaitEventSets as ResourceOwner-ma

Re: WaitEventSet resource leakage

2023-11-15 Thread Heikki Linnakangas
's always allocated in TopMemoryContext, but pass a ResourceOwner instead: WaitEventSet * CreateWaitEventSet(ResourceOwner owner, int nevents) And use owner == NULL to mean session lifetime. -- Heikki Linnakangas Neon (https://neon.tech) From b9ea609855b838369cddb33e4045ac91603dd726 Mon S

Re: ResourceOwner refactoring

2023-11-15 Thread Heikki Linnakangas
On 13/11/2023 01:08, Thomas Munro wrote: On Mon, Nov 13, 2023 at 11:16 AM Heikki Linnakangas wrote: On 11/11/2023 14:00, Alexander Lakhin wrote: 10.11.2023 17:26, Heikki Linnakangas wrote: I think that is surprising behavior from the DSA facility. When you make allocations with dsa_allocate

Re: Cleaning up array_in()

2023-11-12 Thread Heikki Linnakangas
xceeds the maximum allowed (6) LINE 1: select '{{{{{{1}}'::int[]; ^ The error message isn't great, as the literal contains 10 dimensions, not 7 as the error message claims. -- Heikki Linnakangas Neon (https://neon.tech)

Re: ResourceOwner refactoring

2023-11-12 Thread Heikki Linnakangas
On 11/11/2023 14:00, Alexander Lakhin wrote: 10.11.2023 17:26, Heikki Linnakangas wrote: I think that is surprising behavior from the DSA facility. When you make allocations with dsa_allocate() or just call dsa_get_address() on an existing dsa_pointer, you wouldn't expect the cu

Re: AdvanceXLInsertBuffers() vs wal_sync_method=open_datasync

2023-11-10 Thread Heikki Linnakangas
ame logic as in XLogFlush(). I wonder if the 'flexible' argument to XLogWrite() is too inflexible. It would be nice to pass a hard minimum XLogRecPtr that it must write up to, but still allow it to write more than that if it's convenient. -- Heikki Linnakangas Neon (https://neon.tech)

Re: ResourceOwner refactoring

2023-11-10 Thread Heikki Linnakangas
() on an existing dsa_pointer, you wouldn't expect the current resource owner to matter. I think dsa_create/attach() should store the current resource owner in the dsa_area, for use in subsequent operations on the DSA, per attached patch (0002-Fix-dsa.c-with-diff

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2023-11-09 Thread Heikki Linnakangas
On 18/09/2023 07:08, David Rowley wrote: On Fri, 15 Sept 2023 at 22:37, Heikki Linnakangas wrote: I've added a call to LockAssertNoneHeld(false) in there. I don't see it in the patch? hmm. I must've git format-patch before committing that part. I'll try that again...

Re: ResourceOwner refactoring

2023-11-08 Thread Heikki Linnakangas
llocated but empty. However, ResourceOwnerReleas() didn't get the memo, and still checked "owner->hash != NULL" rather than "owner->nhash == 0". -- Heikki Linnakangas Neon (https://neon.tech)

Re: ResourceOwner refactoring

2023-11-08 Thread Heikki Linnakangas
On 07/11/2023 16:00, Alexander Lakhin wrote: 07.11.2023 14:28, Heikki Linnakangas wrote: I feel pretty good about this overall. Barring objections or new cfbot failures, I will commit this in the next few days. A script, that I published in [1], detects several typos in the patches: betwen

Re: ResourceOwner refactoring

2023-11-07 Thread Heikki Linnakangas
ul to avoid padding and to keep frequently-accessed fields close to each other, there's no benefit in keeping them at the beginning of the struct. -- Heikki Linnakangas Neon (https://neon.tech)

Re: btree_gin: Incorrect leftmost interval value

2023-10-27 Thread Heikki Linnakangas
t we didn't need to construct these leftmost values. But I don't think that's supported by the current extractQuery interface. -- Heikki Linnakangas Neon (https://neon.tech)

Re: walwriter interacts quite badly with synchronous_commit=off

2023-10-25 Thread Heikki Linnakangas
On 25/10/2023 21:59, Andres Freund wrote: On 2023-10-25 12:17:03 +0300, Heikki Linnakangas wrote: On 25/10/2023 02:09, Andres Freund wrote: Because of the inherent delay between the checks of XLogCtl->WalWriterSleeping and Latch->is_set, we also sometimes end up with multiple pro

Re: ResourceOwner refactoring

2023-10-25 Thread Heikki Linnakangas
e ResurceOwnerForget(NULL) do nothing, but I think it's better to have the check in the callers. You wouldn't want to silently do nothing when the resource owner is not expected to be NULL. (I'm attaching new patch version in my reply to Andres shortly) -- Heikki Linnakangas Neon (https://neon.tech)

Re: walwriter interacts quite badly with synchronous_commit=off

2023-10-25 Thread Heikki Linnakangas
y, leading to most of the work being done by backends. We should probably wake walsender at the end of XLogInsertRecord() if there is sufficient outstanding WAL. Right, that's basically the same issue that I reasoned through above. I did some quick testing with a few different settings of w

Re: CRC32C Parallel Computation Optimization on ARM

2023-10-24 Thread Heikki Linnakangas
vary it by changing the checkpoint interval and/or pgbench scale factor: if you checkpoint frequently or if the database is larger, you get more full-page images which makes the records larger on average, and vice versa. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Special-case executor expression steps for common combinations

2023-10-12 Thread Heikki Linnakangas
y not. It seems nice to be more explicit when you don't expect a return value. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Is this a problem in GenericXLogFinish()?

2023-10-11 Thread Heikki Linnakangas
t lasts forever? Do you still think it offers a useful check? Yeah, I think this is a useful assertion. It might catch some bugs in extensions too. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Comparing two double values method

2023-10-10 Thread Heikki Linnakangas
. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Checks in RegisterBackgroundWorker.()

2023-10-09 Thread Heikki Linnakangas
On 06/10/2023 13:13, Thomas Munro wrote: On Thu, Sep 28, 2023 at 9:46 AM Heikki Linnakangas wrote: Subject: [PATCH v2 3/3] Fix misleading comment on StartBackgroundWorker(). LGTM. Hmm, maybe I would have called that function "BackgroundWorkerMain()" like several other similar t

Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)

2023-10-06 Thread Heikki Linnakangas
notice the segfault. I missed it because syslogger gets restarted and then it worked. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)

2023-10-05 Thread Heikki Linnakangas
On 29/08/2023 09:58, Heikki Linnakangas wrote: On 29/08/2023 09:21, Heikki Linnakangas wrote: Thinking about this some more, the ListenSockets array is a bit silly anyway. We fill the array starting from index 0, always append to the end, and never remove entries from it. It would seem more

Re: Show various offset arrays for heap WAL records

2023-10-02 Thread Heikki Linnakangas
was not a conscious choice. Otherwise, LGTM. Made these changes and committed. Thank you! -- Heikki Linnakangas Neon (https://neon.tech)

Re: bgwriter doesn't flush WAL stats

2023-10-02 Thread Heikki Linnakangas
The first patch, to flush the bgwriter's WAL stats to the stats collector, seems like a straightforward bug fix, so committed and backpatched that. Thank you! I didn't look at the second patch. -- Heikki Linnakangas Neon (https://neon.tech)

<    1   2   3   4   5   6   7   8   9   10   >