Re: Experiments with Postgres and SSL

2024-04-25 Thread Heikki Linnakangas
On 24/04/2024 23:51, Peter Eisentraut wrote: On 08.04.24 10:38, Heikki Linnakangas wrote: On 08/04/2024 04:25, Heikki Linnakangas wrote: One important open item now is that we need to register a proper ALPN protocol ID with IANA. I sent a request for that: https://mailarchive.ietf.org/arch

Re: Feature request: schema diff tool

2024-04-24 Thread Heikki Linnakangas
there. And you can get pretty far by just running 'pg_dump --schema-only' on both databases and comparing them with 'diff'. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Direct SSL connection with ALPN and HBA rules

2024-04-23 Thread Heikki Linnakangas
GSSAPI request. So that comment applies to sslnegotiation=direct, but not sslnegotiation=requiredirect. Attached patch tries to fix and clarify those. (Note that the client will only attempt GSSAPI encryption if it can find kerberos credentials in the environment.) -- Heikki Linnakangas Neon

Re: POC: make mxidoff 64 bits

2024-04-23 Thread Heikki Linnakangas
for example. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Direct SSL connection with ALPN and HBA rules

2024-04-22 Thread Heikki Linnakangas
On 22/04/2024 10:47, Heikki Linnakangas wrote: On 22/04/2024 10:19, Michael Paquier wrote: On Sat, Apr 20, 2024 at 12:43:24AM +0300, Heikki Linnakangas wrote: On 19/04/2024 19:48, Jacob Champion wrote: On Fri, Apr 19, 2024 at 6:56 AM Heikki Linnakangas wrote: With direct SSL negotiation, we

Re: Direct SSL connection with ALPN and HBA rules

2024-04-22 Thread Heikki Linnakangas
On 22/04/2024 10:19, Michael Paquier wrote: On Sat, Apr 20, 2024 at 12:43:24AM +0300, Heikki Linnakangas wrote: On 19/04/2024 19:48, Jacob Champion wrote: On Fri, Apr 19, 2024 at 6:56 AM Heikki Linnakangas wrote: With direct SSL negotiation, we always require ALPN. (As an aside: I

Re: Direct SSL connection with ALPN and HBA rules

2024-04-19 Thread Heikki Linnakangas
On 19/04/2024 19:48, Jacob Champion wrote: On Fri, Apr 19, 2024 at 6:56 AM Heikki Linnakangas wrote: With direct SSL negotiation, we always require ALPN. (As an aside: I haven't gotten to test the version of the patch that made it into 17 yet, but from a quick glance it looks like we're

Re: Direct SSL connection with ALPN and HBA rules

2024-04-19 Thread Heikki Linnakangas
TLS layer is established with direct SSL, you never need to retry with traditional negotiation, or vice versa. [1] https://www.postgresql.org/message-id/CAAWbhmjetCVgu9pHJFkQ4ejuXuaz2mD1oniXokRHft0usCa7Yg%40mail.gmail.com -- Heikki Linnakangas Neon (https://neon.tech)

Re: AIX support

2024-04-18 Thread Heikki Linnakangas
On 18 April 2024 14:15:43 GMT+03:00, Sriram RK wrote: >Thanks Noah and Team, > >We (IBM-AIX team) looked into this issue > >https://www.postgresql.org/message-id/20240225194322...@rfd.leadboat.com > >This is related to the compiler issue. The compilers xlc(13.1) and gcc(8.0) >have issues. But we

Re: plenty code is confused about function level static

2024-04-18 Thread Heikki Linnakangas
more occurrences I've found with a bit of grepping. +1 -- Heikki Linnakangas Neon (https://neon.tech)

White-box testing heap pruning

2024-04-14 Thread Heikki Linnakangas
ould clear the HEAP_MOVED_IN flag instead. [1] https://www.postgresql.org/message-id/CAAKRu_azf-zH%3DDgVbquZ3tFWjMY1w5pO8m-TXJaMdri8z3933g%40mail.gmail.com -- Heikki Linnakangas Neon (https://neon.tech)From 9dcd528dd42f689e207d808bee388fb233b2e25e Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: S

Re: some LLVM function checks missing in meson

2024-04-13 Thread Heikki Linnakangas
ce(llvm_opt0_orc); llvm_opt0_orc = NULL; } } The autoconf test that set HAVE_DECL_LLVMORCREGISTERPERF was removed in commit e9a9843e13. I believe that's a leftover that should also have been removed. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Fix possible dereference null pointer (src/backend/replication/logical/reorderbuffer.c)

2024-04-13 Thread Heikki Linnakangas
On 11/04/2024 16:37, Ranier Vilela wrote: Em qui., 11 de abr. de 2024 às 09:54, Heikki Linnakangas mailto:hlinn...@iki.fi>> escreveu: On 11/04/2024 15:03, Ranier Vilela wrote: > Em qua., 10 de abr. de 2024 às 18:28, Heikki Linnakangas > mailto:hlinn...@iki.fi>

Re: Typos in the code and README

2024-04-12 Thread Heikki Linnakangas
typo while reading the code. -- Heikki Linnakangas Neon (https://neon.tech) From 5f531b719c176b2f316b6341fa062af508ed2e10 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Sun, 7 Apr 2024 22:34:23 +0300 Subject: [PATCH 1/2] fix typos --- doc/src/sgml/maintenance.sgml

PG_TEST_EXTRAs by theme rather than test name (Re: pgsql: Add tests for libpq gssencmode and sslmode options)

2024-04-12 Thread Heikki Linnakangas
(moved to pgsql-hackers, change subject) On 10/04/2024 18:54, Heikki Linnakangas wrote: On 10/04/2024 17:48, Peter Eisentraut wrote: On 08.04.24 01:50, Heikki Linnakangas wrote: Add tests for libpq gssencmode and sslmode options Why aren't these tests at src/interfaces/libpq/t

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-04-11 Thread Heikki Linnakangas
On 11/04/2024 18:09, Alexander Korotkov wrote: On Thu, Apr 11, 2024 at 1:46 AM Heikki Linnakangas wrote: On 07/04/2024 00:52, Alexander Korotkov wrote: * At first, we check that pg_wal_replay_wait() is called in a non-atomic * context. That is, a procedure call isn't wrapped

Re: Improve eviction algorithm in ReorderBuffer

2024-04-11 Thread Heikki Linnakangas
. Pushed the patch and reverted binaryheap changes. Thank you! -- Heikki Linnakangas Neon (https://neon.tech)

Re: Fix possible dereference null pointer (src/backend/replication/logical/reorderbuffer.c)

2024-04-11 Thread Heikki Linnakangas
On 11/04/2024 15:03, Ranier Vilela wrote: Em qua., 10 de abr. de 2024 às 18:28, Heikki Linnakangas mailto:hlinn...@iki.fi>> escreveu: On 10/04/2024 21:07, Ranier Vilela wrote: > Hi, > > Per Coverity. > > The function ReorderBufferTXNByXid,

Re: Improve eviction algorithm in ReorderBuffer

2024-04-10 Thread Heikki Linnakangas
On 11/04/2024 01:37, Michael Paquier wrote: On Thu, Apr 11, 2024 at 12:20:55AM +0300, Heikki Linnakangas wrote: To move this forward, here's a patch to switch to a pairing heap. In my very quick testing, with the performance test cases posted earlier in this thread [1] [2], I'm seeing

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-04-10 Thread Heikki Linnakangas
oesn't feel quite ready for v17, and IMHO should be reverted. It's a nice feature, so I'd love to have it fixed and reviewed early in the v18 cycle. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Fix possible dereference null pointer (src/backend/replication/logical/reorderbuffer.c)

2024-04-10 Thread Heikki Linnakangas
ld already have its ReorderBufferTXN entry, so ReorderBufferTXN() should never return NULL. It's not surprising if Coverity doesn't understand that, but setting the 'create' flag doesn't seem like the right fix. If we add "Assert(txn != NULL)", does that silence it? -- Heikki Lin

Re: Improve eviction algorithm in ReorderBuffer

2024-04-10 Thread Heikki Linnakangas
On 10/04/2024 08:31, Amit Kapila wrote: On Wed, Apr 10, 2024 at 11:00 AM Heikki Linnakangas wrote: On 10/04/2024 07:45, Michael Paquier wrote: On Tue, Apr 09, 2024 at 09:16:53PM -0700, Jeff Davis wrote: On Wed, 2024-04-10 at 12:13 +0900, Michael Paquier wrote: Wouldn't the best way forward

Re: Improve eviction algorithm in ReorderBuffer

2024-04-09 Thread Heikki Linnakangas
to switch the pairing heap, performance test that, and revert the binary heap changes. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Improve eviction algorithm in ReorderBuffer

2024-04-09 Thread Heikki Linnakangas
hash table. And it's cheap to to insert to (O(1)), so we could probably remove the MAX_HEAP_TXN_COUNT_THRESHOLD, and always keep the heap up-to-date. -- Heikki Linnakangas Neon (https://neon.tech)

Re: libpq.sgml: "server ejectes GSS" -> server rejects GSS

2024-04-08 Thread Heikki Linnakangas
On 09/04/2024 07:40, Erik Rijkers wrote: Typo. fix: -attempted first. If the server ejectes GSS encryption, SSL is +attempted first. If the server rejects GSS encryption, SSL is Fixed, thanks! -- Heikki Linnakangas Neon (https://neon.tech)

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Heikki Linnakangas
: Hard feature freeze deadline This would also give everyone more visibility, so that we're not all surprised by the last minute flood of commits. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Experiments with Postgres and SSL

2024-04-08 Thread Heikki Linnakangas
On 08/04/2024 04:25, Heikki Linnakangas wrote: One important open item now is that we need to register a proper ALPN protocol ID with IANA. I sent a request for that: https://mailarchive.ietf.org/arch/msg/tls-reg-review/9LWPzQfOpbc8dTT7vc9ahNeNaiw/ -- Heikki Linnakangas Neon (https

Re: Experiments with Postgres and SSL

2024-04-07 Thread Heikki Linnakangas
On 08/04/2024 04:28, Tom Lane wrote: Heikki Linnakangas writes: Committed this. Thank you to everyone involved! Looks like perlcritic isn't too happy with the test code: koel and crake say ./src/test/libpq_encryption/t/001_negotiate_encryption.pl: Return value of flagged function ignored

Re: Experiments with Postgres and SSL

2024-04-07 Thread Heikki Linnakangas
ortant open item now is that we need to register a proper ALPN protocol ID with IANA. -- Heikki Linnakangas Neon (https://neon.tech)

Re: LDAP & Kerberos test woes

2024-04-07 Thread Heikki Linnakangas
On 07/04/2024 13:19, Heikki Linnakangas wrote: 1st patch fixes the LDAP setup tests, and 2nd patch fixes the error handling in the END blocks. Committed and backpatched these test fixes. -- Heikki Linnakangas Neon (https://neon.tech)

LDAP & Kerberos test woes

2024-04-07 Thread Heikki Linnakangas
nd 2nd patch fixes the error handling in the END blocks. [1] https://commitfest.postgresql.org/47/4742/ -- Heikki Linnakangas Neon (https://neon.tech)From 6f09361ac0dbac5c2aef59b6a24e92486097cb43 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Sun, 7 Apr 2024 13:06:39 +0300 Subject: [PATCH 1/2

Re: Use streaming read API in ANALYZE

2024-04-03 Thread Heikki Linnakangas
stgres: heikki postgres [local] ANALYZE(_start+0x21)[0x564889971a61] 2024-04-03 20:15:49.157 EEST [262101] LOG: server process (PID 262232) was terminated by signal 6: Aborted -- Heikki Linnakangas Neon (https://neon.tech)

Re: Combine Prune and Freeze records emitted by vacuum

2024-04-03 Thread Heikki Linnakangas
and attached the diff. Applied those changes, and committed. Thank you! Off-list, Melanie reported that there is a small regression with the benchmark script she posted yesterday, after all, but I'm not able to reproduce that. Actually, I think it was noise. Ok, phew. -- Heikki Linnakangas Neon

Re: Combine Prune and Freeze records emitted by vacuum

2024-04-03 Thread Heikki Linnakangas
On 02/04/2024 16:11, Heikki Linnakangas wrote: On 01/04/2024 20:22, Melanie Plageman wrote: Review for 0003-0006 (I didn't have any new thoughts on 0002). I know you didn't modify them much/at all, but I noticed some things in my code that could be better. Ok, here's what I have now. I made

Re: Streaming read-ready sequential scan code

2024-04-02 Thread Heikki Linnakangas
; The SELECT takes about 390 ms on 'master', and 230 ms with the patch. This is pretty much the best case for this patch, real world gains will be much smaller. Nevertheless, nice speedup! -- Heikki Linnakangas Neon (https://neon.tech)

Re: Combine Prune and Freeze records emitted by vacuum

2024-04-02 Thread Heikki Linnakangas
implementation to vacuumlazy.c as is. Would be nice to have just one copy of this in some common place, but I also wasn't sure where to put it. -- Heikki Linnakangas Neon (https://neon.tech) From be8891155c93f3555c49371f9804bdf5ba578f6e Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Tue

Re: Combine Prune and Freeze records emitted by vacuum

2024-04-01 Thread Heikki Linnakangas
han "(prstate.options & HEAP_PRUNE_PAGE_MARK_UNUSED_NOW) != 0" anyway. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Combine Prune and Freeze records emitted by vacuum

2024-04-01 Thread Heikki Linnakangas
On 01/04/2024 19:08, Melanie Plageman wrote: On Mon, Apr 01, 2024 at 05:17:51PM +0300, Heikki Linnakangas wrote: diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c @@ -256,15 +270,16 @@ heap_page_prune(Relation relation, Buffer buffer, tup.t_tableOid

Re: Commitfest Manager for March

2024-04-01 Thread Heikki Linnakangas
On 01/04/2024 09:05, Andrey M. Borodin wrote: I think it makes sense to close this commitfest only after Feature Freeze on April 8, 2024 at 0:00 AoE. What do you think? +1. IIRC that's how it's been done in last commitfest in previous years too. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Streaming I/O, vectored I/O (WIP)

2024-03-29 Thread Heikki Linnakangas
On 29/03/2024 09:01, Thomas Munro wrote: On Fri, Mar 29, 2024 at 9:45 AM Heikki Linnakangas wrote: master (213c959a29):8.0 s streaming-api v13: 9.5 s Hmm, that's not great, and I think I know one factor that has confounded my investigation and the conflicting reports

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-29 Thread Heikki Linnakangas
en marked yet. I haven't finished updating all the comments, but I am really interested to know what you think about heap_prune_chain() now. Looks much better now, thanks! -- Heikki Linnakangas Neon (https://neon.tech)

Re: Experiments with Postgres and SSL

2024-03-28 Thread Heikki Linnakangas
On 28/03/2024 13:15, Matthias van de Meent wrote: On Tue, 5 Mar 2024 at 15:08, Heikki Linnakangas wrote: I hope I didn't joggle your elbow reviewing this, Jacob, but I spent some time rebase and fix various little things: With the recent changes to backend startup committed by you

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-28 Thread Heikki Linnakangas
On 28/03/2024 03:53, Melanie Plageman wrote: On Thu, Mar 28, 2024 at 01:04:04AM +0200, Heikki Linnakangas wrote: One change with this is that live_tuples and many of the other fields are now again updated, even if the caller doesn't need them. It was hard to skip them in a way that would save

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-27 Thread Heikki Linnakangas
() decides what to do with each tuple, and ensures that each tuple is marked only once, and the subroutines update all the variables, add the item to the correct arrays etc. depending on what we're doing with it. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Streaming I/O, vectored I/O (WIP)

2024-03-26 Thread Heikki Linnakangas
On 24/03/2024 15:02, Thomas Munro wrote: On Wed, Mar 20, 2024 at 4:04 AM Heikki Linnakangas wrote: Maybe 'pg_streaming_read_next_buffer' or just 'pg_streaming_read_next', for a shorter name. Hmm. The idea of 'buffer' appearing in a couple of names is that there are conceptually other kinds

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-25 Thread Heikki Linnakangas
On 24/03/2024 18:32, Melanie Plageman wrote: On Thu, Mar 21, 2024 at 9:28 AM Heikki Linnakangas wrote: In heap_page_prune_and_freeze(), we now do some extra work on each live tuple, to set the all_visible_except_removable correctly. And also to update live_tuples, recently_dead_tuples

Re: Recording whether Heap2/PRUNE records are from VACUUM or from opportunistic pruning (Was: Show various offset arrays for heap WAL records)

2024-03-25 Thread Heikki Linnakangas
and vacuum WAL record formats"). Marking this as Committed in the commitfest. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-25 Thread Heikki Linnakangas
On 24/03/2024 21:55, Melanie Plageman wrote: On Sat, Mar 23, 2024 at 01:09:30AM +0200, Heikki Linnakangas wrote: On 20/03/2024 21:17, Melanie Plageman wrote: There is another patch in the commitfest that touches this area: "Recording whether Heap2/PRUNE records are from VACUUM or

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-22 Thread Heikki Linnakangas
/cmp/etc are lifted and shifted from one file to another. When I am reviewing a big diff, it is always helpful to know where I need to review line-by-line. Done. From 5d6fc2ffbdd839e0b69242af16446a46cf6a2dc7 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Wed, 20 Mar 2024 13:49:59 +0200

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-21 Thread Heikki Linnakangas
On 20/03/2024 23:03, Melanie Plageman wrote: On Wed, Mar 20, 2024 at 03:15:49PM +0200, Heikki Linnakangas wrote: diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h index ee0eca8ae02..b2015f5a1ac 100644 --- a/src/include/access/heapam.h +++ b/src/include/access/heapam.h

Re: Refactoring backend fork+exec code

2024-03-20 Thread Heikki Linnakangas
how much value do these assertions carry? If you're intent on keeping them, perhaps casting child_type to int here would suppress the warnings. But personally I think I'd lose the Asserts. Yeah, it's not a very valuable assertion. Removed, thanks! -- Heikki Linnakangas Neon (https://neon.tech)

Re: Streaming I/O, vectored I/O (WIP)

2024-03-19 Thread Heikki Linnakangas
s just 'pgsr->distance = pgsr->max_pinned_buffers' ? -- Heikki Linnakangas Neon (https://neon.tech)

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-19 Thread Heikki Linnakangas
regular and shared iterators be merged, or wrapped under a common interface? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Heikki Linnakangas
postgres user, and print a warning or refuse to start up if they are. (Another way to read this proposal is to rename the GUC that's been discussed in this thread to 'configuration_managed_externally'. That makes it look less like a security feature, and describes the intended use case.) -

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-18 Thread Heikki Linnakangas
to me, albeit a very minor one. Certainly not an optimization, it doesn't affect performance in any way, only what EXPLAIN reports. So committed and backported that to all supported branches. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2024-03-18 Thread Heikki Linnakangas
On 13/03/2024 09:30, Heikki Linnakangas wrote: Attached is a new version of the remaining patches. Committed, with some final cosmetic cleanups. Thanks everyone! -- Heikki Linnakangas Neon (https://neon.tech)

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-17 Thread Heikki Linnakangas
nd commit messages. I'll wait for a new version from you before reviewing more. -- Heikki Linnakangas Neon (https://neon.tech) From 622620a7875ae8c1626e9cd118156e0c734d44ed Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Sun, 17 Mar 2024 22:52:28 +0200 Subject: [PATCH v3heik

Re: Weird test mixup

2024-03-15 Thread Heikki Linnakangas
On 15/03/2024 16:00, Tom Lane wrote: Heikki Linnakangas writes: On 15/03/2024 13:09, Heikki Linnakangas wrote: I committed a patch to do that, to put out the fire. That's turning the buildfarm quite red. Many, but not all animals are failing like this: It may be even worse than

Re: Weird test mixup

2024-03-15 Thread Heikki Linnakangas
On 15/03/2024 14:10, Heikki Linnakangas wrote: On 15/03/2024 13:09, Heikki Linnakangas wrote: I committed a patch to do that, to put out the fire. That's turning the buildfarm quite red. Many, but not all animals are failing like this: --- /home/buildfarm/hippopotamus/buildroot/HEAD

Re: Weird test mixup

2024-03-15 Thread Heikki Linnakangas
On 15/03/2024 13:09, Heikki Linnakangas wrote: On 15/03/2024 01:13, Tom Lane wrote: Michael Paquier writes: Or we could just disable runningcheck because of the concurrency requirement in this test. The test would still be able to run, just less times. No, actually we *must* mark all

Re: Weird test mixup

2024-03-15 Thread Heikki Linnakangas
with the current definition of injection points. The point of installcheck mode is that the tests are supposed to be safe to run in a live installation. Side-effects occurring in other databases are completely not OK. I committed a patch to do that, to put out the fire. -- Heikki Linnakangas Neon (https

Re: Weird test mixup

2024-03-15 Thread Heikki Linnakangas
e installchecks. Oops. It would be nice to automatically detach all the injection points on process exit. You wouldn't always want that, but I think most tests hold a session open throughout the test, and for those it would be handy. -- Heikki Linnakangas Neon (https://neon.tech)

Weird test mixup

2024-03-14 Thread Heikki Linnakangas
hat makes no sense. That injection point is only used by the test in src/test/modules/gin/. Perhaps that ran at the same time as the intarray test? But they run in separate instances, with different data directories. And the 'gin' test passed. I'm completely stumped. Anyone have a theory? -- Heikki Linna

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-14 Thread Heikki Linnakangas
On 14/03/2024 22:00, Melanie Plageman wrote: On Thu, Mar 14, 2024 at 05:30:30PM +0200, Heikki Linnakangas wrote: typedef struct SharedBitmapHeapInstrumentation { int num_workers; BitmapHeapScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER

Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE

2024-03-14 Thread Heikki Linnakangas
ink it's still better to store ParallelBitmapHeapInstrumentationInfo separately in the DSM chunk, not as part of ParallelBitmapHeapState. Attached patch does that, rebased over current master. I didn't address any of the other things that you, Tomas, pointed out, but I think they're valid

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-14 Thread Heikki Linnakangas
On 14/03/2024 12:55, Dilip Kumar wrote: On Thu, Mar 14, 2024 at 4:07 PM Heikki Linnakangas wrote: _SPI_execute_plan() has code to deal with the possibility that the active snapshot is not set. That seems fishy; do we really support SPI without any snapshot? I'm inclined to turn

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-14 Thread Heikki Linnakangas
On 14/03/2024 14:34, Robert Haas wrote: On Thu, Mar 14, 2024 at 6:37 AM Heikki Linnakangas wrote: If es_snapshot was different from the active snapshot, things would get weird, even without parallel query. The scans would use es_snapshot for the visibility checks, but any functions you execute

Re: Flushing large data immediately in pqcomm

2024-03-14 Thread Heikki Linnakangas
ocket_is_send_pending() directly here. pq_is_send_pending() does the same, but it's at a higher level of abstraction. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Can Execute commands for different portals interleave?

2024-03-14 Thread Heikki Linnakangas
with the Bind message. -- Heikki Linnakangas Neon (https://neon.tech)

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-14 Thread Heikki Linnakangas
hot serialization from bitmap heap scan node, to make it consistent with other non-parallel scan nodes (it's not really a parallel scan as far as the table AM is concerned). See attached patch, which is the same as previous patch with some extra assertions. -- Heikki Linnakangas

Re: Vectored I/O in bulk_write.c

2024-03-13 Thread Heikki Linnakangas
ed me most was that smgrwrite() had a different signature than mdwrite(). I'm happy with the way you have it in the v4 patch. -- Heikki Linnakangas Neon (https://neon.tech)

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-03-13 Thread Heikki Linnakangas
be even more sure, I added some code there to assert that the serialized version of node->ss.ps.state->es_snapshot is equal to pstate->phs_snapshot_data, and all the tests passed with that too. I propose that we just remove the code in BitmapHeapScan to serialize the snapshot, per attach

Re: Vectored I/O in bulk_write.c

2024-03-13 Thread Heikki Linnakangas
On 13/03/2024 12:18, Thomas Munro wrote: On Wed, Mar 13, 2024 at 9:57 PM Heikki Linnakangas wrote: Here also is a first attempt at improving the memory allocation and memory layout. ... +typedef union BufferSlot +{ + PGIOAlignedBlock buffer; + dlist_node freelist_node

Re: Vectored I/O in bulk_write.c

2024-03-13 Thread Heikki Linnakangas
oducing the bulk write interface was that now we have a natural place to put all these heuristics and optimizations in. That seems to be a success, AFAICS none of the things discussed here will change the bulk_write API, just the implementation. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Disconnect if socket cannot be put into non-blocking mode

2024-03-12 Thread Heikki Linnakangas
On 11/03/2024 16:44, Heikki Linnakangas wrote: While self-reviewing my "Refactoring backend fork+exec code" patches, I noticed this in pq_init(): /* * In backends (as soon as forked) we operate the underlying socket in * nonblocking mode and use latches to

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-11 Thread Heikki Linnakangas
On 09/03/2024 22:41, Melanie Plageman wrote: On Wed, Mar 6, 2024 at 7:59 AM Heikki Linnakangas wrote: Does GlobalVisTestIsRemovableXid() handle FrozenTransactionId correctly? Okay, so I thought a lot about this, and I don't understand how GlobalVisTestIsRemovableXid() would not handle

Re: Confine vacuum skip logic to lazy_scan_skip

2024-03-11 Thread Heikki Linnakangas
On 11/03/2024 18:15, Melanie Plageman wrote: On Mon, Mar 11, 2024 at 11:29:44AM +0200, Heikki Linnakangas wrote: diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index ac55ebd2ae..1757eb49b7 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src

Disconnect if socket cannot be put into non-blocking mode

2024-03-11 Thread Heikki Linnakangas
iously, it's not cool to continue using the connection as if everything is OK, if we fail to put it into non-blocking mode. We should disconnect. (COMMERROR merely logs the error, it does not bail out like ERROR does) Barring objections, I'll commit and backpatch the attached to fix that. -- Hei

Re: Confine vacuum skip logic to lazy_scan_skip

2024-03-11 Thread Heikki Linnakangas
On 08/03/2024 19:34, Melanie Plageman wrote: On Fri, Mar 08, 2024 at 06:07:33PM +0200, Heikki Linnakangas wrote: On 08/03/2024 02:46, Melanie Plageman wrote: On Wed, Mar 06, 2024 at 10:00:23PM -0500, Melanie Plageman wrote: On Wed, Mar 06, 2024 at 09:55:21PM +0200, Heikki Linnakangas wrote

Re: Failures in constraints regression test, "read only 0 of 8192 bytes"

2024-03-11 Thread Heikki Linnakangas
On 10/03/2024 22:59, Thomas Munro wrote: On Mon, Mar 11, 2024 at 9:30 AM Heikki Linnakangas wrote: Barring objections, I'll commit the attached. +1 Pushed, thanks! I guess the comment for smgrreleaseall() could also be updated. It mentions only PROCSIGNAL_BARRIER_SMGRRELEASE, but I

Re: Failures in constraints regression test, "read only 0 of 8192 bytes"

2024-03-10 Thread Heikki Linnakangas
On 10/03/2024 11:20, Heikki Linnakangas wrote: On 10/03/2024 08:23, Thomas Munro wrote: On Sun, Mar 10, 2024 at 6:48 PM Thomas Munro wrote: I won't be surprised if the answer is: if you're holding a reference, you have to get a pin (referring to bulk_write.c). Ahhh, on second thoughts, I

Re: Failures in constraints regression test, "read only 0 of 8192 bytes"

2024-03-10 Thread Heikki Linnakangas
l RelationCloseSmgr() for relations created in the same transaction anymore. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Make query cancellation keys longer

2024-03-08 Thread Heikki Linnakangas
3.1 fails, but IMHO it's better to not complicate this and accept the break in backwards-compatibility. 9.3 was released in 2013. We dropped pg_dump support for versions older than 9.2 a few years ago, this raises the bar for pg_dump to 9.3 as well. -- Heikki Linnakangas Neon (https://neon.tec

Re: Confine vacuum skip logic to lazy_scan_skip

2024-03-08 Thread Heikki Linnakangas
On 08/03/2024 02:46, Melanie Plageman wrote: On Wed, Mar 06, 2024 at 10:00:23PM -0500, Melanie Plageman wrote: On Wed, Mar 06, 2024 at 09:55:21PM +0200, Heikki Linnakangas wrote: I will say that now all of the variable names are *very* long. I didn't want to remove the "state" from

Re: Confine vacuum skip logic to lazy_scan_skip

2024-03-08 Thread Heikki Linnakangas
It would get rid of this tricky bookkeeping in heap_vac_scan_next_block(). -- Heikki Linnakangas Neon (https://neon.tech) vactest.sql Description: application/sql

Re: Add last_commit_lsn to pg_stat_database

2024-03-07 Thread Heikki Linnakangas
ng the last LSN that did something that needs to be replicated through the publication might be useful, but that's not what what this patch does. -- Heikki Linnakangas Neon (https://neon.tech)

Re: improve ssl error code, 2147483650

2024-03-07 Thread Heikki Linnakangas
it would catch most of the common mistakes in practice, so maybe we should just hold our noses and do it anyway, if the above ERR_SYSTEM_ERROR() method doesn't work. It's sad that we cannot pass a file descriptor or in-memory copy of the file contents to those functions. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Confine vacuum skip logic to lazy_scan_skip

2024-03-06 Thread Heikki Linnakangas
just wondering. -- Heikki Linnakangas Neon (https://neon.tech) From 6e0258c3e31e85526475f46b2e14cbbcbb861909 Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Sat, 30 Dec 2023 16:30:59 -0500 Subject: [PATCH v6 1/9] lazy_scan_skip remove unneeded local var nskippable_blocks

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-06 Thread Heikki Linnakangas
t's not immediate clear what "is provided" means here. Does the caller provide it, or does the function set it, ie. is it an input or output argument? Looking at the code, it's an input, but now it looks a bit weird that an input argument is called 'presult'. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Refactoring backend fork+exec code

2024-03-05 Thread Heikki Linnakangas
On 05/03/2024 11:44, Richard Guo wrote: I noticed that there are still three places in backend_status.c where pgstat_get_beentry_by_backend_id() is referenced.  I think we should replace them with pgstat_get_beentry_by_proc_number(). Fixed, thanks! -- Heikki Linnakangas Neon (https

Re: Experiments with Postgres and SSL

2024-03-05 Thread Heikki Linnakangas
negotiated mode. So if we e.g. had a bug that accidentally disabled direct SSL connection completely and always used negotiated mode, the tests would still pass. I'd like to see some tests that would catch that. -- Heikki Linnakangas Neon (https://neon.tech) From c3b88ffb05a2a8b50e1af3220bf8f5

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-03-05 Thread Heikki Linnakangas
, the transaction might be lost. I think that's completely unacceptable. If you're ok to lose the prepared state of twophase transactions on crash, why don't you create the subscription with 'two_phase=off' to begin with? -- Heikki Linnakangas Neon (https://neon.tech)

Re: Missing LWLock protection in pgstat_reset_replslot()

2024-03-04 Thread Heikki Linnakangas
you pass need_lock=true. So that at least should be changed to false. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Experiments with Postgres and SSL

2024-03-04 Thread Heikki Linnakangas
On 01/03/2024 23:49, Jacob Champion wrote: On Wed, Feb 28, 2024 at 4:10 AM Heikki Linnakangas wrote: I think we'd want to *avoid* changing the major protocol version in a way that would introduce a new roundtrip, though. I'm starting to get up to speed with this patchset. So far I'm mostly

Re: Refactoring backend fork+exec code

2024-03-03 Thread Heikki Linnakangas
On 22/02/2024 02:37, Heikki Linnakangas wrote: On 15/02/2024 07:09, Robert Haas wrote: On Thu, Feb 15, 2024 at 3:07 AM Andres Freund wrote: I think the last remaining question here is about the 0- vs 1-based indexing of BackendIds. Is it a good idea to switch to 0-based indexing? And if we do

Make query cancellation keys longer

2024-02-29 Thread Heikki Linnakangas
table time-constant memcmp() function; I'll add those if there's agreement on this otherwise. -- Heikki Linnakangas Neon (https://neon.tech)From f871e915fee08b3cc27e732646f3a62ec1c3024b Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Thu, 29 Feb 2024 23:20:52 +0200 Subject: [PATCH 1/2] Move

Re: Relation bulk write facility

2024-02-28 Thread Heikki Linnakangas
On 28/02/2024 18:04, Tom Lane wrote: Heikki Linnakangas writes: On 28/02/2024 00:30, Thomas Munro wrote: I agree, this should be an error, and we should then remove the test. Done. The commit that added that test added a support function "get_columns_length" which is now unuse

Re: Experiments with Postgres and SSL

2024-02-28 Thread Heikki Linnakangas
missed "avoid" there. I meant: I think we'd want to *avoid* changing the major protocol version in a way that would introduce a new roundtrip, though. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Relation bulk write facility

2024-02-28 Thread Heikki Linnakangas
8 00:24:01 +0400, Heikki Linnakangas wrote: The problem is due to this hunk: @@ -401,10 +376,6 @@ install-lib-static: $(stlib) installdirs-lib install-lib-shared: $(shlib) installdirs-lib ifdef soname -# we don't install $(shlib) on AIX -# (see http://archives.postgresql.org/

Remove --with-CC autoconf option

2024-02-27 Thread Heikki Linnakangas
. PGAC_ARG_REQ(with, CC, [CMD], [set compiler (deprecated)], [CC=$with_CC]) -- Heikki Linnakangas Neon (https://neon.tech)

  1   2   3   4   5   6   7   8   9   10   >