Re: Lowering the default wal_blocksize to 4K

2023-10-12 Thread Robert Haas
On Thu, Oct 12, 2023 at 9:57 AM Ants Aasma wrote: > This reminds me that xlp_tli is not being used to its full potential right > now either. We only check that it's not going backwards, but there is at > least one not very hard to hit way to get postgres to silently replay on the > wrong

Re: Lowering the default wal_blocksize to 4K

2023-10-12 Thread Ants Aasma
On Thu, 12 Oct 2023 at 16:36, Robert Haas wrote: > On Wed, Oct 11, 2023 at 4:28 PM Thomas Munro > wrote: > > That leaves only the segments where a record starts exactly on the > > first usable byte of a segment, which is why I was trying to think of > > a way to cover that case too. I

Re: Lowering the default wal_blocksize to 4K

2023-10-12 Thread Robert Haas
On Wed, Oct 11, 2023 at 6:11 PM Andres Freund wrote: > I think the question is what the point of the crosschecks in long page headers > is. It's pretty easy to see what the point of the xlp_sysid check is - make it > less likely to accidentally replay WAL from a different system. It's much >

Re: Lowering the default wal_blocksize to 4K

2023-10-12 Thread Robert Haas
On Wed, Oct 11, 2023 at 4:28 PM Thomas Munro wrote: > That leaves only the segments where a record starts exactly on the > first usable byte of a segment, which is why I was trying to think of > a way to cover that case too. I suggested we could notice and insert > a new record at that place.

Re: Lowering the default wal_blocksize to 4K

2023-10-11 Thread Andres Freund
Hi, On 2023-10-11 16:09:21 +0200, Matthias van de Meent wrote: > On Wed, 11 Oct 2023 at 01:29, Andres Freund wrote: > > > After that we'll only have the system ID left from the extended > > > header, which we could store across 2 pages in the (current) alignment > > > losses of xlp_rem_len -

Re: Lowering the default wal_blocksize to 4K

2023-10-11 Thread Andres Freund
Hi, On 2023-10-11 16:05:02 -0400, Robert Haas wrote: > On Tue, Oct 10, 2023 at 7:29 PM Andres Freund wrote: > > > Hmm. I don't think we should remove those checks, as I can see people > > > that would want to change their XLog block size with e.g. > > > pg_reset_wal. > > > > I don't think that's

Re: Lowering the default wal_blocksize to 4K

2023-10-11 Thread Thomas Munro
On Thu, Oct 12, 2023 at 9:27 AM Thomas Munro wrote: > On Thu, Oct 12, 2023 at 9:05 AM Robert Haas wrote: > > But if we do want to keep those cross-checks, why not take what Thomas > > proposed a little further and move all of xlp_sysid, xlp_seg_size, and > > xlp_xlog_blcksz into

Re: Lowering the default wal_blocksize to 4K

2023-10-11 Thread Thomas Munro
On Thu, Oct 12, 2023 at 9:05 AM Robert Haas wrote: > But if we do want to keep those cross-checks, why not take what Thomas > proposed a little further and move all of xlp_sysid, xlp_seg_size, and > xlp_xlog_blcksz into XLOG_CHECKPOINT_REDO? Then long and short page > headers would become

Re: Lowering the default wal_blocksize to 4K

2023-10-11 Thread Robert Haas
On Tue, Oct 10, 2023 at 7:29 PM Andres Freund wrote: > > Hmm. I don't think we should remove those checks, as I can see people > > that would want to change their XLog block size with e.g. > > pg_reset_wal. > > I don't think that's something we need to address in every physical > segment. For

Re: Lowering the default wal_blocksize to 4K

2023-10-11 Thread Matthias van de Meent
On Wed, 11 Oct 2023 at 01:29, Andres Freund wrote: > > Hi, > > On 2023-10-10 21:30:44 +0200, Matthias van de Meent wrote: > > On Tue, 10 Oct 2023 at 06:14, Andres Freund wrote: > > > On 2023-10-09 23:16:30 -0400, Tom Lane wrote: > > >> Andres Freund writes: > > >>> There's an alternative

Re: Lowering the default wal_blocksize to 4K

2023-10-10 Thread Andres Freund
Hi, On 2023-10-11 14:39:12 +1300, Thomas Munro wrote: > On Wed, Oct 11, 2023 at 12:29 PM Andres Freund wrote: > > On 2023-10-10 21:30:44 +0200, Matthias van de Meent wrote: > > > On Tue, 10 Oct 2023 at 06:14, Andres Freund wrote: > > > > I was thinking we should perhaps do the opposite, namely

Re: Lowering the default wal_blocksize to 4K

2023-10-10 Thread Thomas Munro
On Wed, Oct 11, 2023 at 12:29 PM Andres Freund wrote: > On 2023-10-10 21:30:44 +0200, Matthias van de Meent wrote: > > On Tue, 10 Oct 2023 at 06:14, Andres Freund wrote: > > > I was thinking we should perhaps do the opposite, namely getting rid of > > > short > > > page headers. The overhead in

Re: Lowering the default wal_blocksize to 4K

2023-10-10 Thread Andres Freund
Hi, On 2023-10-10 21:30:44 +0200, Matthias van de Meent wrote: > On Tue, 10 Oct 2023 at 06:14, Andres Freund wrote: > > On 2023-10-09 23:16:30 -0400, Tom Lane wrote: > >> Andres Freund writes: > >>> There's an alternative approach we could take, which is to write in 4KB > >>> increments, while

Re: Lowering the default wal_blocksize to 4K

2023-10-10 Thread Matthias van de Meent
On Tue, 10 Oct 2023 at 06:14, Andres Freund wrote: > > Hi, > > On 2023-10-09 23:16:30 -0400, Tom Lane wrote: >> Andres Freund writes: >>> There's an alternative approach we could take, which is to write in 4KB >>> increments, while keeping 8KB pages. With the current format that's not >>>

Re: Lowering the default wal_blocksize to 4K

2023-10-10 Thread Matthias van de Meent
On Tue, 10 Oct 2023 at 01:08, Andres Freund wrote: > > Hi, > > I've mentioned this to a few people before, but forgot to start an actual > thread. So here we go: > > I think we should lower the default wal_blocksize / XLOG_BLCKSZ to 4096, from > the current 8192. Seems like a good idea. > It's

Re: Lowering the default wal_blocksize to 4K

2023-10-09 Thread Andres Freund
Hi, On 2023-10-09 23:16:30 -0400, Tom Lane wrote: > Andres Freund writes: > > There's an alternative approach we could take, which is to write in 4KB > > increments, while keeping 8KB pages. With the current format that's not > > obviously a bad idea. But given there aren't really advantages in

Re: Lowering the default wal_blocksize to 4K

2023-10-09 Thread Tom Lane
Andres Freund writes: > There's an alternative approach we could take, which is to write in 4KB > increments, while keeping 8KB pages. With the current format that's not > obviously a bad idea. But given there aren't really advantages in 8KB WAL > pages, it seems we should just go for 4KB? Seems

Re: Lowering the default wal_blocksize to 4K

2023-10-09 Thread Bruce Momjian
On Mon, Oct 9, 2023 at 04:36:20PM -0700, Andres Freund wrote: > Hi, > > On 2023-10-09 19:26:54 -0400, Bruce Momjian wrote: > > On Mon, Oct 9, 2023 at 04:08:05PM -0700, Andres Freund wrote: > > > There's an alternative approach we could take, which is to write in 4KB > > > increments, while

Re: Lowering the default wal_blocksize to 4K

2023-10-09 Thread Andres Freund
Hi, On 2023-10-09 19:26:54 -0400, Bruce Momjian wrote: > On Mon, Oct 9, 2023 at 04:08:05PM -0700, Andres Freund wrote: > > There's an alternative approach we could take, which is to write in 4KB > > increments, while keeping 8KB pages. With the current format that's not > > obviously a bad idea.

Re: Lowering the default wal_blocksize to 4K

2023-10-09 Thread Bruce Momjian
On Mon, Oct 9, 2023 at 04:08:05PM -0700, Andres Freund wrote: > There's an alternative approach we could take, which is to write in 4KB > increments, while keeping 8KB pages. With the current format that's not > obviously a bad idea. But given there aren't really advantages in 8KB WAL > pages, it

Lowering the default wal_blocksize to 4K

2023-10-09 Thread Andres Freund
Hi, I've mentioned this to a few people before, but forgot to start an actual thread. So here we go: I think we should lower the default wal_blocksize / XLOG_BLCKSZ to 4096, from the current 8192. The reason is that a) We don't gain much from a blocksize above 4096, as we already do one write