Re: Potential stack overflow in incremental base backup

2024-05-16 Thread Thomas Munro
On Tue, Apr 16, 2024 at 4:10 AM Robert Haas wrote: > On Wed, Apr 10, 2024 at 9:55 PM Thomas Munro wrote: > > To rescue my initdb --rel-segsize project[1] for v18, I will have a go > > at making that dynamic. It looks like we don't actually need to > > allocate that until we get to the

Re: Potential stack overflow in incremental base backup

2024-04-15 Thread Robert Haas
On Wed, Apr 10, 2024 at 9:55 PM Thomas Munro wrote: > Pushed. That fixes the stack problem. Cool. > Of course we still have this: > > /* > * Since this array is relatively large, avoid putting it on the stack. > * But we don't need it at all if this is not an incremental backup.

Re: Potential stack overflow in incremental base backup

2024-04-10 Thread Thomas Munro
On Thu, Apr 11, 2024 at 12:11 AM Robert Haas wrote: > On Wed, Apr 10, 2024 at 6:21 AM Thomas Munro wrote: > > Could we just write the blocks directly into the output array, and > > then transpose them directly in place if start_blkno > 0? See > > attached. I may be missing something, but the

Re: Potential stack overflow in incremental base backup

2024-04-10 Thread Robert Haas
On Wed, Apr 10, 2024 at 6:21 AM Thomas Munro wrote: > Could we just write the blocks directly into the output array, and > then transpose them directly in place if start_blkno > 0? See > attached. I may be missing something, but the only downside I can > think of is that the output array is

Re: Potential stack overflow in incremental base backup

2024-04-10 Thread Thomas Munro
On Fri, Mar 8, 2024 at 6:53 AM Robert Haas wrote: > ... We could > avoid transposing relative block numbers to absolute block numbers > whenever start_blkno is 0, ... Could we just write the blocks directly into the output array, and then transpose them directly in place if start_blkno > 0?

Re: Potential stack overflow in incremental base backup

2024-03-22 Thread Thomas Munro
On Fri, Mar 8, 2024 at 6:53 AM Robert Haas wrote: > But I think that's really only necessary if we're actually going to > get rid of the idea of segmented relations altogether, which I don't > think is happening at least for v17, and maybe not ever. Yeah, I consider the feedback on ext4's size

Re: Potential stack overflow in incremental base backup

2024-03-07 Thread Robert Haas
On Wed, Mar 6, 2024 at 6:29 AM Alvaro Herrera wrote: > On 2024-Mar-06, Thomas Munro wrote: > > Even on the heap, 16GB is too much to assume we can allocate during a > > base backup. I don't claim that's a real-world problem for > > incremental backup right now in master, because I don't have any

Re: Potential stack overflow in incremental base backup

2024-03-07 Thread Robert Haas
On Wed, Mar 6, 2024 at 12:44 AM Thomas Munro wrote: > I'll have to move that sucker onto the heap (we banned C99 variable > length arrays and we don't use nonstandard alloca()), but I think the > coding in master is already a bit dodgy: that's 131072 * > sizeof(BlockNumber) = 512kB with default

Re: Potential stack overflow in incremental base backup

2024-03-06 Thread Alvaro Herrera
On 2024-Mar-06, Thomas Munro wrote: > Even on the heap, 16GB is too much to assume we can allocate during a > base backup. I don't claim that's a real-world problem for > incremental backup right now in master, because I don't have any > evidence that anyone ever really uses --with-segsize (do

Potential stack overflow in incremental base backup

2024-03-05 Thread Thomas Munro
Hi Robert, I was rebasing my patch to convert RELSEG_SIZE into an initdb-time setting, and thus runtime variable, and I noticed this stack variable in src/backend/backup/basebackup_incremental.c: GetFileBackupMethod(IncrementalBackupInfo *ib, const char *path,