Hi Matthias, On Thu, Dec 12, 2024 at 10:49 PM Matthias van de Meent < boekewurm+postg...@gmail.com> wrote:
> On Thu, 12 Dec 2024 at 22:28, Andreas 'ads' Scherbaum <a...@pgug.de> wrote: > > > > > > Hello, > > > > I'm running a couple of large tests, and in this particular test I have > > a few million tables more. > > > > At some point it fails, and I gathered the following trace: > > > > > > 2024-12-12 22:22:55.307 CET [1496210] ERROR: invalid DSA memory alloc > > request size 1073741824 > > 2024-12-12 22:22:55.307 CET [1496210] BACKTRACE: > > postgres: ads tabletest [local] CREATE TABLE(+0x15e570) > > [0x6309c379c570] > > postgres: ads tabletest [local] CREATE > > TABLE(dshash_find_or_insert+0x1a4) [0x6309c39882d4] > > postgres: ads tabletest [local] CREATE > > TABLE(pgstat_get_entry_ref+0x440) [0x6309c3b0a530] > > It looks like the dshash table used in the pgstats system uses > resize(), which only specifies DSA_ALLOC_ZERO, not DSA_ALLOC_HUGE, > causing issues when the table grows larger than 1 GB. > > I expect that error to disappear when you replace the > dsa_allocate0(...) call in dshash.c's resize function with > dsa_allocate_extended(..., DSA_ALLOC_HUGE | DSA_ALLOC_ZERO) as > attached, but haven't tested it due to a lack of database with > millions of relations. > Can confirm that the crash no longer happens when applying your patch. Was able to both continue the old and crashed test, as well as run a new test: tabletest=# select count(*) from information_schema.tables; count ---------- 20000211 (1 row) Thanks, -- Andreas 'ads' Scherbaum German PostgreSQL User Group European PostgreSQL User Group - Board of Directors Volunteer Regional Contact, Germany - PostgreSQL Project