Hi, On 2026-10-08 14:56:24 +0530, Ashutosh Bapat wrote: > On Thu, May 28, 2026 at 9:37 PM Andres Freund <[email protected]> wrote: > > 3) I think we should make the memory context failures crash. Perhaps by > > emitting WARNINGs and then crashing if any of them occurred. > > > > As we trigger memory context checking during commit/abort handling, just > > using PANIC won't always reliably work, E.g. the AllocSetCheck() in > > AllocSetReset(), can trigger recursive PANICs due to the > > MemoryContextReset() in errstart(), if the corruption in in ErrorContext. > > Whether that's a real issue worth worrying about, IDK. > > > > 4) For this I prototyped making the valgrind annotations more generic and > > using the address sanitizer interface to mark memory as poisoned / > > unpoisoned. That doesn't provide quite all the checking that valgrind > > can > > do (it doesn't track uninitialized memory), but it's considerably better > > than our memory context checking, and *much* *much* faster than valgrind. > > IIUC, you are proposing to use sanitizer interface to implement > crashing behaviour mentioned in 3? Or these two are separate things?
Separate. Using asan is pretty expensive (~2-3x slower tests, considerably higher memory usage), there's absolutely no reason to not have our own memory context checking infrastructure only really be usable if you use sanitizers. I think there's also corruption that only our internal checking will find. Running all tests without asan: real 1m34.240s user 6m27.832s sys 4m20.235s with asan: real 3m31.336s user 12m44.840s sys 9m13.033s > > 5) I think we should add a mode to all our allocators that just turns every > > allocation into a large allocation (perhaps with something slightly > > different for slab). In that mode asan, valgrind, et al would be more > > precise. Of course that'll have rather deleterious performance effects, > > but I think it'd still be quite useful for debugging. > > I think this has an advantage of its own, we will benefit from memory > sanity check improvements in system calls without necessarily > implementing them in PostgreSQL and also use-after-free failure not > necessarily covered by other proposals. I'm not really following, which memory sanity checking in system calls are you thinking of? FWIW, I'd started hacking on the asan piece a while back, here's my last WIP patches on this topic. I ran out of time to work on this at the time, and haven't found my way back since. No guarantees that anything therein is correct. Greetings, Andres Freund
>From a8bc48b660604a846c6a84ad77cd407236401c4e Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Tue, 26 May 2026 16:52:19 -0400 Subject: [PATCH v2 1/3] WIP: mmgr: Use larger and guaranteed to exist sentinel --- src/include/pg_config_manual.h | 11 ++++ src/include/utils/memdebug.h | 21 +++++--- src/backend/utils/mmgr/aset.c | 82 +++++++++++++++-------------- src/backend/utils/mmgr/bump.c | 20 +++---- src/backend/utils/mmgr/generation.c | 20 +++---- src/backend/utils/mmgr/mcxt.c | 6 +-- src/backend/utils/mmgr/slab.c | 16 +++--- 7 files changed, 88 insertions(+), 88 deletions(-) diff --git a/src/include/pg_config_manual.h b/src/include/pg_config_manual.h index 521b49b8888..25b77470cef 100644 --- a/src/include/pg_config_manual.h +++ b/src/include/pg_config_manual.h @@ -278,6 +278,17 @@ #define MEMORY_CONTEXT_CHECKING #endif +/* + * Number of bytes to extend each allocation by, to check for buffer + * overflows. It's probably good for this to be >= sizeof(size_t), to be able + * to detect overflows in arrays. + */ +#if defined(USE_ASSERT_CHECKING) || defined(USE_VALGRIND) +#define MEMORY_CONTEXT_SENTINEL_SIZE 16 +#else +#define MEMORY_CONTEXT_SENTINEL_SIZE 0 +#endif + /* * Define this to cause palloc()'d memory to be filled with random data, to * facilitate catching code that depends on the contents of uninitialized diff --git a/src/include/utils/memdebug.h b/src/include/utils/memdebug.h index ceb6864d435..4e73e6fbb00 100644 --- a/src/include/utils/memdebug.h +++ b/src/include/utils/memdebug.h @@ -53,20 +53,27 @@ set_sentinel(void *base, Size offset) { char *ptr = (char *) base + offset; - VALGRIND_MAKE_MEM_UNDEFINED(ptr, 1); - *ptr = 0x7E; - VALGRIND_MAKE_MEM_NOACCESS(ptr, 1); + VALGRIND_MAKE_MEM_UNDEFINED(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); + memset(ptr, 0x7E, MEMORY_CONTEXT_SENTINEL_SIZE); + VALGRIND_MAKE_MEM_NOACCESS(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); } static inline bool sentinel_ok(const void *base, Size offset) { const char *ptr = (const char *) base + offset; - bool ret; + bool ret = true; - VALGRIND_MAKE_MEM_DEFINED(ptr, 1); - ret = *ptr == 0x7E; - VALGRIND_MAKE_MEM_NOACCESS(ptr, 1); + VALGRIND_MAKE_MEM_DEFINED(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); + for (Size i = 0; i < MEMORY_CONTEXT_SENTINEL_SIZE; i++) + { + if (*ptr++ != 0x7e) + { + ret = false; + break; + } + } + VALGRIND_MAKE_MEM_NOACCESS(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); return ret; } diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c index 6a9ea367107..75eb294e9da 100644 --- a/src/backend/utils/mmgr/aset.c +++ b/src/backend/utils/mmgr/aset.c @@ -743,12 +743,8 @@ AllocSetAllocLarge(MemoryContext context, Size size, int flags) /* validate 'size' is within the limits for the given 'flags' */ MemoryContextCheckSize(context, size, flags); -#ifdef MEMORY_CONTEXT_CHECKING - /* ensure there's always space for the sentinel byte */ - chunk_size = MAXALIGN(size + 1); -#else - chunk_size = MAXALIGN(size); -#endif + /* ensure there's space for the sentinel, if needed */ + chunk_size = MAXALIGN(size + MEMORY_CONTEXT_SENTINEL_SIZE); blksize = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ; block = (AllocBlock) malloc(blksize); @@ -771,7 +767,7 @@ AllocSetAllocLarge(MemoryContext context, Size size, int flags) #ifdef MEMORY_CONTEXT_CHECKING chunk->requested_size = size; /* set mark to catch clobber of "unused" space */ - Assert(size < chunk_size); + Assert(size + MEMORY_CONTEXT_SENTINEL_SIZE <= chunk_size); set_sentinel(MemoryChunkGetPointer(chunk), size); #endif #ifdef RANDOMIZE_ALLOCATED_MEMORY @@ -820,6 +816,13 @@ AllocSetAllocChunkFromBlock(MemoryContext context, AllocBlock block, chunk = (MemoryChunk *) (block->freeptr); + /* + * Ensure there's space for the sentinel, if needed. Note this is + * intentionally done after determining fidx, the space for the sentinel + * is in addition to what the size class guarantees. + */ + chunk_size += MEMORY_CONTEXT_SENTINEL_SIZE; + /* Prepare to initialize the chunk header. */ VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ); @@ -832,8 +835,8 @@ AllocSetAllocChunkFromBlock(MemoryContext context, AllocBlock block, #ifdef MEMORY_CONTEXT_CHECKING chunk->requested_size = size; /* set mark to catch clobber of "unused" space */ - if (size < chunk_size) - set_sentinel(MemoryChunkGetPointer(chunk), size); + Assert(size + MEMORY_CONTEXT_SENTINEL_SIZE <= chunk_size); + set_sentinel(MemoryChunkGetPointer(chunk), size); #endif #ifdef RANDOMIZE_ALLOCATED_MEMORY /* fill the allocated space with junk */ @@ -884,11 +887,11 @@ AllocSetAllocFromNewBlock(MemoryContext context, Size size, int flags, * left in the block, this loop cannot iterate more than * ALLOCSET_NUM_FREELISTS-1 times. */ - while (availspace >= ((1 << ALLOC_MINBITS) + ALLOC_CHUNKHDRSZ)) + while (availspace >= ((1 << ALLOC_MINBITS) + ALLOC_CHUNKHDRSZ + MEMORY_CONTEXT_SENTINEL_SIZE)) { AllocFreeListLink *link; MemoryChunk *chunk; - Size availchunk = availspace - ALLOC_CHUNKHDRSZ; + Size availchunk = availspace - ALLOC_CHUNKHDRSZ - MEMORY_CONTEXT_SENTINEL_SIZE; int a_fidx = AllocSetFreeIndex(availchunk); /* @@ -907,8 +910,8 @@ AllocSetAllocFromNewBlock(MemoryContext context, Size size, int flags, /* Prepare to initialize the chunk header. */ VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ); - block->freeptr += (availchunk + ALLOC_CHUNKHDRSZ); - availspace -= (availchunk + ALLOC_CHUNKHDRSZ); + block->freeptr += (availchunk + ALLOC_CHUNKHDRSZ + MEMORY_CONTEXT_SENTINEL_SIZE); + availspace -= (availchunk + ALLOC_CHUNKHDRSZ + MEMORY_CONTEXT_SENTINEL_SIZE); /* store the freelist index in the value field */ MemoryChunkSetHdrMask(chunk, block, a_fidx, MCTX_ASET_ID); @@ -942,7 +945,8 @@ AllocSetAllocFromNewBlock(MemoryContext context, Size size, int flags, * If initBlockSize is less than ALLOC_CHUNK_LIMIT, we could need more * space... but try to keep it a power of 2. */ - required_size = chunk_size + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ; + required_size = chunk_size + ALLOC_BLOCKHDRSZ + + ALLOC_CHUNKHDRSZ + MEMORY_CONTEXT_SENTINEL_SIZE; while (blksize < required_size) blksize <<= 1; @@ -1036,11 +1040,10 @@ AllocSetAlloc(MemoryContext context, Size size, int flags) * If one is found, remove it from the free list, make it again a member * of the alloc set and return its data address. * - * Note that we don't attempt to ensure there's space for the sentinel - * byte here. We expect a large proportion of allocations to be for sizes - * which are already a power of 2. If we were to always make space for a - * sentinel byte in MEMORY_CONTEXT_CHECKING builds, then we'd end up - * doubling the memory requirements for such allocations. + * Note that we always have space for the sentinel. To avoid wasting a lot + * of space - we expect a large proportion of allocations to be for sizes + * which are already a power of 2 - the space for the sentinel is added + * after the rounding to power of 2. */ fidx = AllocSetFreeIndex(size); chunk = set->freelist[fidx]; @@ -1060,9 +1063,11 @@ AllocSetAlloc(MemoryContext context, Size size, int flags) #ifdef MEMORY_CONTEXT_CHECKING chunk->requested_size = size; + /* set mark to catch clobber of "unused" space */ - if (size < GetChunkSizeFromFreeListIdx(fidx)) - set_sentinel(MemoryChunkGetPointer(chunk), size); + /* space for sentinel is added in addition to the freelist size */ + Assert(size <= GetChunkSizeFromFreeListIdx(fidx)); + set_sentinel(MemoryChunkGetPointer(chunk), size); #endif #ifdef RANDOMIZE_ALLOCATED_MEMORY /* fill the allocated space with junk */ @@ -1092,7 +1097,7 @@ AllocSetAlloc(MemoryContext context, Size size, int flags) * If there is enough room in the active allocation block, we will put the * chunk into that block. Else must start a new one. */ - if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ))) + if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ + MEMORY_CONTEXT_SENTINEL_SIZE))) return AllocSetAllocFromNewBlock(context, size, flags, fidx); /* There's enough space on the current block, so allocate from that */ @@ -1195,10 +1200,9 @@ AllocSetFree(void *pointer) elog(ERROR, "detected double pfree in %s %p", set->header.name, chunk); /* Test for someone scribbling on unused space in chunk */ - if (chunk->requested_size < GetChunkSizeFromFreeListIdx(fidx)) - if (!sentinel_ok(pointer, chunk->requested_size)) - elog(WARNING, "detected write past chunk end in %s %p", - set->header.name, chunk); + if (!sentinel_ok(pointer, chunk->requested_size)) + elog(WARNING, "detected write past chunk end in %s %p", + set->header.name, chunk); #endif #ifdef CLOBBER_FREED_MEMORY @@ -1275,18 +1279,13 @@ AllocSetRealloc(void *pointer, Size size, int flags) #ifdef MEMORY_CONTEXT_CHECKING /* Test for someone scribbling on unused space in chunk */ - Assert(chunk->requested_size < oldchksize); if (!sentinel_ok(pointer, chunk->requested_size)) elog(WARNING, "detected write past chunk end in %s %p", set->header.name, chunk); #endif -#ifdef MEMORY_CONTEXT_CHECKING - /* ensure there's always space for the sentinel byte */ - chksize = MAXALIGN(size + 1); -#else - chksize = MAXALIGN(size); -#endif + /* ensure there's space for the sentinel, if needed */ + chksize = MAXALIGN(size + MEMORY_CONTEXT_SENTINEL_SIZE); /* Do the realloc */ blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ; @@ -1398,10 +1397,9 @@ AllocSetRealloc(void *pointer, Size size, int flags) elog(ERROR, "detected realloc of freed chunk in %s %p", set->header.name, chunk); /* Test for someone scribbling on unused space in chunk */ - if (chunk->requested_size < oldchksize) - if (!sentinel_ok(pointer, chunk->requested_size)) - elog(WARNING, "detected write past chunk end in %s %p", - set->header.name, chunk); + if (!sentinel_ok(pointer, chunk->requested_size)) + elog(WARNING, "detected write past chunk end in %s %p", + set->header.name, chunk); #endif /* @@ -1435,8 +1433,7 @@ AllocSetRealloc(void *pointer, Size size, int flags) oldchksize - size); /* set mark to catch clobber of "unused" space */ - if (size < oldchksize) - set_sentinel(pointer, size); + set_sentinel(pointer, size); #else /* !MEMORY_CONTEXT_CHECKING */ /* @@ -1752,6 +1749,13 @@ AllocSetCheck(MemoryContext context) chsize = GetChunkSizeFromFreeListIdx(fidx); /* aligned chunk size */ + /* + * Need to include the overhead for the sentinel in the + * calculation, otherwise the offset of the next chunk would + * be wrong. + */ + chsize += MEMORY_CONTEXT_SENTINEL_SIZE; + /* * Check the stored block offset correctly references this * block. diff --git a/src/backend/utils/mmgr/bump.c b/src/backend/utils/mmgr/bump.c index 9bb579935db..63decf89453 100644 --- a/src/backend/utils/mmgr/bump.c +++ b/src/backend/utils/mmgr/bump.c @@ -324,12 +324,8 @@ BumpAllocLarge(MemoryContext context, Size size, int flags) /* validate 'size' is within the limits for the given 'flags' */ MemoryContextCheckSize(context, size, flags); -#ifdef MEMORY_CONTEXT_CHECKING - /* ensure there's always space for the sentinel byte */ - chunk_size = MAXALIGN(size + 1); -#else - chunk_size = MAXALIGN(size); -#endif + /* ensure there's space for the sentinel, if needed */ + chunk_size = MAXALIGN(size + MEMORY_CONTEXT_SENTINEL_SIZE); required_size = chunk_size + Bump_CHUNKHDRSZ; blksize = required_size + Bump_BLOCKHDRSZ; @@ -357,7 +353,7 @@ BumpAllocLarge(MemoryContext context, Size size, int flags) chunk->requested_size = size; /* set mark to catch clobber of "unused" space */ - Assert(size < chunk_size); + Assert(size + MEMORY_CONTEXT_SENTINEL_SIZE <= chunk_size); set_sentinel(MemoryChunkGetPointer(chunk), size); #endif #ifdef RANDOMIZE_ALLOCATED_MEMORY @@ -421,7 +417,7 @@ BumpAllocChunkFromBlock(MemoryContext context, BumpBlock *block, Size size, MemoryChunkSetHdrMask(chunk, block, chunk_size, MCTX_BUMP_ID); chunk->requested_size = size; /* set mark to catch clobber of "unused" space */ - Assert(size < chunk_size); + Assert(size + MEMORY_CONTEXT_SENTINEL_SIZE <= chunk_size); set_sentinel(MemoryChunkGetPointer(chunk), size); #ifdef RANDOMIZE_ALLOCATED_MEMORY @@ -523,12 +519,8 @@ BumpAlloc(MemoryContext context, Size size, int flags) Assert(BumpIsValid(set)); -#ifdef MEMORY_CONTEXT_CHECKING - /* ensure there's always space for the sentinel byte */ - chunk_size = MAXALIGN(size + 1); -#else - chunk_size = MAXALIGN(size); -#endif + /* ensure there's space for the sentinel, if needed */ + chunk_size = MAXALIGN(size + MEMORY_CONTEXT_SENTINEL_SIZE); /* * If requested size exceeds maximum for chunks we hand the request off to diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c index 609c9bdc9a6..805f2cd34cc 100644 --- a/src/backend/utils/mmgr/generation.c +++ b/src/backend/utils/mmgr/generation.c @@ -372,12 +372,8 @@ GenerationAllocLarge(MemoryContext context, Size size, int flags) /* validate 'size' is within the limits for the given 'flags' */ MemoryContextCheckSize(context, size, flags); -#ifdef MEMORY_CONTEXT_CHECKING - /* ensure there's always space for the sentinel byte */ - chunk_size = MAXALIGN(size + 1); -#else - chunk_size = MAXALIGN(size); -#endif + /* ensure there's space for the sentinel, if needed */ + chunk_size = MAXALIGN(size + MEMORY_CONTEXT_SENTINEL_SIZE); required_size = chunk_size + Generation_CHUNKHDRSZ; blksize = required_size + Generation_BLOCKHDRSZ; @@ -407,7 +403,7 @@ GenerationAllocLarge(MemoryContext context, Size size, int flags) #ifdef MEMORY_CONTEXT_CHECKING chunk->requested_size = size; /* set mark to catch clobber of "unused" space */ - Assert(size < chunk_size); + Assert(size + MEMORY_CONTEXT_SENTINEL_SIZE <= chunk_size); set_sentinel(MemoryChunkGetPointer(chunk), size); #endif #ifdef RANDOMIZE_ALLOCATED_MEMORY @@ -559,12 +555,8 @@ GenerationAlloc(MemoryContext context, Size size, int flags) Assert(GenerationIsValid(set)); -#ifdef MEMORY_CONTEXT_CHECKING - /* ensure there's always space for the sentinel byte */ - chunk_size = MAXALIGN(size + 1); -#else - chunk_size = MAXALIGN(size); -#endif + /* ensure there's space for the sentinel, if needed */ + chunk_size = MAXALIGN(size + MEMORY_CONTEXT_SENTINEL_SIZE); /* * If requested size exceeds maximum for chunks we hand the request off to @@ -1207,7 +1199,7 @@ GenerationCheck(MemoryContext context) name, block, chunk); /* check sentinel */ - Assert(chunk->requested_size < chunksize); + Assert(chunk->requested_size + MEMORY_CONTEXT_SENTINEL_SIZE <= chunksize); if (!sentinel_ok(chunk, Generation_CHUNKHDRSZ + chunk->requested_size)) elog(WARNING, "problem in Generation %s: detected write past chunk end in block %p, chunk %p", name, block, chunk); diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c index 917cc0ac771..5284c365a32 100644 --- a/src/backend/utils/mmgr/mcxt.c +++ b/src/backend/utils/mmgr/mcxt.c @@ -1529,10 +1529,8 @@ MemoryContextAllocAligned(MemoryContext context, */ alloc_size = size + PallocAlignedExtraBytes(alignto); -#ifdef MEMORY_CONTEXT_CHECKING - /* ensure there's space for a sentinel byte */ - alloc_size += 1; -#endif + /* ensure there's space for the sentinel */ + alloc_size += MEMORY_CONTEXT_SENTINEL_SIZE; /* * Perform the actual allocation, but do not pass down MCXT_ALLOC_ZERO. diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c index 2ad325547fd..d5cae029cad 100644 --- a/src/backend/utils/mmgr/slab.c +++ b/src/backend/utils/mmgr/slab.c @@ -342,12 +342,8 @@ SlabContextCreate(MemoryContext parent, chunkSize = sizeof(MemoryChunk *); /* length of the maxaligned chunk including the chunk header */ -#ifdef MEMORY_CONTEXT_CHECKING - /* ensure there's always space for the sentinel byte */ - fullChunkSize = Slab_CHUNKHDRSZ + MAXALIGN(chunkSize + 1); -#else - fullChunkSize = Slab_CHUNKHDRSZ + MAXALIGN(chunkSize); -#endif + /* ensure there's space for the sentinel, if needed */ + fullChunkSize = Slab_CHUNKHDRSZ + MAXALIGN(chunkSize + MEMORY_CONTEXT_SENTINEL_SIZE); Assert(fullChunkSize <= MEMORYCHUNK_MAX_VALUE); @@ -541,7 +537,7 @@ SlabAllocSetupNewChunk(MemoryContext context, SlabBlock *block, #ifdef MEMORY_CONTEXT_CHECKING chunk->requested_size = size; /* slab mark to catch clobber of "unused" space */ - Assert(slab->chunkSize < (slab->fullChunkSize - Slab_CHUNKHDRSZ)); + Assert(slab->chunkSize + MEMORY_CONTEXT_SENTINEL_SIZE <= (slab->fullChunkSize - Slab_CHUNKHDRSZ)); set_sentinel(MemoryChunkGetPointer(chunk), size); VALGRIND_MAKE_MEM_NOACCESS(((char *) chunk) + Slab_CHUNKHDRSZ + slab->chunkSize, @@ -755,7 +751,7 @@ SlabFree(void *pointer) elog(ERROR, "detected double pfree in %s %p", slab->header.name, chunk); /* Test for someone scribbling on unused space in chunk */ - Assert(slab->chunkSize < (slab->fullChunkSize - Slab_CHUNKHDRSZ)); + Assert(slab->chunkSize + MEMORY_CONTEXT_SENTINEL_SIZE <= (slab->fullChunkSize - Slab_CHUNKHDRSZ)); if (!sentinel_ok(pointer, slab->chunkSize)) elog(WARNING, "detected write past chunk end in %s %p", slab->header.name, chunk); @@ -1165,8 +1161,8 @@ SlabCheck(MemoryContext context) elog(WARNING, "problem in slab %s: bogus block link in block %p, chunk %p", name, block, chunk); - /* check the sentinel byte is intact */ - Assert(slab->chunkSize < (slab->fullChunkSize - Slab_CHUNKHDRSZ)); + /* check the sentinel is intact */ + Assert(slab->chunkSize + MEMORY_CONTEXT_SENTINEL_SIZE <= (slab->fullChunkSize - Slab_CHUNKHDRSZ)); if (!sentinel_ok(chunk, Slab_CHUNKHDRSZ + slab->chunkSize)) elog(WARNING, "problem in slab %s: detected write past chunk end in block %p, chunk %p", name, block, chunk); base-commit: e01fc6315f19a794301ca8889d37ae629862e2ac -- 2.54.0.450.g9ac3f193c0
>From f329f7a382ae6b165974eca62eb3c3b57c8b4053 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Thu, 23 Apr 2026 19:08:43 -0400 Subject: [PATCH v2 2/3] WIP: mmgr: Improve asan support for individual allocations --- src/include/pg_config_manual.h | 25 ++++- src/include/utils/memdebug.h | 75 +++++++++++-- src/backend/access/nbtree/nbtpage.c | 10 +- src/backend/nodes/list.c | 20 ++-- src/backend/postmaster/autovacuum.c | 6 +- src/backend/postmaster/pmchild.c | 4 +- src/backend/storage/aio/method_worker.c | 4 +- src/backend/storage/aio/read_stream.c | 22 ++-- src/backend/storage/buffer/bufmgr.c | 42 ++++---- src/backend/storage/buffer/localbuf.c | 6 +- src/backend/utils/cache/inval.c | 14 +-- src/backend/utils/cache/plancache.c | 5 +- src/backend/utils/hash/dynahash.c | 20 ++-- src/backend/utils/misc/ps_status.c | 4 +- src/backend/utils/mmgr/alignedalloc.c | 24 ++--- src/backend/utils/mmgr/aset.c | 120 ++++++++++----------- src/backend/utils/mmgr/bump.c | 32 +++--- src/backend/utils/mmgr/generation.c | 68 ++++++------ src/backend/utils/mmgr/mcxt.c | 39 ++++--- src/backend/utils/mmgr/memdebug.c | 4 +- src/backend/utils/mmgr/slab.c | 46 ++++---- src/backend/utils/sort/logtape.c | 8 +- src/interfaces/libpq/fe-auth-oauth.c | 6 +- src/test/modules/test_escape/test_escape.c | 12 +-- 24 files changed, 345 insertions(+), 271 deletions(-) diff --git a/src/include/pg_config_manual.h b/src/include/pg_config_manual.h index 25b77470cef..2420d4188be 100644 --- a/src/include/pg_config_manual.h +++ b/src/include/pg_config_manual.h @@ -260,6 +260,21 @@ */ /* #define USE_VALGRIND */ +/* + * If we're being compiled with asan support, automatically turn on USE_ASAN. + */ +#ifdef __SANITIZE_ADDRESS__ +#define USE_ASAN +#endif + +/* + * If either valgrind or asan is in use, default to annotating memory with + * additional information. + */ +#if defined(USE_VALGRIND) || defined(USE_ASAN) +#define USE_MEMORY_ANNOTATIONS +#endif + /* * Define this to cause pfree()'d memory to be cleared immediately, to * facilitate catching bugs that refer to already-freed values. @@ -270,11 +285,11 @@ #endif /* - * Define this to check memory allocation errors (scribbling on more - * bytes than were allocated). Right now, this gets defined - * automatically if --enable-cassert or USE_VALGRIND. + * Define this to check memory allocation errors (scribbling on more bytes + * than were allocated). Right now, this gets defined automatically if + * --enable-cassert, USE_VALGRIND or asan is used. */ -#if defined(USE_ASSERT_CHECKING) || defined(USE_VALGRIND) +#if defined(USE_ASSERT_CHECKING) || defined(USE_MEMORY_ANNOTATIONS) #define MEMORY_CONTEXT_CHECKING #endif @@ -283,7 +298,7 @@ * overflows. It's probably good for this to be >= sizeof(size_t), to be able * to detect overflows in arrays. */ -#if defined(USE_ASSERT_CHECKING) || defined(USE_VALGRIND) +#if defined(USE_ASSERT_CHECKING) || defined(USE_MEMORY_ANNOTATIONS) #define MEMORY_CONTEXT_SENTINEL_SIZE 16 #else #define MEMORY_CONTEXT_SENTINEL_SIZE 0 diff --git a/src/include/utils/memdebug.h b/src/include/utils/memdebug.h index 4e73e6fbb00..0fc511a06bf 100644 --- a/src/include/utils/memdebug.h +++ b/src/include/utils/memdebug.h @@ -17,9 +17,12 @@ #ifndef MEMDEBUG_H #define MEMDEBUG_H + #ifdef USE_VALGRIND + #include <valgrind/memcheck.h> -#else + +#else /* !USE_VALGRIND */ #define VALGRIND_CHECK_MEM_IS_DEFINED(addr, size) do {} while (0) #define VALGRIND_CREATE_MEMPOOL(context, redzones, zeroed) do {} while (0) #define VALGRIND_DESTROY_MEMPOOL(context) do {} while (0) @@ -30,7 +33,63 @@ #define VALGRIND_MEMPOOL_FREE(context, addr) do {} while (0) #define VALGRIND_MEMPOOL_CHANGE(context, optr, nptr, size) do {} while (0) #define VALGRIND_MEMPOOL_TRIM(context, addr, size) do {} while (0) -#endif +#endif /* !USE_VALGRIND */ + + +#ifdef USE_ASAN + +#define ASAN_DEFINE_REGION_MACROS +#include <sanitizer/asan_interface.h> + +#else /* !__SANITIZE_ADDRESS__ */ + +#define ASAN_POISON_MEMORY_REGION(addr, size) do {} while (0) +#define ASAN_UNPOISON_MEMORY_REGION(addr, size) do {} while (0) + +#endif /* !__SANITIZE_ADDRESS__ */ + + +#define PG_ANNOTATE_MEM_DEFINED(addr, size) \ + do { \ + VALGRIND_MAKE_MEM_DEFINED(addr, size); \ + ASAN_UNPOISON_MEMORY_REGION(addr, size); \ + } while (0) + +#define PG_ANNOTATE_MEM_NOACCESS(addr, size) \ + do { \ + VALGRIND_MAKE_MEM_NOACCESS(addr, size); \ + ASAN_POISON_MEMORY_REGION(addr, size); \ + } while (0) + +#define PG_ANNOTATE_MEM_UNDEFINED(addr, size) \ + do { \ + VALGRIND_MAKE_MEM_UNDEFINED(addr, size); \ + ASAN_UNPOISON_MEMORY_REGION(addr, size); \ + } while (0) + +#define PG_ANNOTATE_MEMPOOL_ALLOC(context, addr, size) \ + do { \ + VALGRIND_MEMPOOL_ALLOC(context, addr, size); \ + ASAN_UNPOISON_MEMORY_REGION(addr, size); \ + } while (0) + +#define PG_ANNOTATE_MEMPOOL_FREE(context, addr) \ + do { \ + VALGRIND_MEMPOOL_FREE(context, addr); \ + /* XXX: without size we can't do anything for asan */ \ + } while (0) + +#define PG_ANNOTATE_MEMPOOL_CHANGE(context, optr, nptr, size) \ + do { \ + VALGRIND_MEMPOOL_CHANGE(context, optr, nptr, size); \ + /* XXX: probably nothing to do here for asan? */ \ + } while (0) + +#define PG_ANNOTATE_MEMPOOL_TRIM(context, addr, size) \ + do { \ + VALGRIND_MEMPOOL_TRIM(context, addr, size); \ + /* XXX: probably nothing to do here for asan? */ \ + } while (0) #ifdef CLOBBER_FREED_MEMORY @@ -39,9 +98,9 @@ static inline void wipe_mem(void *ptr, size_t size) { - VALGRIND_MAKE_MEM_UNDEFINED(ptr, size); + PG_ANNOTATE_MEM_UNDEFINED(ptr, size); memset(ptr, 0x7F, size); - VALGRIND_MAKE_MEM_NOACCESS(ptr, size); + PG_ANNOTATE_MEM_NOACCESS(ptr, size); } #endif /* CLOBBER_FREED_MEMORY */ @@ -53,9 +112,9 @@ set_sentinel(void *base, Size offset) { char *ptr = (char *) base + offset; - VALGRIND_MAKE_MEM_UNDEFINED(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); + PG_ANNOTATE_MEM_UNDEFINED(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); memset(ptr, 0x7E, MEMORY_CONTEXT_SENTINEL_SIZE); - VALGRIND_MAKE_MEM_NOACCESS(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); + PG_ANNOTATE_MEM_NOACCESS(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); } static inline bool @@ -64,7 +123,7 @@ sentinel_ok(const void *base, Size offset) const char *ptr = (const char *) base + offset; bool ret = true; - VALGRIND_MAKE_MEM_DEFINED(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); + PG_ANNOTATE_MEM_DEFINED(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); for (Size i = 0; i < MEMORY_CONTEXT_SENTINEL_SIZE; i++) { if (*ptr++ != 0x7e) @@ -73,7 +132,7 @@ sentinel_ok(const void *base, Size offset) break; } } - VALGRIND_MAKE_MEM_NOACCESS(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); + PG_ANNOTATE_MEM_NOACCESS(ptr, MEMORY_CONTEXT_SENTINEL_SIZE); return ret; } diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 410514d8e10..d3f2568b405 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -962,7 +962,7 @@ _bt_allocbuf(Relation rel, Relation heaprel) */ buf = ExtendBufferedRel(BMR_REL(rel), MAIN_FORKNUM, NULL, EB_LOCK_FIRST); if (!RelationUsesLocalBuffers(rel)) - VALGRIND_MAKE_MEM_DEFINED(BufferGetPage(buf), BLCKSZ); + PG_ANNOTATE_MEM_DEFINED(BufferGetPage(buf), BLCKSZ); /* Initialize the new page before returning it */ page = BufferGetPage(buf); @@ -1029,7 +1029,7 @@ _bt_relbuf(Relation rel, Buffer buf) */ VALGRIND_CHECK_MEM_IS_DEFINED(BufferGetPage(buf), BLCKSZ); if (!RelationUsesLocalBuffers(rel)) - VALGRIND_MAKE_MEM_NOACCESS(BufferGetPage(buf), BLCKSZ); + PG_ANNOTATE_MEM_NOACCESS(BufferGetPage(buf), BLCKSZ); UnlockReleaseBuffer(buf); } @@ -1068,7 +1068,7 @@ _bt_lockbuf(Relation rel, Buffer buf, int access) * lock/pin held, though. */ if (!RelationUsesLocalBuffers(rel)) - VALGRIND_MAKE_MEM_DEFINED(BufferGetPage(buf), BLCKSZ); + PG_ANNOTATE_MEM_DEFINED(BufferGetPage(buf), BLCKSZ); } /* @@ -1087,7 +1087,7 @@ _bt_unlockbuf(Relation rel, Buffer buf) LockBuffer(buf, BUFFER_LOCK_UNLOCK); if (!RelationUsesLocalBuffers(rel)) - VALGRIND_MAKE_MEM_NOACCESS(BufferGetPage(buf), BLCKSZ); + PG_ANNOTATE_MEM_NOACCESS(BufferGetPage(buf), BLCKSZ); } /* @@ -1105,7 +1105,7 @@ _bt_conditionallockbuf(Relation rel, Buffer buf) return false; if (!RelationUsesLocalBuffers(rel)) - VALGRIND_MAKE_MEM_DEFINED(BufferGetPage(buf), BLCKSZ); + PG_ANNOTATE_MEM_DEFINED(BufferGetPage(buf), BLCKSZ); return true; } diff --git a/src/backend/nodes/list.c b/src/backend/nodes/list.c index 51de24eb460..1263fff06d0 100644 --- a/src/backend/nodes/list.c +++ b/src/backend/nodes/list.c @@ -34,13 +34,13 @@ * help flush out now-broken assumptions. Defining DEBUG_LIST_MEMORY_USAGE * while building this file causes the List operations to forcibly move * all cells in a list whenever a cell is added or deleted. In combination - * with MEMORY_CONTEXT_CHECKING and/or Valgrind, this can usually expose + * with MEMORY_CONTEXT_CHECKING, Valgrind or asan, this can usually expose * broken code. It's a bit expensive though, as there's many more palloc * cycles and a lot more data-copying than in a default build. * - * By default, we enable this when building for Valgrind. + * By default, we enable this when building for Valgrind or asan. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS #define DEBUG_LIST_MEMORY_USAGE #endif @@ -192,14 +192,14 @@ enlarge_list(List *list, int min_size) * We must not move the list header, so it's unsafe to try to reclaim * the initial_elements[] space via repalloc. In debugging builds, * however, we can clear that space and/or mark it inaccessible. - * (wipe_mem includes VALGRIND_MAKE_MEM_NOACCESS.) + * (wipe_mem includes PG_ANNOTATE_MEM_NOACCESS.) */ #ifdef CLOBBER_FREED_MEMORY wipe_mem(list->initial_elements, list->max_length * sizeof(ListCell)); #else - VALGRIND_MAKE_MEM_NOACCESS(list->initial_elements, - list->max_length * sizeof(ListCell)); + PG_ANNOTATE_MEM_NOACCESS(list->initial_elements, + list->max_length * sizeof(ListCell)); #endif } else @@ -813,8 +813,8 @@ list_delete_nth_cell(List *list, int n) wipe_mem(list->initial_elements, list->max_length * sizeof(ListCell)); #else - VALGRIND_MAKE_MEM_NOACCESS(list->initial_elements, - list->max_length * sizeof(ListCell)); + PG_ANNOTATE_MEM_NOACCESS(list->initial_elements, + list->max_length * sizeof(ListCell)); #endif } list->elements = newelems; @@ -1025,8 +1025,8 @@ list_delete_first_n(List *list, int n) wipe_mem(list->initial_elements, list->max_length * sizeof(ListCell)); #else - VALGRIND_MAKE_MEM_NOACCESS(list->initial_elements, - list->max_length * sizeof(ListCell)); + PG_ANNOTATE_MEM_NOACCESS(list->initial_elements, + list->max_length * sizeof(ListCell)); #endif } list->elements = newelems; diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index 0ec01c5da6f..b331b38980f 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -351,7 +351,7 @@ typedef struct * avl_dbase structs. Make it global to ensure the compiler doesn't * optimize it away. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS extern avl_dbase *avl_dbase_array; avl_dbase *avl_dbase_array; #endif @@ -1054,7 +1054,7 @@ rebuild_database_list(Oid newdb) /* put all the hash elements into an array */ dbary = palloc_array(avl_dbase, nelems); /* keep Valgrind quiet */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS avl_dbase_array = dbary; #endif @@ -2674,7 +2674,7 @@ deleted: * going away soon, it's not a problem normally. But when using Valgrind, * release some stuff to reduce complaints about leaked storage. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS hash_destroy(table_toast_map); FreeTupleDesc(pg_class_desc); if (bstrategy) diff --git a/src/backend/postmaster/pmchild.c b/src/backend/postmaster/pmchild.c index 312cb2d8aba..174a3cd75e8 100644 --- a/src/backend/postmaster/pmchild.c +++ b/src/backend/postmaster/pmchild.c @@ -64,7 +64,7 @@ dlist_head ActiveChildList; * PMChild structs. Make it global to ensure the compiler doesn't * optimize it away. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS extern PMChild *pmchild_array; PMChild *pmchild_array; #endif @@ -138,7 +138,7 @@ InitPostmasterChildSlots(void) /* Allocate enough slots, and make sure Valgrind doesn't complain */ slots = palloc_array(PMChild, num_pmchild_slots); -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS pmchild_array = slots; #endif diff --git a/src/backend/storage/aio/method_worker.c b/src/backend/storage/aio/method_worker.c index cf75b2816b7..b4ff938dd65 100644 --- a/src/backend/storage/aio/method_worker.c +++ b/src/backend/storage/aio/method_worker.c @@ -921,13 +921,13 @@ IoWorkerMain(const void *startup_data, size_t startup_data_len) * that, explicitly allow access to the memory before reading data * into it. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS { struct iovec *iov; uint16 iov_length = pgaio_io_get_iovec_length(ioh, &iov); for (int i = 0; i < iov_length; i++) - VALGRIND_MAKE_MEM_UNDEFINED(iov[i].iov_base, iov[i].iov_len); + PG_ANNOTATE_MEM_UNDEFINED(iov[i].iov_base, iov[i].iov_len); } #endif diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c index fddf6ef441d..12902ca3681 100644 --- a/src/backend/storage/aio/read_stream.c +++ b/src/backend/storage/aio/read_stream.c @@ -274,14 +274,14 @@ read_stream_get_block(ReadStream *stream, void *per_buffer_data) else { /* - * Tell Valgrind that the per-buffer data is undefined. That replaces - * the "noaccess" state that was set when the consumer moved past this - * entry last time around the queue, and should also catch callbacks - * that fail to initialize data that the buffer consumer later - * accesses. On the first go around, it is undefined already. + * Tell Valgrind et al that the per-buffer data is undefined. That + * replaces the "noaccess" state that was set when the consumer moved + * past this entry last time around the queue, and should also catch + * callbacks that fail to initialize data that the buffer consumer + * later accesses. On the first go around, it is undefined already. */ - VALGRIND_MAKE_MEM_UNDEFINED(per_buffer_data, - stream->per_buffer_data_size); + PG_ANNOTATE_MEM_UNDEFINED(per_buffer_data, + stream->per_buffer_data_size); blocknum = stream->callback(stream, stream->callback_private_data, per_buffer_data); @@ -1294,7 +1294,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data) stream->buffers[stream->queue_size + oldest_buffer_index] = InvalidBuffer; -#if defined(CLOBBER_FREED_MEMORY) || defined(USE_VALGRIND) +#if defined(CLOBBER_FREED_MEMORY) || defined(USE_MEMORY_ANNOTATIONS) /* * The caller will get access to the per-buffer data, until the next call. @@ -1314,10 +1314,10 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data) #if defined(CLOBBER_FREED_MEMORY) /* This also tells Valgrind the memory is "noaccess". */ wipe_mem(prev_per_buffer_data, stream->per_buffer_data_size); -#elif defined(USE_VALGRIND) +#elif defined(USE_MEMORY_ANNOTATIONS) /* Tell it ourselves. */ - VALGRIND_MAKE_MEM_NOACCESS(prev_per_buffer_data, - stream->per_buffer_data_size); + PG_ANNOTATE_MEM_NOACCESS(prev_per_buffer_data, + stream->per_buffer_data_size); #endif } #endif diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index 5c82865a084..37ae6ef244a 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -3368,12 +3368,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy, * false spuriously: when WaitReadBuffers() calls StartBufferIO(), * it'll see that it's now valid. * - * Note: We deliberately avoid a Valgrind client request here. - * Individual access methods can optionally superimpose buffer page - * client requests on top of our client requests to enforce that - * buffers are only accessed while locked (and pinned). It's possible - * that the buffer page is legitimately non-accessible here. We - * cannot meddle with that. + * Note: We deliberately avoid a memory annotation here. Individual + * access methods can optionally superimpose buffer page client + * requests on top of our client requests to enforce that buffers are + * only accessed while locked (and pinned). It's possible that the + * buffer page is legitimately non-accessible here. We cannot meddle + * with that. */ result = (pg_atomic_read_u64(&buf->state) & BM_VALID) != 0; @@ -3502,13 +3502,13 @@ UnpinBufferNoOwner(BufferDesc *buf) uint64 old_buf_state; /* - * Mark buffer non-accessible to Valgrind. + * Annotate buffer as non-accessible. * * Note that the buffer may have already been marked non-accessible * within access method code that enforces that buffers are only * accessed while a buffer lock is held. */ - VALGRIND_MAKE_MEM_NOACCESS(BufHdrGetBlock(buf), BLCKSZ); + PG_ANNOTATE_MEM_NOACCESS(BufHdrGetBlock(buf), BLCKSZ); /* * I'd better not still hold the buffer content lock. Can't use @@ -3542,16 +3542,16 @@ TrackNewBufferPin(Buffer buf) ResourceOwnerRememberBuffer(CurrentResourceOwner, buf); /* - * This is the first pin for this page by this backend, mark its page as - * defined to valgrind. While the page contents might not actually be - * valid yet, we don't currently guarantee that such pages are marked - * undefined or non-accessible. + * This is the first pin for this page by this backend, annotate its page + * as defined. While the page contents might not actually be valid yet, we + * don't currently guarantee that such pages are marked undefined or + * non-accessible. * * It's not necessarily the prettiest to do this here, but otherwise we'd * need this block of code in multiple places. */ - VALGRIND_MAKE_MEM_DEFINED(BufHdrGetBlock(GetBufferDescriptor(buf - 1)), - BLCKSZ); + PG_ANNOTATE_MEM_DEFINED(BufHdrGetBlock(GetBufferDescriptor(buf - 1)), + BLCKSZ); } #define ST_SORT sort_checkpoint_bufferids @@ -5212,8 +5212,8 @@ FlushRelationBuffers(Relation rel) ResourceOwnerEnlarge(CurrentResourceOwner); /* - * Pin/unpin mostly to make valgrind work, but it also seems - * like the right thing to do. + * Pin/unpin mostly to make valgrind et al work, but it also + * seems like the right thing to do. */ PinLocalBuffer(bufHdr, false); @@ -5658,7 +5658,7 @@ UnlockReleaseBuffer(Buffer buffer) if (likely(ref->data.refcount == 0)) { /* See comment in UnpinBufferNoOwner() */ - VALGRIND_MAKE_MEM_NOACCESS(BufHdrGetBlock(buf), BLCKSZ); + PG_ANNOTATE_MEM_NOACCESS(BufHdrGetBlock(buf), BLCKSZ); sub |= BUF_REFCOUNT_ONE; ForgetPrivateRefCountEntry(ref); @@ -8634,9 +8634,9 @@ buffer_readv_complete_one(PgAioTargetData *td, uint8 buf_off, Buffer buffer, * having been marked as inaccessible. The completion might also be * executed in a different process. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS if (!BufferIsPinned(buffer)) - VALGRIND_MAKE_MEM_DEFINED(bufdata, BLCKSZ); + PG_ANNOTATE_MEM_DEFINED(bufdata, BLCKSZ); #endif if (!PageIsVerified((Page) bufdata, tag.blockNum, piv_flags, @@ -8658,9 +8658,9 @@ buffer_readv_complete_one(PgAioTargetData *td, uint8 buf_off, Buffer buffer, *ignored_checksum = true; /* undo what we did above */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS if (!BufferIsPinned(buffer)) - VALGRIND_MAKE_MEM_NOACCESS(bufdata, BLCKSZ); + PG_ANNOTATE_MEM_NOACCESS(bufdata, BLCKSZ); #endif /* diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c index 4870c8e13d0..bda80cd89ab 100644 --- a/src/backend/storage/buffer/localbuf.c +++ b/src/backend/storage/buffer/localbuf.c @@ -852,7 +852,7 @@ PinLocalBuffer(BufferDesc *buf_hdr, bool adjust_usagecount) * GetLocalBufferStorage(). */ if (LocalBufHdrGetBlock(buf_hdr) != NULL) - VALGRIND_MAKE_MEM_DEFINED(LocalBufHdrGetBlock(buf_hdr), BLCKSZ); + PG_ANNOTATE_MEM_DEFINED(LocalBufHdrGetBlock(buf_hdr), BLCKSZ); } LocalRefCount[bufid]++; ResourceOwnerRememberBuffer(CurrentResourceOwner, @@ -890,7 +890,7 @@ UnpinLocalBufferNoOwner(Buffer buffer) pg_atomic_unlocked_write_u64(&buf_hdr->state, buf_state); /* see comment in UnpinBufferNoOwner */ - VALGRIND_MAKE_MEM_NOACCESS(LocalBufHdrGetBlock(buf_hdr), BLCKSZ); + PG_ANNOTATE_MEM_NOACCESS(LocalBufHdrGetBlock(buf_hdr), BLCKSZ); } } @@ -980,7 +980,7 @@ GetLocalBufferStorage(void) * Valgrind doesn't recognize io_uring reads causing undefined memory to * become defined. */ - VALGRIND_MAKE_MEM_DEFINED(this_buf, BLCKSZ); + PG_ANNOTATE_MEM_DEFINED(this_buf, BLCKSZ); return (Block) this_buf; } diff --git a/src/backend/utils/cache/inval.c b/src/backend/utils/cache/inval.c index 81a5d433bc7..3b6d5cd1612 100644 --- a/src/backend/utils/cache/inval.c +++ b/src/backend/utils/cache/inval.c @@ -439,7 +439,7 @@ AddCatcacheInvalidationMessage(InvalidationMsgsGroup *group, * another process has written since, filling the previously uninitialized * bytes */ - VALGRIND_MAKE_MEM_DEFINED(&msg, sizeof(msg)); + PG_ANNOTATE_MEM_DEFINED(&msg, sizeof(msg)); AddInvalidationMessage(group, CatCacheMsgs, &msg); } @@ -457,7 +457,7 @@ AddCatalogInvalidationMessage(InvalidationMsgsGroup *group, msg.cat.dbId = dbId; msg.cat.catId = catId; /* check AddCatcacheInvalidationMessage() for an explanation */ - VALGRIND_MAKE_MEM_DEFINED(&msg, sizeof(msg)); + PG_ANNOTATE_MEM_DEFINED(&msg, sizeof(msg)); AddInvalidationMessage(group, CatCacheMsgs, &msg); } @@ -487,7 +487,7 @@ AddRelcacheInvalidationMessage(InvalidationMsgsGroup *group, invalmsg.rc.dbId = dbId; invalmsg.rc.relId = relId; /* check AddCatcacheInvalidationMessage() for an explanation */ - VALGRIND_MAKE_MEM_DEFINED(&invalmsg, sizeof(invalmsg)); + PG_ANNOTATE_MEM_DEFINED(&invalmsg, sizeof(invalmsg)); AddInvalidationMessage(group, RelCacheMsgs, &invalmsg); } @@ -517,7 +517,7 @@ AddRelsyncInvalidationMessage(InvalidationMsgsGroup *group, invalmsg.rs.dbId = dbId; invalmsg.rs.relid = relId; /* check AddCatcacheInvalidationMessage() for an explanation */ - VALGRIND_MAKE_MEM_DEFINED(&invalmsg, sizeof(invalmsg)); + PG_ANNOTATE_MEM_DEFINED(&invalmsg, sizeof(invalmsg)); AddInvalidationMessage(group, RelCacheMsgs, &invalmsg); } @@ -545,7 +545,7 @@ AddSnapshotInvalidationMessage(InvalidationMsgsGroup *group, invalmsg.sn.dbId = dbId; invalmsg.sn.relId = relId; /* check AddCatcacheInvalidationMessage() for an explanation */ - VALGRIND_MAKE_MEM_DEFINED(&invalmsg, sizeof(invalmsg)); + PG_ANNOTATE_MEM_DEFINED(&invalmsg, sizeof(invalmsg)); AddInvalidationMessage(group, RelCacheMsgs, &invalmsg); } @@ -1759,7 +1759,7 @@ CacheInvalidateSmgr(RelFileLocatorBackend rlocator) msg.sm.backend_lo = rlocator.backend & 0xffff; msg.sm.rlocator = rlocator.locator; /* check AddCatcacheInvalidationMessage() for an explanation */ - VALGRIND_MAKE_MEM_DEFINED(&msg, sizeof(msg)); + PG_ANNOTATE_MEM_DEFINED(&msg, sizeof(msg)); SendSharedInvalidMessages(&msg, 1); } @@ -1787,7 +1787,7 @@ CacheInvalidateRelmap(Oid databaseId) msg.rm.id = SHAREDINVALRELMAP_ID; msg.rm.dbId = databaseId; /* check AddCatcacheInvalidationMessage() for an explanation */ - VALGRIND_MAKE_MEM_DEFINED(&msg, sizeof(msg)); + PG_ANNOTATE_MEM_DEFINED(&msg, sizeof(msg)); SendSharedInvalidMessages(&msg, 1); } diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index a1b406cee29..ccb0c1c2921 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -494,9 +494,10 @@ CompleteCachedPlan(CachedPlanSource *plansource, /* * Also save the result tuple descriptor. PlanCacheComputeResultDesc may * leak some cruft; normally we just accept that to save a copy step, but - * in USE_VALGRIND mode be tidy by running it in the caller's context. + * in USE_MEMORY_ANNOTATIONS mode be tidy by running it in the caller's + * context. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS MemoryContextSwitchTo(oldcxt); plansource->resultDesc = PlanCacheComputeResultDesc(querytree_list); if (plansource->resultDesc) diff --git a/src/backend/utils/hash/dynahash.c b/src/backend/utils/hash/dynahash.c index 0db984a26c0..f4a93c02274 100644 --- a/src/backend/utils/hash/dynahash.c +++ b/src/backend/utils/hash/dynahash.c @@ -241,12 +241,12 @@ struct HTAB Size keysize; /* hash key length in bytes */ /* - * In a USE_VALGRIND build, non-shared hashtables keep an slist chain of - * all the element blocks they have allocated. This pacifies Valgrind, - * which would otherwise often claim that the element blocks are "possibly - * lost" for lack of any non-interior pointers to their starts. + * In a USE_MEMORY_ANNOTATIONS build, non-shared hashtables keep an slist + * chain of all the element blocks they have allocated. This pacifies + * Valgrind, which would otherwise often claim that the element blocks are + * "possibly lost" for lack of any non-interior pointers to their starts. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS slist_head element_blocks; #endif }; @@ -1658,7 +1658,7 @@ element_alloc(HTAB *hashp, int nelem, int freelist_idx) requestSize = nelem * elementSize; /* Add space for slist_node list link if we need one. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS if (!hashp->isshared) requestSize += MAXALIGN(sizeof(slist_node)); #endif @@ -1670,11 +1670,11 @@ element_alloc(HTAB *hashp, int nelem, int freelist_idx) return false; /* - * If USE_VALGRIND, each allocated block of elements of a non-shared - * hashtable is chained into a list, so that Valgrind won't think it's - * been leaked. + * If USE_MEMORY_ANNOTATIONS, each allocated block of elements of a + * non-shared hashtable is chained into a list, so that Valgrind won't + * think it's been leaked. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATION if (hashp->isshared) firstElement = (HASHELEMENT *) allocedBlock; else diff --git a/src/backend/utils/misc/ps_status.c b/src/backend/utils/misc/ps_status.c index cde10dd59d2..d748c72681d 100644 --- a/src/backend/utils/misc/ps_status.c +++ b/src/backend/utils/misc/ps_status.c @@ -106,7 +106,7 @@ static char **save_argv; * data is leaked. To fix that, keep our own statically-allocated copy of the * pointer. (Oddly, this doesn't seem to be a problem for "argv".) */ -#if defined(PS_USE_CLOBBER_ARGV) && defined(USE_VALGRIND) +#if defined(PS_USE_CLOBBER_ARGV) && defined(USE_MEMORY_ANNOTATIONS) extern char **ps_status_new_environ; char **ps_status_new_environ; #endif @@ -219,7 +219,7 @@ save_ps_display_args(int argc, char **argv) environ = new_environ; /* See notes about Valgrind above. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS ps_status_new_environ = new_environ; #endif } diff --git a/src/backend/utils/mmgr/alignedalloc.c b/src/backend/utils/mmgr/alignedalloc.c index 3208f8f256e..65332933e20 100644 --- a/src/backend/utils/mmgr/alignedalloc.c +++ b/src/backend/utils/mmgr/alignedalloc.c @@ -31,7 +31,7 @@ AlignedAllocFree(void *pointer) MemoryChunk *chunk = PointerGetMemoryChunk(pointer); void *unaligned; - VALGRIND_MAKE_MEM_DEFINED(chunk, sizeof(MemoryChunk)); + PG_ANNOTATE_MEM_DEFINED(chunk, sizeof(MemoryChunk)); Assert(!MemoryChunkIsExternal(chunk)); @@ -51,8 +51,8 @@ AlignedAllocFree(void *pointer) * the unaligned chunk, keeping Valgrind happy. Then when we return to * the outer pfree, that will clean up the vchunk for the aligned chunk. */ - VALGRIND_MEMPOOL_ALLOC(GetMemoryChunkContext(unaligned), unaligned, - (char *) pointer - (char *) unaligned); + PG_ANNOTATE_MEMPOOL_ALLOC(GetMemoryChunkContext(unaligned), unaligned, + (char *) pointer - (char *) unaligned); /* Recursively pfree the unaligned chunk */ pfree(unaligned); @@ -76,7 +76,7 @@ AlignedAllocRealloc(void *pointer, Size size, int flags) Size old_size; void *newptr; - VALGRIND_MAKE_MEM_DEFINED(redirchunk, sizeof(MemoryChunk)); + PG_ANNOTATE_MEM_DEFINED(redirchunk, sizeof(MemoryChunk)); alignto = MemoryChunkGetValue(redirchunk); unaligned = MemoryChunkGetBlock(redirchunk); @@ -118,7 +118,7 @@ AlignedAllocRealloc(void *pointer, Size size, int flags) /* Cope cleanly with OOM */ if (unlikely(newptr == NULL)) { - VALGRIND_MAKE_MEM_NOACCESS(redirchunk, sizeof(MemoryChunk)); + PG_ANNOTATE_MEM_NOACCESS(redirchunk, sizeof(MemoryChunk)); return MemoryContextAllocationFailure(ctx, size, flags); } @@ -129,7 +129,7 @@ AlignedAllocRealloc(void *pointer, Size size, int flags) * entire old_size as defined. That's slightly annoying, but probably not * worth improving. */ - VALGRIND_MAKE_MEM_DEFINED(pointer, old_size); + PG_ANNOTATE_MEM_DEFINED(pointer, old_size); memcpy(newptr, pointer, Min(size, old_size)); /* @@ -138,8 +138,8 @@ AlignedAllocRealloc(void *pointer, Size size, int flags) * pfree'ing the old unaligned chunk, keeping Valgrind happy. Then when * we return to repalloc, it will move the vchunk for the aligned chunk. */ - VALGRIND_MEMPOOL_ALLOC(ctx, unaligned, - (char *) pointer - (char *) unaligned); + PG_ANNOTATE_MEMPOOL_ALLOC(ctx, unaligned, + (char *) pointer - (char *) unaligned); pfree(unaligned); @@ -156,13 +156,13 @@ AlignedAllocGetChunkContext(void *pointer) MemoryChunk *redirchunk = PointerGetMemoryChunk(pointer); MemoryContext cxt; - VALGRIND_MAKE_MEM_DEFINED(redirchunk, sizeof(MemoryChunk)); + PG_ANNOTATE_MEM_DEFINED(redirchunk, sizeof(MemoryChunk)); Assert(!MemoryChunkIsExternal(redirchunk)); cxt = GetMemoryChunkContext(MemoryChunkGetBlock(redirchunk)); - VALGRIND_MAKE_MEM_NOACCESS(redirchunk, sizeof(MemoryChunk)); + PG_ANNOTATE_MEM_NOACCESS(redirchunk, sizeof(MemoryChunk)); return cxt; } @@ -179,12 +179,12 @@ AlignedAllocGetChunkSpace(void *pointer) void *unaligned; Size space; - VALGRIND_MAKE_MEM_DEFINED(redirchunk, sizeof(MemoryChunk)); + PG_ANNOTATE_MEM_DEFINED(redirchunk, sizeof(MemoryChunk)); unaligned = MemoryChunkGetBlock(redirchunk); space = GetMemoryChunkSpace(unaligned); - VALGRIND_MAKE_MEM_NOACCESS(redirchunk, sizeof(MemoryChunk)); + PG_ANNOTATE_MEM_NOACCESS(redirchunk, sizeof(MemoryChunk)); return space; } diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c index 75eb294e9da..48d74421d11 100644 --- a/src/backend/utils/mmgr/aset.c +++ b/src/backend/utils/mmgr/aset.c @@ -471,7 +471,7 @@ AllocSetContextCreateInternal(MemoryContext parent, * Valgrind doesn't distinguish between these vchunks and those created by * mcxt.c for the user-accessible-data chunks we allocate. */ - VALGRIND_MEMPOOL_ALLOC(set, set, FIRST_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, set, FIRST_BLOCKHDRSZ); /* Fill in the initial block's block header */ block = KeeperBlock(set); @@ -482,7 +482,7 @@ AllocSetContextCreateInternal(MemoryContext parent, block->next = NULL; /* Mark unallocated space NOACCESS; leave the block header alone. */ - VALGRIND_MAKE_MEM_NOACCESS(block->freeptr, block->endptr - block->freeptr); + PG_ANNOTATE_MEM_NOACCESS(block->freeptr, block->endptr - block->freeptr); /* Remember block as part of block list */ set->blocks = block; @@ -580,7 +580,7 @@ AllocSetReset(MemoryContext context) wipe_mem(datastart, block->freeptr - datastart); #else /* wipe_mem() would have done this */ - VALGRIND_MAKE_MEM_NOACCESS(datastart, block->freeptr - datastart); + PG_ANNOTATE_MEM_NOACCESS(datastart, block->freeptr - datastart); #endif block->freeptr = datastart; block->prev = NULL; @@ -600,7 +600,7 @@ AllocSetReset(MemoryContext context) * the user-data vchunks within will go away in the TRIM below. * Otherwise Valgrind complains about leaked allocations. */ - VALGRIND_MEMPOOL_FREE(set, block); + PG_ANNOTATE_MEMPOOL_FREE(set, block); free(block); } @@ -615,7 +615,7 @@ AllocSetReset(MemoryContext context) * keeper-block header. This gets rid of the vchunks for whatever user * data is getting discarded by the context reset. */ - VALGRIND_MEMPOOL_TRIM(set, set, FIRST_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_TRIM(set, set, FIRST_BLOCKHDRSZ); /* Reset block size allocation sequence, too */ set->nextBlockSize = set->initBlockSize; @@ -705,7 +705,7 @@ AllocSetDelete(MemoryContext context) if (!IsKeeperBlock(set, block)) { /* As in AllocSetReset, free block-header vchunks explicitly */ - VALGRIND_MEMPOOL_FREE(set, block); + PG_ANNOTATE_MEMPOOL_FREE(set, block); free(block); } @@ -752,7 +752,7 @@ AllocSetAllocLarge(MemoryContext context, Size size, int flags) return MemoryContextAllocationFailure(context, size, flags); /* Make a vchunk covering the new block's header */ - VALGRIND_MEMPOOL_ALLOC(set, block, ALLOC_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, block, ALLOC_BLOCKHDRSZ); context->mem_allocated += blksize; @@ -795,11 +795,11 @@ AllocSetAllocLarge(MemoryContext context, Size size, int flags) } /* Ensure any padding bytes are marked NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, - chunk_size - size); + PG_ANNOTATE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, + chunk_size - size); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); return MemoryChunkGetPointer(chunk); } @@ -824,7 +824,7 @@ AllocSetAllocChunkFromBlock(MemoryContext context, AllocBlock block, chunk_size += MEMORY_CONTEXT_SENTINEL_SIZE; /* Prepare to initialize the chunk header. */ - VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ); block->freeptr += (chunk_size + ALLOC_CHUNKHDRSZ); Assert(block->freeptr <= block->endptr); @@ -844,11 +844,11 @@ AllocSetAllocChunkFromBlock(MemoryContext context, AllocBlock block, #endif /* Ensure any padding bytes are marked NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, - chunk_size - size); + PG_ANNOTATE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, + chunk_size - size); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); return MemoryChunkGetPointer(chunk); } @@ -909,7 +909,7 @@ AllocSetAllocFromNewBlock(MemoryContext context, Size size, int flags, chunk = (MemoryChunk *) (block->freeptr); /* Prepare to initialize the chunk header. */ - VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ); block->freeptr += (availchunk + ALLOC_CHUNKHDRSZ + MEMORY_CONTEXT_SENTINEL_SIZE); availspace -= (availchunk + ALLOC_CHUNKHDRSZ + MEMORY_CONTEXT_SENTINEL_SIZE); @@ -921,9 +921,9 @@ AllocSetAllocFromNewBlock(MemoryContext context, Size size, int flags, /* push this chunk onto the free list */ link = GetFreeListLink(chunk); - VALGRIND_MAKE_MEM_DEFINED(link, sizeof(AllocFreeListLink)); + PG_ANNOTATE_MEM_DEFINED(link, sizeof(AllocFreeListLink)); link->next = set->freelist[a_fidx]; - VALGRIND_MAKE_MEM_NOACCESS(link, sizeof(AllocFreeListLink)); + PG_ANNOTATE_MEM_NOACCESS(link, sizeof(AllocFreeListLink)); set->freelist[a_fidx] = chunk; } @@ -969,7 +969,7 @@ AllocSetAllocFromNewBlock(MemoryContext context, Size size, int flags, return MemoryContextAllocationFailure(context, size, flags); /* Make a vchunk covering the new block's header */ - VALGRIND_MEMPOOL_ALLOC(set, block, ALLOC_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, block, ALLOC_BLOCKHDRSZ); context->mem_allocated += blksize; @@ -978,8 +978,8 @@ AllocSetAllocFromNewBlock(MemoryContext context, Size size, int flags, block->endptr = ((char *) block) + blksize; /* Mark unallocated space NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS(block->freeptr, - blksize - ALLOC_BLOCKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(block->freeptr, + blksize - ALLOC_BLOCKHDRSZ); block->prev = NULL; block->next = set->blocks; @@ -1052,14 +1052,14 @@ AllocSetAlloc(MemoryContext context, Size size, int flags) AllocFreeListLink *link = GetFreeListLink(chunk); /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); Assert(fidx == MemoryChunkGetValue(chunk)); /* pop this chunk off the freelist */ - VALGRIND_MAKE_MEM_DEFINED(link, sizeof(AllocFreeListLink)); + PG_ANNOTATE_MEM_DEFINED(link, sizeof(AllocFreeListLink)); set->freelist[fidx] = link->next; - VALGRIND_MAKE_MEM_NOACCESS(link, sizeof(AllocFreeListLink)); + PG_ANNOTATE_MEM_NOACCESS(link, sizeof(AllocFreeListLink)); #ifdef MEMORY_CONTEXT_CHECKING chunk->requested_size = size; @@ -1075,11 +1075,11 @@ AllocSetAlloc(MemoryContext context, Size size, int flags) #endif /* Ensure any padding bytes are marked NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, - GetChunkSizeFromFreeListIdx(fidx) - size); + PG_ANNOTATE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, + GetChunkSizeFromFreeListIdx(fidx) - size); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); return MemoryChunkGetPointer(chunk); } @@ -1115,7 +1115,7 @@ AllocSetFree(void *pointer) MemoryChunk *chunk = PointerGetMemoryChunk(pointer); /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { @@ -1156,7 +1156,7 @@ AllocSetFree(void *pointer) #endif /* As in AllocSetReset, free block-header vchunks explicitly */ - VALGRIND_MEMPOOL_FREE(set, block); + PG_ANNOTATE_MEMPOOL_FREE(set, block); free(block); } @@ -1209,9 +1209,9 @@ AllocSetFree(void *pointer) wipe_mem(pointer, GetChunkSizeFromFreeListIdx(fidx)); #endif /* push this chunk onto the top of the free list */ - VALGRIND_MAKE_MEM_DEFINED(link, sizeof(AllocFreeListLink)); + PG_ANNOTATE_MEM_DEFINED(link, sizeof(AllocFreeListLink)); link->next = set->freelist[fidx]; - VALGRIND_MAKE_MEM_NOACCESS(link, sizeof(AllocFreeListLink)); + PG_ANNOTATE_MEM_NOACCESS(link, sizeof(AllocFreeListLink)); set->freelist[fidx] = chunk; #ifdef MEMORY_CONTEXT_CHECKING @@ -1247,7 +1247,7 @@ AllocSetRealloc(void *pointer, Size size, int flags) int fidx; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { @@ -1295,7 +1295,7 @@ AllocSetRealloc(void *pointer, Size size, int flags) if (newblock == NULL) { /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); return MemoryContextAllocationFailure(&set->header, size, flags); } @@ -1303,7 +1303,7 @@ AllocSetRealloc(void *pointer, Size size, int flags) * Move the block-header vchunk explicitly. (mcxt.c will take care of * moving the vchunk for the user data.) */ - VALGRIND_MEMPOOL_CHANGE(set, block, newblock, ALLOC_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_CHANGE(set, block, newblock, ALLOC_BLOCKHDRSZ); block = newblock; /* updated separately, not to underflow when (oldblksize > blksize) */ @@ -1342,10 +1342,10 @@ AllocSetRealloc(void *pointer, Size size, int flags) * Make sure not to mark too many bytes in case chunk->requested_size * < size < oldchksize. */ -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS if (Min(size, oldchksize) > chunk->requested_size) - VALGRIND_MAKE_MEM_UNDEFINED((char *) pointer + chunk->requested_size, - Min(size, oldchksize) - chunk->requested_size); + PG_ANNOTATE_MEM_UNDEFINED((char *) pointer + chunk->requested_size, + Min(size, oldchksize) - chunk->requested_size); #endif #endif @@ -1363,14 +1363,14 @@ AllocSetRealloc(void *pointer, Size size, int flags) * old portion DEFINED. Make sure not to mark memory beyond the new * allocation in case it's smaller than the old one. */ - VALGRIND_MAKE_MEM_DEFINED(pointer, Min(size, oldchksize)); + PG_ANNOTATE_MEM_DEFINED(pointer, Min(size, oldchksize)); #endif /* Ensure any padding bytes are marked NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS((char *) pointer + size, chksize - size); + PG_ANNOTATE_MEM_NOACCESS((char *) pointer + size, chksize - size); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); return pointer; } @@ -1426,11 +1426,11 @@ AllocSetRealloc(void *pointer, Size size, int flags) * Otherwise, mark the obsolete part NOACCESS. */ if (size > oldrequest) - VALGRIND_MAKE_MEM_UNDEFINED((char *) pointer + oldrequest, - size - oldrequest); + PG_ANNOTATE_MEM_UNDEFINED((char *) pointer + oldrequest, + size - oldrequest); else - VALGRIND_MAKE_MEM_NOACCESS((char *) pointer + size, - oldchksize - size); + PG_ANNOTATE_MEM_NOACCESS((char *) pointer + size, + oldchksize - size); /* set mark to catch clobber of "unused" space */ set_sentinel(pointer, size); @@ -1441,12 +1441,12 @@ AllocSetRealloc(void *pointer, Size size, int flags) * the old request or shrinking it, so we conservatively mark the * entire new allocation DEFINED. */ - VALGRIND_MAKE_MEM_NOACCESS(pointer, oldchksize); - VALGRIND_MAKE_MEM_DEFINED(pointer, size); + PG_ANNOTATE_MEM_NOACCESS(pointer, oldchksize); + PG_ANNOTATE_MEM_DEFINED(pointer, size); #endif /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); return pointer; } @@ -1473,7 +1473,7 @@ AllocSetRealloc(void *pointer, Size size, int flags) if (newPointer == NULL) { /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); return MemoryContextAllocationFailure((MemoryContext) set, size, flags); } @@ -1485,12 +1485,12 @@ AllocSetRealloc(void *pointer, Size size, int flags) * chunk defined to avoid errors as we copy the currently-NOACCESS * trailing bytes. */ - VALGRIND_MAKE_MEM_UNDEFINED(newPointer, size); + PG_ANNOTATE_MEM_UNDEFINED(newPointer, size); #ifdef MEMORY_CONTEXT_CHECKING oldsize = chunk->requested_size; #else oldsize = oldchksize; - VALGRIND_MAKE_MEM_DEFINED(pointer, oldsize); + PG_ANNOTATE_MEM_DEFINED(pointer, oldsize); #endif /* transfer existing data (certain to fit) */ @@ -1515,7 +1515,7 @@ AllocSetGetChunkContext(void *pointer) AllocSet set; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) block = ExternalChunkGetBlock(chunk); @@ -1523,7 +1523,7 @@ AllocSetGetChunkContext(void *pointer) block = (AllocBlock) MemoryChunkGetBlock(chunk); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); Assert(AllocBlockIsValid(block)); set = block->aset; @@ -1543,14 +1543,14 @@ AllocSetGetChunkSpace(void *pointer) int fidx; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { AllocBlock block = ExternalChunkGetBlock(chunk); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); Assert(AllocBlockIsValid(block)); @@ -1561,7 +1561,7 @@ AllocSetGetChunkSpace(void *pointer) Assert(FreeListIdxIsValid(fidx)); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); return GetChunkSizeFromFreeListIdx(fidx) + ALLOC_CHUNKHDRSZ; } @@ -1629,16 +1629,16 @@ AllocSetStats(MemoryContext context, AllocFreeListLink *link = GetFreeListLink(chunk); /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); Assert(MemoryChunkGetValue(chunk) == fidx); - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); freechunks++; freespace += chksz + ALLOC_CHUNKHDRSZ; - VALGRIND_MAKE_MEM_DEFINED(link, sizeof(AllocFreeListLink)); + PG_ANNOTATE_MEM_DEFINED(link, sizeof(AllocFreeListLink)); chunk = link->next; - VALGRIND_MAKE_MEM_NOACCESS(link, sizeof(AllocFreeListLink)); + PG_ANNOTATE_MEM_NOACCESS(link, sizeof(AllocFreeListLink)); } } @@ -1727,7 +1727,7 @@ AllocSetCheck(MemoryContext context) dsize; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { @@ -1786,7 +1786,7 @@ AllocSetCheck(MemoryContext context) /* if chunk is allocated, disallow access to the chunk header */ if (dsize != InvalidAllocSize) - VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ); blk_data += chsize; nchunks++; diff --git a/src/backend/utils/mmgr/bump.c b/src/backend/utils/mmgr/bump.c index 63decf89453..2c750533d7d 100644 --- a/src/backend/utils/mmgr/bump.c +++ b/src/backend/utils/mmgr/bump.c @@ -195,7 +195,7 @@ BumpContextCreate(MemoryContext parent, const char *name, Size minContextSize, /* See comments about Valgrind interactions in aset.c */ VALGRIND_CREATE_MEMPOOL(set, 0, false); /* This vchunk covers the BumpContext and the keeper block header */ - VALGRIND_MEMPOOL_ALLOC(set, set, FIRST_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, set, FIRST_BLOCKHDRSZ); dlist_init(&set->blocks); @@ -276,7 +276,7 @@ BumpReset(MemoryContext context) * header. This gets rid of the vchunks for whatever user data is getting * discarded by the context reset. */ - VALGRIND_MEMPOOL_TRIM(set, set, FIRST_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_TRIM(set, set, FIRST_BLOCKHDRSZ); /* Reset block size allocation sequence, too */ set->nextBlockSize = set->initBlockSize; @@ -335,7 +335,7 @@ BumpAllocLarge(MemoryContext context, Size size, int flags) return MemoryContextAllocationFailure(context, size, flags); /* Make a vchunk covering the new block's header */ - VALGRIND_MEMPOOL_ALLOC(set, block, Bump_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, block, Bump_BLOCKHDRSZ); context->mem_allocated += blksize; @@ -370,11 +370,11 @@ BumpAllocLarge(MemoryContext context, Size size, int flags) #ifdef MEMORY_CONTEXT_CHECKING /* Ensure any padding bytes are marked NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, - chunk_size - size); + PG_ANNOTATE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, + chunk_size - size); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Bump_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Bump_CHUNKHDRSZ); return MemoryChunkGetPointer(chunk); #else @@ -412,7 +412,7 @@ BumpAllocChunkFromBlock(MemoryContext context, BumpBlock *block, Size size, #ifdef MEMORY_CONTEXT_CHECKING /* Prepare to initialize the chunk header. */ - VALGRIND_MAKE_MEM_UNDEFINED(chunk, Bump_CHUNKHDRSZ); + PG_ANNOTATE_MEM_UNDEFINED(chunk, Bump_CHUNKHDRSZ); MemoryChunkSetHdrMask(chunk, block, chunk_size, MCTX_BUMP_ID); chunk->requested_size = size; @@ -426,11 +426,11 @@ BumpAllocChunkFromBlock(MemoryContext context, BumpBlock *block, Size size, #endif /* Ensure any padding bytes are marked NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, - chunk_size - size); + PG_ANNOTATE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, + chunk_size - size); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Bump_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Bump_CHUNKHDRSZ); return MemoryChunkGetPointer(chunk); #else @@ -475,7 +475,7 @@ BumpAllocFromNewBlock(MemoryContext context, Size size, int flags, return MemoryContextAllocationFailure(context, size, flags); /* Make a vchunk covering the new block's header */ - VALGRIND_MEMPOOL_ALLOC(set, block, Bump_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, block, Bump_BLOCKHDRSZ); context->mem_allocated += blksize; @@ -559,7 +559,7 @@ BumpBlockInit(BumpContext *context, BumpBlock *block, Size blksize) block->endptr = ((char *) block) + blksize; /* Mark unallocated space NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS(block->freeptr, blksize - Bump_BLOCKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(block->freeptr, blksize - Bump_BLOCKHDRSZ); } /* @@ -580,7 +580,7 @@ BumpBlockIsEmpty(BumpBlock *block) static inline void BumpBlockMarkEmpty(BumpBlock *block) { -#if defined(USE_VALGRIND) || defined(CLOBBER_FREED_MEMORY) +#if defined(USE_MEMORY_ANNOTATIONS) || defined(CLOBBER_FREED_MEMORY) char *datastart = ((char *) block) + Bump_BLOCKHDRSZ; #endif @@ -588,7 +588,7 @@ BumpBlockMarkEmpty(BumpBlock *block) wipe_mem(datastart, block->freeptr - datastart); #else /* wipe_mem() would have done this */ - VALGRIND_MAKE_MEM_NOACCESS(datastart, block->freeptr - datastart); + PG_ANNOTATE_MEM_NOACCESS(datastart, block->freeptr - datastart); #endif /* Reset the block, but don't return it to malloc */ @@ -625,7 +625,7 @@ BumpBlockFree(BumpContext *set, BumpBlock *block) #endif /* As in aset.c, free block-header vchunks explicitly */ - VALGRIND_MEMPOOL_FREE(set, block); + PG_ANNOTATE_MEMPOOL_FREE(set, block); free(block); } @@ -792,7 +792,7 @@ BumpCheck(MemoryContext context) Size chunksize; /* allow access to the chunk header */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Bump_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Bump_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c index 805f2cd34cc..7481842738b 100644 --- a/src/backend/utils/mmgr/generation.c +++ b/src/backend/utils/mmgr/generation.c @@ -227,7 +227,7 @@ GenerationContextCreate(MemoryContext parent, /* See comments about Valgrind interactions in aset.c */ VALGRIND_CREATE_MEMPOOL(set, 0, false); /* This vchunk covers the GenerationContext and the keeper block header */ - VALGRIND_MEMPOOL_ALLOC(set, set, FIRST_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, set, FIRST_BLOCKHDRSZ); dlist_init(&set->blocks); @@ -323,7 +323,7 @@ GenerationReset(MemoryContext context) * keeper-block header. This gets rid of the vchunks for whatever user * data is getting discarded by the context reset. */ - VALGRIND_MEMPOOL_TRIM(set, set, FIRST_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_TRIM(set, set, FIRST_BLOCKHDRSZ); /* set it so new allocations to make use of the keeper block */ set->block = KeeperBlock(set); @@ -382,7 +382,7 @@ GenerationAllocLarge(MemoryContext context, Size size, int flags) return MemoryContextAllocationFailure(context, size, flags); /* Make a vchunk covering the new block's header */ - VALGRIND_MEMPOOL_ALLOC(set, block, Generation_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, block, Generation_BLOCKHDRSZ); context->mem_allocated += blksize; @@ -415,11 +415,11 @@ GenerationAllocLarge(MemoryContext context, Size size, int flags) dlist_push_head(&set->blocks, &block->node); /* Ensure any padding bytes are marked NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, - chunk_size - size); + PG_ANNOTATE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, + chunk_size - size); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); return MemoryChunkGetPointer(chunk); } @@ -440,7 +440,7 @@ GenerationAllocChunkFromBlock(MemoryContext context, GenerationBlock *block, Generation_CHUNKHDRSZ + chunk_size); /* Prepare to initialize the chunk header. */ - VALGRIND_MAKE_MEM_UNDEFINED(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_UNDEFINED(chunk, Generation_CHUNKHDRSZ); block->nchunks += 1; block->freeptr += (Generation_CHUNKHDRSZ + chunk_size); @@ -460,11 +460,11 @@ GenerationAllocChunkFromBlock(MemoryContext context, GenerationBlock *block, #endif /* Ensure any padding bytes are marked NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, - chunk_size - size); + PG_ANNOTATE_MEM_NOACCESS((char *) MemoryChunkGetPointer(chunk) + size, + chunk_size - size); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); return MemoryChunkGetPointer(chunk); } @@ -507,7 +507,7 @@ GenerationAllocFromNewBlock(MemoryContext context, Size size, int flags, return MemoryContextAllocationFailure(context, size, flags); /* Make a vchunk covering the new block's header */ - VALGRIND_MEMPOOL_ALLOC(set, block, Generation_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(set, block, Generation_BLOCKHDRSZ); context->mem_allocated += blksize; @@ -636,8 +636,8 @@ GenerationBlockInit(GenerationContext *context, GenerationBlock *block, block->endptr = ((char *) block) + blksize; /* Mark unallocated space NOACCESS. */ - VALGRIND_MAKE_MEM_NOACCESS(block->freeptr, - blksize - Generation_BLOCKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(block->freeptr, + blksize - Generation_BLOCKHDRSZ); } /* @@ -647,7 +647,7 @@ GenerationBlockInit(GenerationContext *context, GenerationBlock *block, static inline void GenerationBlockMarkEmpty(GenerationBlock *block) { -#if defined(USE_VALGRIND) || defined(CLOBBER_FREED_MEMORY) +#if defined(USE_MEMORY_ANNOTATIONS) || defined(CLOBBER_FREED_MEMORY) char *datastart = ((char *) block) + Generation_BLOCKHDRSZ; #endif @@ -655,7 +655,7 @@ GenerationBlockMarkEmpty(GenerationBlock *block) wipe_mem(datastart, block->freeptr - datastart); #else /* wipe_mem() would have done this */ - VALGRIND_MAKE_MEM_NOACCESS(datastart, block->freeptr - datastart); + PG_ANNOTATE_MEM_NOACCESS(datastart, block->freeptr - datastart); #endif /* Reset the block, but don't return it to malloc */ @@ -696,7 +696,7 @@ GenerationBlockFree(GenerationContext *set, GenerationBlock *block) #endif /* As in aset.c, free block-header vchunks explicitly */ - VALGRIND_MEMPOOL_FREE(set, block); + PG_ANNOTATE_MEMPOOL_FREE(set, block); free(block); } @@ -718,7 +718,7 @@ GenerationFree(void *pointer) #endif /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { @@ -832,7 +832,7 @@ GenerationRealloc(void *pointer, Size size, int flags) Size oldsize; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { @@ -910,11 +910,11 @@ GenerationRealloc(void *pointer, Size size, int flags) * Otherwise, mark the obsolete part NOACCESS. */ if (size > oldrequest) - VALGRIND_MAKE_MEM_UNDEFINED((char *) pointer + oldrequest, - size - oldrequest); + PG_ANNOTATE_MEM_UNDEFINED((char *) pointer + oldrequest, + size - oldrequest); else - VALGRIND_MAKE_MEM_NOACCESS((char *) pointer + size, - oldsize - size); + PG_ANNOTATE_MEM_NOACCESS((char *) pointer + size, + oldsize - size); /* set mark to catch clobber of "unused" space */ set_sentinel(pointer, size); @@ -925,12 +925,12 @@ GenerationRealloc(void *pointer, Size size, int flags) * the old request or shrinking it, so we conservatively mark the * entire new allocation DEFINED. */ - VALGRIND_MAKE_MEM_NOACCESS(pointer, oldsize); - VALGRIND_MAKE_MEM_DEFINED(pointer, size); + PG_ANNOTATE_MEM_NOACCESS(pointer, oldsize); + PG_ANNOTATE_MEM_DEFINED(pointer, size); #endif /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); return pointer; } @@ -942,7 +942,7 @@ GenerationRealloc(void *pointer, Size size, int flags) if (newPointer == NULL) { /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); return MemoryContextAllocationFailure((MemoryContext) set, size, flags); } @@ -954,11 +954,11 @@ GenerationRealloc(void *pointer, Size size, int flags) * defined to avoid errors as we copy the currently-NOACCESS trailing * bytes. */ - VALGRIND_MAKE_MEM_UNDEFINED(newPointer, size); + PG_ANNOTATE_MEM_UNDEFINED(newPointer, size); #ifdef MEMORY_CONTEXT_CHECKING oldsize = chunk->requested_size; #else - VALGRIND_MAKE_MEM_DEFINED(pointer, oldsize); + PG_ANNOTATE_MEM_DEFINED(pointer, oldsize); #endif /* transfer existing data (certain to fit) */ @@ -981,7 +981,7 @@ GenerationGetChunkContext(void *pointer) GenerationBlock *block; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) block = ExternalChunkGetBlock(chunk); @@ -989,7 +989,7 @@ GenerationGetChunkContext(void *pointer) block = (GenerationBlock *) MemoryChunkGetBlock(chunk); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); Assert(GenerationBlockIsValid(block)); return &block->context->header; @@ -1007,7 +1007,7 @@ GenerationGetChunkSpace(void *pointer) Size chunksize; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { @@ -1020,7 +1020,7 @@ GenerationGetChunkSpace(void *pointer) chunksize = MemoryChunkGetValue(chunk); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); return Generation_CHUNKHDRSZ + chunksize; } @@ -1164,7 +1164,7 @@ GenerationCheck(MemoryContext context) Size chunksize; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Generation_CHUNKHDRSZ); if (MemoryChunkIsExternal(chunk)) { @@ -1209,7 +1209,7 @@ GenerationCheck(MemoryContext context) /* if chunk is allocated, disallow access to the chunk header */ if (chunk->requested_size != InvalidAllocSize) - VALGRIND_MAKE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Generation_CHUNKHDRSZ); } /* diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c index 5284c365a32..43ba11dc5c6 100644 --- a/src/backend/utils/mmgr/mcxt.c +++ b/src/backend/utils/mmgr/mcxt.c @@ -223,12 +223,12 @@ GetMemoryChunkMethodID(const void *pointer) Assert(pointer == (const void *) MAXALIGN(pointer)); /* Allow access to the uint64 header */ - VALGRIND_MAKE_MEM_DEFINED((char *) pointer - sizeof(uint64), sizeof(uint64)); + PG_ANNOTATE_MEM_DEFINED((char *) pointer - sizeof(uint64), sizeof(uint64)); header = *((const uint64 *) ((const char *) pointer - sizeof(uint64))); /* Disallow access to the uint64 header */ - VALGRIND_MAKE_MEM_NOACCESS((char *) pointer - sizeof(uint64), sizeof(uint64)); + PG_ANNOTATE_MEM_NOACCESS((char *) pointer - sizeof(uint64), sizeof(uint64)); return (MemoryContextMethodID) (header & MEMORY_CONTEXT_METHODID_MASK); } @@ -245,12 +245,12 @@ GetMemoryChunkHeader(const void *pointer) uint64 header; /* Allow access to the uint64 header */ - VALGRIND_MAKE_MEM_DEFINED((char *) pointer - sizeof(uint64), sizeof(uint64)); + PG_ANNOTATE_MEM_DEFINED((char *) pointer - sizeof(uint64), sizeof(uint64)); header = *((const uint64 *) ((const char *) pointer - sizeof(uint64))); /* Disallow access to the uint64 header */ - VALGRIND_MAKE_MEM_NOACCESS((char *) pointer - sizeof(uint64), sizeof(uint64)); + PG_ANNOTATE_MEM_NOACCESS((char *) pointer - sizeof(uint64), sizeof(uint64)); return header; } @@ -1254,7 +1254,7 @@ MemoryContextAlloc(MemoryContext context, Size size) */ ret = context->methods->alloc(context, size, 0); - VALGRIND_MEMPOOL_ALLOC(context, ret, size); + PG_ANNOTATE_MEMPOOL_ALLOC(context, ret, size); return ret; } @@ -1278,7 +1278,7 @@ MemoryContextAllocZero(MemoryContext context, Size size) ret = context->methods->alloc(context, size, 0); - VALGRIND_MEMPOOL_ALLOC(context, ret, size); + PG_ANNOTATE_MEMPOOL_ALLOC(context, ret, size); MemSetAligned(ret, 0, size); @@ -1307,7 +1307,7 @@ MemoryContextAllocExtended(MemoryContext context, Size size, int flags) if (unlikely(ret == NULL)) return NULL; - VALGRIND_MEMPOOL_ALLOC(context, ret, size); + PG_ANNOTATE_MEMPOOL_ALLOC(context, ret, size); if ((flags & MCXT_ALLOC_ZERO) != 0) MemSetAligned(ret, 0, size); @@ -1412,8 +1412,7 @@ palloc(Size size) ret = context->methods->alloc(context, size, 0); /* We expect OOM to be handled by the alloc function */ Assert(ret != NULL); - VALGRIND_MEMPOOL_ALLOC(context, ret, size); - + PG_ANNOTATE_MEMPOOL_ALLOC(context, ret, size); return ret; } @@ -1432,7 +1431,7 @@ palloc0(Size size) ret = context->methods->alloc(context, size, 0); /* We expect OOM to be handled by the alloc function */ Assert(ret != NULL); - VALGRIND_MEMPOOL_ALLOC(context, ret, size); + PG_ANNOTATE_MEMPOOL_ALLOC(context, ret, size); MemSetAligned(ret, 0, size); @@ -1459,7 +1458,7 @@ palloc_extended(Size size, int flags) return NULL; } - VALGRIND_MEMPOOL_ALLOC(context, ret, size); + PG_ANNOTATE_MEMPOOL_ALLOC(context, ret, size); if ((flags & MCXT_ALLOC_ZERO) != 0) MemSetAligned(ret, 0, size); @@ -1586,8 +1585,8 @@ MemoryContextAllocAligned(MemoryContext context, * padding bytes before it, and any wasted trailing bytes) will be marked * NOACCESS, which is what we want. */ - VALGRIND_MEMPOOL_FREE(context, unaligned); - VALGRIND_MEMPOOL_ALLOC(context, aligned, size); + PG_ANNOTATE_MEMPOOL_FREE(context, unaligned); + PG_ANNOTATE_MEMPOOL_ALLOC(context, aligned, size); /* Now zero (and make DEFINED) just the aligned chunk, if requested */ if ((flags & MCXT_ALLOC_ZERO) != 0) @@ -1624,13 +1623,13 @@ palloc_aligned(Size size, Size alignto, int flags) void pfree(void *pointer) { -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS MemoryContext context = GetMemoryChunkContext(pointer); #endif MCXT_METHOD(pointer, free_p) (pointer); - VALGRIND_MEMPOOL_FREE(context, pointer); + PG_ANNOTATE_MEMPOOL_FREE(context, pointer); } /* @@ -1640,7 +1639,7 @@ pfree(void *pointer) void * repalloc(void *pointer, Size size) { -#if defined(USE_ASSERT_CHECKING) || defined(USE_VALGRIND) +#if defined(USE_ASSERT_CHECKING) || defined(USE_MEMORY_ANNOTATIONS) MemoryContext context = GetMemoryChunkContext(pointer); #endif void *ret; @@ -1662,7 +1661,7 @@ repalloc(void *pointer, Size size) */ ret = MCXT_METHOD(pointer, realloc) (pointer, size, 0); - VALGRIND_MEMPOOL_CHANGE(context, pointer, ret, size); + PG_ANNOTATE_MEMPOOL_CHANGE(context, pointer, ret, size); return ret; } @@ -1675,7 +1674,7 @@ repalloc(void *pointer, Size size) void * repalloc_extended(void *pointer, Size size, int flags) { -#if defined(USE_ASSERT_CHECKING) || defined(USE_VALGRIND) +#if defined(USE_ASSERT_CHECKING) || defined(USE_MEMORY_ANNOTATIONS) MemoryContext context = GetMemoryChunkContext(pointer); #endif void *ret; @@ -1699,7 +1698,7 @@ repalloc_extended(void *pointer, Size size, int flags) if (unlikely(ret == NULL)) return NULL; - VALGRIND_MEMPOOL_CHANGE(context, pointer, ret, size); + PG_ANNOTATE_MEMPOOL_CHANGE(context, pointer, ret, size); return ret; } @@ -1878,7 +1877,7 @@ MemoryContextAllocHuge(MemoryContext context, Size size) */ ret = context->methods->alloc(context, size, MCXT_ALLOC_HUGE); - VALGRIND_MEMPOOL_ALLOC(context, ret, size); + PG_ANNOTATE_MEMPOOL_ALLOC(context, ret, size); return ret; } diff --git a/src/backend/utils/mmgr/memdebug.c b/src/backend/utils/mmgr/memdebug.c index d6f0dc718c2..2c277d18e26 100644 --- a/src/backend/utils/mmgr/memdebug.c +++ b/src/backend/utils/mmgr/memdebug.c @@ -79,14 +79,14 @@ randomize_mem(char *ptr, size_t size) int ctr; ctr = save_ctr; - VALGRIND_MAKE_MEM_UNDEFINED(ptr, size); + PG_ANNOTATE_MEM_UNDEFINED(ptr, size); while (remaining-- > 0) { *ptr++ = ctr; if (++ctr > 251) ctr = 1; } - VALGRIND_MAKE_MEM_UNDEFINED(ptr - size, size); + PG_ANNOTATE_MEM_UNDEFINED(ptr - size, size); save_ctr = ctr; } diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c index d5cae029cad..80e7efdb9db 100644 --- a/src/backend/utils/mmgr/slab.c +++ b/src/backend/utils/mmgr/slab.c @@ -282,7 +282,7 @@ SlabGetNextFreeChunk(SlabContext *slab, SlabBlock *block) * Pop the chunk from the linked list of free chunks. The pointer to * the next free chunk is stored in the chunk itself. */ - VALGRIND_MAKE_MEM_DEFINED(SlabChunkGetPointer(chunk), sizeof(MemoryChunk *)); + PG_ANNOTATE_MEM_DEFINED(SlabChunkGetPointer(chunk), sizeof(MemoryChunk *)); block->freehead = *(MemoryChunk **) SlabChunkGetPointer(chunk); /* check nothing stomped on the free chunk's memory */ @@ -376,7 +376,7 @@ SlabContextCreate(MemoryContext parent, /* See comments about Valgrind interactions in aset.c */ VALGRIND_CREATE_MEMPOOL(slab, 0, false); /* This vchunk covers the SlabContext only */ - VALGRIND_MEMPOOL_ALLOC(slab, slab, sizeof(SlabContext)); + PG_ANNOTATE_MEMPOOL_ALLOC(slab, slab, sizeof(SlabContext)); /* Fill in SlabContext-specific header fields */ slab->chunkSize = (uint32) chunkSize; @@ -454,7 +454,7 @@ SlabReset(MemoryContext context) #endif /* As in aset.c, free block-header vchunks explicitly */ - VALGRIND_MEMPOOL_FREE(slab, block); + PG_ANNOTATE_MEMPOOL_FREE(slab, block); free(block); context->mem_allocated -= slab->blockSize; @@ -474,7 +474,7 @@ SlabReset(MemoryContext context) #endif /* As in aset.c, free block-header vchunks explicitly */ - VALGRIND_MEMPOOL_FREE(slab, block); + PG_ANNOTATE_MEMPOOL_FREE(slab, block); free(block); context->mem_allocated -= slab->blockSize; @@ -487,7 +487,7 @@ SlabReset(MemoryContext context) * the vchunks for whatever user data is getting discarded by the context * reset. */ - VALGRIND_MEMPOOL_TRIM(slab, slab, sizeof(SlabContext)); + PG_ANNOTATE_MEMPOOL_TRIM(slab, slab, sizeof(SlabContext)); slab->curBlocklistIndex = 0; @@ -530,7 +530,7 @@ SlabAllocSetupNewChunk(MemoryContext context, SlabBlock *block, Assert(SlabChunkMod(slab, block, chunk) == 0); /* Prepare to initialize the chunk header. */ - VALGRIND_MAKE_MEM_UNDEFINED(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_UNDEFINED(chunk, Slab_CHUNKHDRSZ); MemoryChunkSetHdrMask(chunk, block, MAXALIGN(slab->chunkSize), MCTX_SLAB_ID); @@ -539,10 +539,10 @@ SlabAllocSetupNewChunk(MemoryContext context, SlabBlock *block, /* slab mark to catch clobber of "unused" space */ Assert(slab->chunkSize + MEMORY_CONTEXT_SENTINEL_SIZE <= (slab->fullChunkSize - Slab_CHUNKHDRSZ)); set_sentinel(MemoryChunkGetPointer(chunk), size); - VALGRIND_MAKE_MEM_NOACCESS(((char *) chunk) + Slab_CHUNKHDRSZ + - slab->chunkSize, - slab->fullChunkSize - - (slab->chunkSize + Slab_CHUNKHDRSZ)); + PG_ANNOTATE_MEM_NOACCESS(((char *) chunk) + Slab_CHUNKHDRSZ + + slab->chunkSize, + slab->fullChunkSize - + (slab->chunkSize + Slab_CHUNKHDRSZ)); #endif #ifdef RANDOMIZE_ALLOCATED_MEMORY @@ -551,7 +551,7 @@ SlabAllocSetupNewChunk(MemoryContext context, SlabBlock *block, #endif /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); return MemoryChunkGetPointer(chunk); } @@ -590,7 +590,7 @@ SlabAllocFromNewBlock(MemoryContext context, Size size, int flags) return MemoryContextAllocationFailure(context, size, flags); /* Make a vchunk covering the new block's header */ - VALGRIND_MEMPOOL_ALLOC(slab, block, Slab_BLOCKHDRSZ); + PG_ANNOTATE_MEMPOOL_ALLOC(slab, block, Slab_BLOCKHDRSZ); block->slab = slab; context->mem_allocated += slab->blockSize; @@ -732,7 +732,7 @@ SlabFree(void *pointer) int newBlocklistIdx; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); block = MemoryChunkGetBlock(chunk); @@ -829,7 +829,7 @@ SlabFree(void *pointer) #endif /* As in aset.c, free block-header vchunks explicitly */ - VALGRIND_MEMPOOL_FREE(slab, block); + PG_ANNOTATE_MEMPOOL_FREE(slab, block); free(block); slab->header.mem_allocated -= slab->blockSize; @@ -866,12 +866,12 @@ SlabRealloc(void *pointer, Size size, int flags) SlabContext *slab; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); block = MemoryChunkGetBlock(chunk); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); /* * Try to verify that we have a sane block pointer: the block header @@ -902,12 +902,12 @@ SlabGetChunkContext(void *pointer) SlabBlock *block; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); block = MemoryChunkGetBlock(chunk); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); Assert(SlabBlockIsValid(block)); @@ -927,12 +927,12 @@ SlabGetChunkSpace(void *pointer) SlabContext *slab; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); block = MemoryChunkGetBlock(chunk); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); Assert(SlabBlockIsValid(block)); slab = block->slab; @@ -1109,7 +1109,7 @@ SlabCheck(MemoryContext context) slab->isChunkFree[chunkidx] = true; /* read pointer of the next free chunk */ - VALGRIND_MAKE_MEM_DEFINED(MemoryChunkGetPointer(cur_chunk), sizeof(MemoryChunk *)); + PG_ANNOTATE_MEM_DEFINED(MemoryChunkGetPointer(cur_chunk), sizeof(MemoryChunk *)); cur_chunk = *(MemoryChunk **) SlabChunkGetPointer(cur_chunk); } @@ -1146,12 +1146,12 @@ SlabCheck(MemoryContext context) SlabBlock *chunkblock; /* Allow access to the chunk header. */ - VALGRIND_MAKE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_DEFINED(chunk, Slab_CHUNKHDRSZ); chunkblock = (SlabBlock *) MemoryChunkGetBlock(chunk); /* Disallow access to the chunk header. */ - VALGRIND_MAKE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); + PG_ANNOTATE_MEM_NOACCESS(chunk, Slab_CHUNKHDRSZ); /* * check the chunk's blockoffset correctly points back to diff --git a/src/backend/utils/sort/logtape.c b/src/backend/utils/sort/logtape.c index 8e3edcdf1ab..b976cdf1e57 100644 --- a/src/backend/utils/sort/logtape.c +++ b/src/backend/utils/sort/logtape.c @@ -881,8 +881,8 @@ LogicalTapeRewindForRead(LogicalTape *lt, size_t buffer_size) * once. Tell valgrind that its contents are defined, so it * doesn't bleat. */ - VALGRIND_MAKE_MEM_DEFINED(lt->buffer + lt->nbytes, - lt->buffer_size - lt->nbytes); + PG_ANNOTATE_MEM_DEFINED(lt->buffer + lt->nbytes, + lt->buffer_size - lt->nbytes); TapeBlockSetNBytes(lt->buffer, lt->nbytes); ltsWriteBlock(lt->tapeSet, lt->curBlockNumber, lt->buffer); @@ -997,8 +997,8 @@ LogicalTapeFreeze(LogicalTape *lt, TapeShare *share) * small amount of data that we do not fill the buffer even once. Tell * valgrind that its contents are defined, so it doesn't bleat. */ - VALGRIND_MAKE_MEM_DEFINED(lt->buffer + lt->nbytes, - lt->buffer_size - lt->nbytes); + PG_ANNOTATE_MEM_DEFINED(lt->buffer + lt->nbytes, + lt->buffer_size - lt->nbytes); TapeBlockSetNBytes(lt->buffer, lt->nbytes); ltsWriteBlock(lt->tapeSet, lt->curBlockNumber, lt->buffer); diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c index 826f7461cb3..31d7d93e517 100644 --- a/src/interfaces/libpq/fe-auth-oauth.c +++ b/src/interfaces/libpq/fe-auth-oauth.c @@ -1466,7 +1466,7 @@ pqClearOAuthToken(PGconn *conn) static void poison_req_v2(PGoauthBearerRequestV2 *request, bool poison) { -#ifdef USE_VALGRIND +#ifdef USE_MEMORY_ANNOTATIONS void *const base = (char *) request + sizeof(request->v1); const size_t len = sizeof(*request) - sizeof(request->v1); #endif @@ -1482,7 +1482,7 @@ poison_req_v2(PGoauthBearerRequestV2 *request, bool poison) */ Assert(!request->error); - VALGRIND_MAKE_MEM_NOACCESS(base, len); + PG_ANNOTATE_MEM_NOACCESS(base, len); } else { @@ -1493,7 +1493,7 @@ poison_req_v2(PGoauthBearerRequestV2 *request, bool poison) * stack regions, we can't be any stricter without tracking the * original state of the memory. */ - VALGRIND_MAKE_MEM_DEFINED(base, len); + PG_ANNOTATE_MEM_DEFINED(base, len); /* Undo our mask. */ request->issuer = POISON_MASK(request->issuer); diff --git a/src/test/modules/test_escape/test_escape.c b/src/test/modules/test_escape/test_escape.c index 4b556f73891..aa5954b3c53 100644 --- a/src/test/modules/test_escape/test_escape.c +++ b/src/test/modules/test_escape/test_escape.c @@ -224,8 +224,8 @@ test_gb18030_json(pe_test_config *tc) raw_buf = createPQExpBuffer(); appendBinaryPQExpBuffer(raw_buf, input, input_len); appendPQExpBufferStr(raw_buf, NEVER_ACCESS_STR); - VALGRIND_MAKE_MEM_NOACCESS(&raw_buf->data[input_len], - raw_buf->len - input_len); + PG_ANNOTATE_MEM_NOACCESS(&raw_buf->data[input_len], + raw_buf->len - input_len); /* name to describe the test */ testname = createPQExpBuffer(); @@ -716,8 +716,8 @@ test_one_vector_escape(pe_test_config *tc, const pe_test_vector *tv, const pe_te */ appendPQExpBufferStr(raw_buf, NEVER_ACCESS_STR); - VALGRIND_MAKE_MEM_NOACCESS(&raw_buf->data[tv->escape_len], - raw_buf->len - tv->escape_len); + PG_ANNOTATE_MEM_NOACCESS(&raw_buf->data[tv->escape_len], + raw_buf->len - tv->escape_len); } else { @@ -725,8 +725,8 @@ test_one_vector_escape(pe_test_config *tc, const pe_test_vector *tv, const pe_te appendPQExpBufferChar(raw_buf, 0); appendPQExpBufferStr(raw_buf, NEVER_ACCESS_STR); - VALGRIND_MAKE_MEM_NOACCESS(&raw_buf->data[tv->escape_len + 1], - raw_buf->len - tv->escape_len - 1); + PG_ANNOTATE_MEM_NOACCESS(&raw_buf->data[tv->escape_len + 1], + raw_buf->len - tv->escape_len - 1); } /* call the to-be-tested escape function */ -- 2.54.0.450.g9ac3f193c0
>From 9d84396d22c5484fa84934e29eb70a34aaf353aa Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Tue, 26 May 2026 18:12:28 -0400 Subject: [PATCH v2 3/3] Fix spurious valgrind errors within NUMA views Author: Reviewed-by: Discussion: https://postgr.es/m/ Backpatch-through: --- src/include/port/pg_numa.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/include/port/pg_numa.h b/src/include/port/pg_numa.h index 1b668fe1d91..fca9f1318db 100644 --- a/src/include/port/pg_numa.h +++ b/src/include/port/pg_numa.h @@ -20,6 +20,8 @@ extern PGDLLIMPORT int pg_numa_get_max_node(void); #ifdef USE_LIBNUMA +#include "utils/memdebug.h" + /* * This is required on Linux, before pg_numa_query_pages() as we * need to page-fault before move_pages(2) syscall returns valid results. @@ -29,6 +31,14 @@ pg_numa_touch_mem_if_required(void *ptr) { volatile uint64 touch pg_attribute_unused(); + /* + * Other annotations, like in UnpinBufferNoOwner(), might have made this + * inaccessible. We don't know what the state is, so we can't restore the + * prior state afterwards. That seems ok, this isn't used commonly while + * other things are going on. + */ + PG_ANNOTATE_MEM_DEFINED(ptr, sizeof(uint64)); + touch = *(volatile uint64 *) ptr; } -- 2.54.0.450.g9ac3f193c0
