Re: Ubsan complaint on kestrel

2025-03-03 Thread Tom Lane
Andres Freund writes: > ../../../../../home/andres/src/postgresql/src/backend/nodes/nodeFuncs.c:2712:6: > runtime error: call to function assign_query_collations_walker through > pointer to incorrect function type 'bool (*)(struct Node *, void *)' > /srv/dev/build/postgres/m-dev-assert-clang-san

Re: Ubsan complaint on kestrel

2025-03-03 Thread Andres Freund
Hi, On 2025-03-03 16:49:09 -0500, Tom Lane wrote: > Andres Freund writes: > > I wish the sanitizer treated mismatches of void * arguments against a "real > > type" different from other mismatches, but ... > > Indeed. I think we have enough coverage of that via compile-time > checks, though -- i

Re: Ubsan complaint on kestrel

2025-03-03 Thread Andres Freund
Hi, On 2025-03-03 15:00:43 -0500, Tom Lane wrote: > Andres Freund writes: > > I just upgraded buildfarm animal kestrel (a buildfarm animal running with > > ubsan) to a newer version of clang. Unfortunately this causes it to fail. > > ... > > Tom, do you see any r

Re: Ubsan complaint on kestrel

2025-03-03 Thread Tom Lane
Andres Freund writes: > I just upgraded buildfarm animal kestrel (a buildfarm animal running with > ubsan) to a newer version of clang. Unfortunately this causes it to fail. > ... > Tom, do you see any reason to not instead do the typecase inside > string_compare()?

Ubsan complaint on kestrel

2025-03-03 Thread Andres Freund
Hi, I just upgraded buildfarm animal kestrel (a buildfarm animal running with ubsan) to a newer version of clang. Unfortunately this causes it to fail. The buildfarm output doesn't contain enough information to debug that unfortunately. But I triggered it manually and meson-logs/testlog.tx

Re: Fix some ubsan/asan related issues

2024-10-01 Thread Tristan Partin
On Mon Sep 16, 2024 at 9:29 AM EDT, Junwang Zhao wrote: I tried your v1-0002, it works at compile phase but failed to run initdb with the following leak detected: = ==64983==ERROR: LeakSanitizer: detected memory leaks Direct leak o

Re: Fix some ubsan/asan related issues

2024-09-16 Thread Junwang Zhao
Hi Tristan, On Tue, Feb 6, 2024 at 11:53 AM Tristan Partin wrote: > > On Tue Jan 30, 2024 at 3:58 PM CST, Andres Freund wrote: > > Hi, > > > > On 2024-01-30 09:59:25 -0600, Tristan Partin wrote: > > > From 331cec1c9db6ff60dcc6d9ba62a9c8be4e5e95ed Mon Sep 17 00:00:00 2001 > > > From: Tristan Parti

Re: Fix some ubsan/asan related issues

2024-02-05 Thread Tristan Partin
On Tue Jan 30, 2024 at 3:58 PM CST, Andres Freund wrote: Hi, On 2024-01-30 09:59:25 -0600, Tristan Partin wrote: > From 331cec1c9db6ff60dcc6d9ba62a9c8be4e5e95ed Mon Sep 17 00:00:00 2001 > From: Tristan Partin > Date: Mon, 29 Jan 2024 18:03:39 -0600 > Subject: [PATCH v1 1/3] Refuse to register m

Re: Fix some ubsan/asan related issues

2024-01-31 Thread Tristan Partin
On Tue Jan 30, 2024 at 10:00 PM CST, Alexander Lakhin wrote: Hello, 30.01.2024 18:57, Tristan Partin wrote: > Patch 1: > > Passing NULL as a second argument to memcpy breaks ubsan, ... Maybe you would like to fix also one more similar place, reached with: create extension xml

Re: Fix some ubsan/asan related issues

2024-01-30 Thread Alexander Lakhin
Hello, 30.01.2024 18:57, Tristan Partin wrote: Patch 1: Passing NULL as a second argument to memcpy breaks ubsan, ... Maybe you would like to fix also one more similar place, reached with: create extension xml2; select xslt_process('', $$http://www.w3.org/1999/XSL

Re: Fix some ubsan/asan related issues

2024-01-30 Thread Andres Freund
Hi, On 2024-01-30 09:59:25 -0600, Tristan Partin wrote: > From 331cec1c9db6ff60dcc6d9ba62a9c8be4e5e95ed Mon Sep 17 00:00:00 2001 > From: Tristan Partin > Date: Mon, 29 Jan 2024 18:03:39 -0600 > Subject: [PATCH v1 1/3] Refuse to register message in LogLogicalMessage if > NULL > If this occurs, t

Re: Fix some ubsan/asan related issues

2024-01-30 Thread Andres Freund
Hi, On 2024-01-30 22:05:28 +0200, Heikki Linnakangas wrote: > On 30/01/2024 17:57, Tristan Partin wrote: > > In my effort to try to see if the test suite would pass with asan > > enabled, I ran into a max_stack_depth issue. I tried maxing it out > > (hence, the patch), but that still didn't remedy

Re: Fix some ubsan/asan related issues

2024-01-30 Thread Heikki Linnakangas
On 30/01/2024 17:57, Tristan Partin wrote: Patch 1: Passing NULL as a second argument to memcpy breaks ubsan, and there didn't seem to be anything preventing that in the LogLogicalMessage() codepath. Here is a preventative measure in LogLogicalMessage() and an Assert() in CopyXLogRecord

Re: Fix some ubsan/asan related issues

2024-01-30 Thread Tristan Partin
Spend so much time writing out the email, I once again forget attachments...UGH. -- Tristan Partin Neon (https://neon.tech) From 331cec1c9db6ff60dcc6d9ba62a9c8be4e5e95ed Mon Sep 17 00:00:00 2001 From: Tristan Partin Date: Mon, 29 Jan 2024 18:03:39 -0600 Subject: [PATCH v1 1/3] Refuse to registe

Fix some ubsan/asan related issues

2024-01-30 Thread Tristan Partin
Patch 1: Passing NULL as a second argument to memcpy breaks ubsan, and there didn't seem to be anything preventing that in the LogLogicalMessage() codepath. Here is a preventative measure in LogLogicalMessage() and an Assert() in CopyXLogRecordToWAL(). Patch 2: Support building

Re: UBSan pointer overflow in xlogreader.c

2023-12-08 Thread Robert Haas
On Thu, Dec 7, 2023 at 10:18 PM Thomas Munro wrote: > On Fri, Dec 8, 2023 at 3:57 AM Robert Haas wrote: > > On Tue, Dec 5, 2023 at 4:01 PM Nathan Bossart > > wrote: > > > +1 > > > > So, Thomas ... any chance you could commit this? So that my patch > > stops making cfbot sad? > > Done. Thanks b

Re: UBSan pointer overflow in xlogreader.c

2023-12-07 Thread Thomas Munro
On Fri, Dec 8, 2023 at 3:57 AM Robert Haas wrote: > On Tue, Dec 5, 2023 at 4:01 PM Nathan Bossart > wrote: > > +1 > > So, Thomas ... any chance you could commit this? So that my patch > stops making cfbot sad? Done. Thanks both for the reviews.

Re: UBSan pointer overflow in xlogreader.c

2023-12-07 Thread Robert Haas
On Tue, Dec 5, 2023 at 4:01 PM Nathan Bossart wrote: > +1 So, Thomas ... any chance you could commit this? So that my patch stops making cfbot sad? -- Robert Haas EDB: http://www.enterprisedb.com

Re: UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Nathan Bossart
On Tue, Dec 05, 2023 at 03:48:33PM -0500, Robert Haas wrote: > The patch LGTM, too. Thanks for investigating and writing the code. > The part about how the reserved kernel memory prevents the bug from > appearing on 32-bit systems but not 64-bit systems running in 32-bit > mode is pretty interestin

Re: UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Robert Haas
On Tue, Dec 5, 2023 at 1:04 PM Nathan Bossart wrote: > On Wed, Dec 06, 2023 at 12:03:53AM +1300, Thomas Munro wrote: > > xlogreader.c has a pointer overflow bug, as revealed by the > > combination of -fsanitize=undefined -m32, the new 039_end_of_wal.pl > > test and Robert's incremental backup patc

Re: UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Nathan Bossart
On Wed, Dec 06, 2023 at 12:03:53AM +1300, Thomas Munro wrote: > xlogreader.c has a pointer overflow bug, as revealed by the > combination of -fsanitize=undefined -m32, the new 039_end_of_wal.pl > test and Robert's incremental backup patch[1]. The bad code tests > whether an object could fit using

UBSan pointer overflow in xlogreader.c

2023-12-05 Thread Thomas Munro
Hi, xlogreader.c has a pointer overflow bug, as revealed by the combination of -fsanitize=undefined -m32, the new 039_end_of_wal.pl test and Robert's incremental backup patch[1]. The bad code tests whether an object could fit using something like base + size <= end, which can be converted to some

Re: Improving asan/ubsan support

2023-12-01 Thread Alexander Lakhin
aren't produced with -O0, -O1... Maybe it's another gcc bug, I'm not sure how to deal with it. (I can research this issue, if it makes any sense.) So I would say that cost of providing/maintaining full support for asan (hwasan), ubsan is not near zero, unfortunately. I would estimate it

Re: Improving asan/ubsan support

2023-12-01 Thread Tristan Partin
On Fri Dec 1, 2023 at 3:00 AM CST, Alexander Lakhin wrote: [ this thread separated from [1] as the discussion focus shifted ] H Andres, 29.11.2023 22:39, Andres Freund wrote: >> I use the following: >> ASAN_OPTIONS=detect_leaks=0:abort_on_error=1:print_stacktrace=1:\ >> disable_coredump=0:stric

Improving asan/ubsan support

2023-12-01 Thread Alexander Lakhin
[ this thread separated from [1] as the discussion focus shifted ] H Andres, 29.11.2023 22:39, Andres Freund wrote: I use the following: ASAN_OPTIONS=detect_leaks=0:abort_on_error=1:print_stacktrace=1:\ disable_coredump=0:strict_string_checks=1:check_initialization_order=1:\ strict_init_order=1

Re: ubsan

2023-09-27 Thread Alexander Lakhin
Hello Andres, 22.11.2022 02:15, Andres Freund wrote: Hi, On 2022-09-29 18:17:55 -0700, Andres Freund wrote: Attached is a rebased version of this patch. Hopefully with a reasonable amount of comments? I kind of wanted to add a comment to reached_main, but it just seems to end up restating the

Re: ubsan

2022-11-21 Thread Andres Freund
Hi, On November 21, 2022 3:42:38 PM PST, Justin Pryzby wrote: >On Mon, Nov 21, 2022 at 03:15:03PM -0800, Andres Freund wrote: >> Hi, >> >> On 2022-09-29 18:17:55 -0700, Andres Freund wrote: >> > Attached is a rebased version of this patch. Hopefully with a reasonable >> > amount of comments? I

Re: ubsan

2022-11-21 Thread Justin Pryzby
On Mon, Nov 21, 2022 at 03:15:03PM -0800, Andres Freund wrote: > Hi, > > On 2022-09-29 18:17:55 -0700, Andres Freund wrote: > > Attached is a rebased version of this patch. Hopefully with a reasonable > > amount of comments? I kind of wanted to add a comment to reached_main, but > > it > > just

Re: ubsan

2022-11-21 Thread Andres Freund
Hi, On 2022-09-29 18:17:55 -0700, Andres Freund wrote: > Attached is a rebased version of this patch. Hopefully with a reasonable > amount of comments? I kind of wanted to add a comment to reached_main, but it > just seems to end up restating the variable name... I've now pushed a version of thi

Re: ubsan fails on 32bit builds

2022-11-17 Thread Thomas Munro
On Fri, Nov 18, 2022 at 9:13 AM Andres Freund wrote: > Agreed. I had started on a set of patches for some of the SHM_QUEUE uses, but > somehow we ended up a bit stuck on the naming of dlist_delete variant that > afterwards zeroes next/prev so we can replace SHMQueueIsDetached() uses. > > Should pr

Re: ubsan fails on 32bit builds

2022-11-17 Thread Tom Lane
Andres Freund writes: > On 2022-11-17 14:20:47 -0500, Robert Haas wrote: >> Not that I object to a targeted fix > Should we backpatch this fix? Likely this doesn't cause active breakage > outside of 32bit builds under ubsan, but that's not an unreasonable thi

Re: ubsan fails on 32bit builds

2022-11-17 Thread Andres Freund
back into waitQueue->links. Which we then can enqueue > > again. > > Not that I object to a targeted fix Should we backpatch this fix? Likely this doesn't cause active breakage outside of 32bit builds under ubsan, but that's not an unreasonable thing to want to do in

Re: ubsan fails on 32bit builds

2022-11-17 Thread Robert Haas
On Wed, Nov 16, 2022 at 8:42 PM Andres Freund wrote: > Afaict the problem is that > proc = (PGPROC *) &(waitQueue->links); > > is a gross gross hack - this isn't actually a PGPROC, it's pointing to an > SHM_QUEUE, but *not* one embedded in PGPROC. It kinda works because ->links >

Re: ubsan fails on 32bit builds

2022-11-16 Thread Andres Freund
the point of this hack, even leaving ubsan's valid complaints > aside. Why bother having this, sometimes, fake PGPROC pointer when we could > just use a SHM_QUEUE* to determine the insertion point? As done in the attached patch. With this ubsan passes both on 32bit and 64bit. Greetings

ubsan fails on 32bit builds

2022-11-16 Thread Andres Freund
Hi, I am working on polishing my patch to make CI use sanitizers. Unfortunately using -fsanitize=alignment,undefined causes tests to fail on 32bit builds. https://cirrus-ci.com/task/5092504471601152 https://api.cirrus-ci.com/v1/artifact/task/5092504471601152/testrun/build-32/testrun/recovery/022_

Re: ubsan

2022-09-29 Thread Andres Freund
9 -0700 Subject: [PATCH v2 1/3] Add a hacky workaround to make ubsan and ps_status.c compatible At least on linux set_ps_display() breaks /proc/$pid/environ. The sanitizer library uses /proc/$pid/environ to implement getenv() because it wants to work independent of libc. When just using undefined

Re: ubsan

2022-03-25 Thread Andres Freund
Hi, On 2022-03-23 15:55:28 -0700, Andres Freund wrote: > Originally I'd planned to mix them into existing members, but I think it'd be > better to have dedicated ones. Applied for a few new buildfarm names for: > {gcc,clang}-{-fsanitize=undefined,-fsanitize=address}. They're now enabled... taman

Re: ubsan

2022-03-25 Thread David Steele
On 3/23/22 16:55, Andres Freund wrote: It's particularly impressive that the cost of running with ASAN is *so* much lower than valgrind. On my workstation a check-world with -fsanitize=alignment,undefined,address takes 3min17s, vs 1min10s or so without -fsanitize. Not something to always use, b

Re: ubsan

2022-03-23 Thread Noah Misch
On Wed, Mar 23, 2022 at 03:58:09PM -0400, Tom Lane wrote: > Andres Freund writes: > > I think we should backpatch both, based on the reasoning in > > 46ab07ffda9d6c8e63360ded2d4568aa160a7700 ? > > Yeah, I suppose. Is anyone going to step up and run a buildfarm > m

Re: ubsan

2022-03-23 Thread Tom Lane
Andres Freund writes: > Running with asan found an existing use-after-free bug in pg_waldump (*), a > bug in > dshash_seq_next() next that probably can't be hit in HEAD and a bug in my > shared memory stats patch. I count that as a success. Nice! regards, tom lane

Re: ubsan

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 13:12:34 -0700, Andres Freund wrote: > I'm planning to enable it on two of mine. Looks like gcc and clang find > slightly different things, so I was intending to enable it on one of each. Originally I'd planned to mix them into existing members, but I think it'd be better to hav

Re: ubsan

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 15:58:09 -0400, Tom Lane wrote: > Andres Freund writes: > > I think we should backpatch both, based on the reasoning in > > 46ab07ffda9d6c8e63360ded2d4568aa160a7700 ? > > Yeah, I suppose. Is anyone going to step up and run a buildfarm > member w

Re: ubsan

2022-03-23 Thread Tom Lane
Andres Freund writes: > I think we should backpatch both, based on the reasoning in > 46ab07ffda9d6c8e63360ded2d4568aa160a7700 ? Yeah, I suppose. Is anyone going to step up and run a buildfarm member with ubsan enabled? (I'm already checking -fsanitize=alignment on longfin, b

Re: ubsan

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 11:21:37 -0700, Andres Freund wrote: > On 2022-03-23 13:54:50 -0400, Tom Lane wrote: > > Andres Freund writes: > > > I tried to run postgres with ubsan to debug something. > > > > For 0001, could we just replace configure's dlopen check wi

Re: ubsan

2022-03-23 Thread Andres Freund
Hi, On 2022-03-23 13:54:50 -0400, Tom Lane wrote: > Andres Freund writes: > > I tried to run postgres with ubsan to debug something. > > For 0001, could we just replace configure's dlopen check with the > dlsym check? Or are you afraid of reverse-case failures? Yea, I

Re: ubsan

2022-03-23 Thread Tom Lane
Andres Freund writes: > I tried to run postgres with ubsan to debug something. For 0001, could we just replace configure's dlopen check with the dlsym check? Or are you afraid of reverse-case failures? 0002: ugh, but my only real complaint is that __ubsan_default_options needs more t

ubsan

2022-03-23 Thread Andres Freund
Hi, I tried to run postgres with ubsan to debug something. I ran into two main issues: 1) Despite Tom's recent efforts in [1], I see two ubsan failures in HEAD. These are easy enough to fix as per the attached, although the fix for the GetConfigOptionByNum() isn't great -