[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 Iain Sandoe changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #16 from Iain Sandoe --- fixed for 9.3
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #15 from Iain Sandoe --- Author: iains Date: Sun Aug 18 19:11:29 2019 New Revision: 274625 URL: https://gcc.gnu.org/viewcvs?rev=274625=gcc=rev Log: Darwin, libsanitizer - remove reference to __cxa_rethrow_primary_exception. For some Darwin versions the absence of the rethrow_primary_exception symbol causes almost all sanitizer tests to fail. The symbol is not present in libstdc++ and, therefore is is correct to remove the reference to it for all platforms. We do this by adding a new guard "ASAN_HAS_CXA_RETHROW_PRIMARY_EXCEPTION". For the backported version, to avoid bumping the library version, this is only applied for Darwin (because of the way that symbol interposing works on Darwin, this does not alter the exported symbols list). libsanitizer/ 2019-08-18 Iain Sandoe Backport from mainline 2019-06-18 Iain Sandoe PR libsanitizer/87880 * asan/asan_interceptors.h: (ASAN_INTERCEPT___CXA_RETHROW_PRIMARY_EXCEPTION): New. * asan/Makefile.am (DEFS): Add (for Darwin only) ASAN_HAS_CXA_RETHROW_PRIMARY_EXCEPTION, defined to 0. * asan/Makefile.in: Regenerated. Modified: branches/gcc-9-branch/libsanitizer/ChangeLog branches/gcc-9-branch/libsanitizer/asan/Makefile.am branches/gcc-9-branch/libsanitizer/asan/Makefile.in branches/gcc-9-branch/libsanitizer/asan/asan_interceptors.h
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 Iain Sandoe changed: What|Removed |Added Status|WAITING |NEW Summary|[9/10 regression] All macOS |[9 regression] All macOS |asan execution tests FAIL |asan execution tests FAIL --- Comment #14 from Iain Sandoe --- fixed on trunk. NOTE: on earlier Darwin platforms (10.7-10.15) it is highly recommended to build llvm-symbolizer and then: export ASAN_SYMBOLIZER_PATH=/path/to/llvm-symbolizer This is because the "atos" implementation for these versions does not seem to be 100% compatible with the newer sanitisers - the effect is extremely long timeouts on test fails (so we're talking about hours to run the asan.exp suite if that happens). AFAIK, atos is not open-sourced for Darwin, so the best fix is to use the LLVM version - which is. --- The following modification is being tested for inclusion in 9.3 (it just makes the additional check dependent on USING_MAC_INTERPOSE which is fine since the symbol table for libasan is not affected by this on Darwin/Mac OS). There is no soname bump needed for this (although, technically, the fault is not fixed for non-Darwin, in practice the problem has not manifest there anyway) diff --git a/libsanitizer/asan/Makefile.am b/libsanitizer/asan/Makefile.am index 867240d244..6efbc1df7f 100644 --- a/libsanitizer/asan/Makefile.am +++ b/libsanitizer/asan/Makefile.am @@ -5,7 +5,7 @@ gcc_version := $(shell @get_gcc_base_ver@ $(top_srcdir)/../gcc/BASE-VER) DEFS = -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DASAN_HAS_EXCEPTIONS=1 -DASAN_NEEDS_SEGV=1 -DCAN_SANITIZE_UB=0 if USING_MAC_INTERPOSE -DEFS += -DMAC_INTERPOSE_FUNCTIONS -DMISSING_BLOCKS_SUPPORT +DEFS += -DMAC_INTERPOSE_FUNCTIONS -DMISSING_BLOCKS_SUPPORT -DASAN_HAS_CXA_RETHROW_PRIMARY_EXCEPTION=0 endif AM_CXXFLAGS = -Wall -W -Wno-unused-parameter -Wwrite-strings -pedantic -Wno-long-long -fPIC -fno-builtin -fno-exceptions -fno-rtti -fomit-frame-pointer -funwind-tables -fvisibility=hidden -Wno-variadic-macros -fno-ipa-icf AM_CXXFLAGS += $(LIBSTDCXX_RAW_CXX_CXXFLAGS)
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #10 from Jakub Jelinek --- I guess on Linux nothing ever calls __cxa_rethrow_primary_exception, so it isn't a big deal that libasan provides a wrapper for it. If you want to introduce ASAN_HAS_CXA_RETHROW_PRIMARY_EXCEPTION, it would need to be introduced some way that either it would default to 1 and gcc/libsanitizer/asan/Makefile.in was able to override it, so e.g. #ifndef ASAN_HAS_CXA_RETHROW_PRIMARY_EXCEPTION # define ASAN_HAS_CXA_RETHROW_PRIMARY_EXCEPTION 1 #endif in say asan/asan_internal.h and add -DASAN_HAS_CXA_RETHROW_PRIMARY_EXCEPTION=0 to libsanitizer/asan/Makefile.*.
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #9 from Iain Sandoe --- So.. this is somewhat puzzling. Essentially, it seems that on macOS and Linux (at least) currently ASAN_INTERCEPT___CXA_RETHROW_PRIMARY_EXCEPTION is set unconditionally to "on" (for any target that has ASAN_HAS_EXCEPTIONS). libsupc++'s implementation doesn't include this, so .. .. the mystery is not so much why it's failing sometimes on macOS - so much as why it's not failing always, when the c++ library is libstdc++ (and also on Linux). Hypothesis, could be that since the search is being carried out in the flat namespace, any library that happens to pull in libc++ (as a dependent) will cause the libc++abi version of __cxa_rethrow_primary_exception to become visible in that flat namespace. using the following hack the results go from wholesale fail on C to a handfull of fails and on c++ similarly. I guess this could be a dyld bug too - if the lookup is supposed to be a weak reference (not checked). .. obviously, this is incomplete .. it just jams the hook off (not looked at the configuration stuff yet, or whether there's a weak symbol problem). [this appears distinct from failure mode on Darwin10 and also distinct from some fails caused by missing symbolization (atos) on m64 / darwin14] diff --git a/libsanitizer/asan/asan_interceptors.h b/libsanitizer/asan/asan_interceptors.h index b599ebb..10b6d2c 100644 --- a/libsanitizer/asan/asan_interceptors.h +++ b/libsanitizer/asan/asan_interceptors.h @@ -79,7 +79,11 @@ void InitializePlatformInterceptors(); #if ASAN_HAS_EXCEPTIONS && !SANITIZER_WINDOWS && !SANITIZER_SOLARIS && \ !SANITIZER_NETBSD # define ASAN_INTERCEPT___CXA_THROW 1 -# define ASAN_INTERCEPT___CXA_RETHROW_PRIMARY_EXCEPTION 1 +# if ASAN_HAS_CXA_RETHROW_PRIMARY_EXCEPTION +# define ASAN_INTERCEPT___CXA_RETHROW_PRIMARY_EXCEPTION 1 +#else +# define ASAN_INTERCEPT___CXA_RETHROW_PRIMARY_EXCEPTION 0 +# endif # if defined(_GLIBCXX_SJLJ_EXCEPTIONS) || (SANITIZER_IOS && defined(__arm__)) # define ASAN_INTERCEPT__UNWIND_SJLJ_RAISEEXCEPTION 1 # else
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #8 from Iain Sandoe --- I can see this on x86_64-darwin15 .. but not on darwin16, 17 or 18. please can you confirm that you definitely see this on darwin18? (and what your bootstrap compiler / config is). my tests were on 267418 with GCC5 as the bootstrap compiler on x86-64-darwin15 (for Ada support). On darwin16 it makes no difference if I use GCC or clang as the bootstrap compiler, on darwin17 and 18 I've only tried with clang as the bootstrap so far. Do you have SIP enabled, and did you install the target libs before running the tests?
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #7 from Iain Sandoe --- I do not see any wholesale fail of asan when building/testing on darwin16 for minversion darwin14 (using the darwin14 SDK). Here, the symbol shows as dynamically looked-up. (this _is_ a problem in fact [although a different one from what you're seeing]) ** since AFAIR dyld will fall back to looking in /usr/lib for the dynamic lookup and that would resolve a symbol from a different unwinder stack from the one in actual use. *sigh* the presence of two unwinder/c++abi impls. remains the largest pain on the platform. anyway, assuming that the symbol is present in /usr/lib/libc++abi.dylib for both the cases you mention [*and* the mojave dyld lookup rules haven't changed] it's not clear to me why you're failing with a "symbol not found" at runtime. $ nm -mapv x86_64-apple-darwin14/libsanitizer/asan/.libs/libasan.dylib |grep cxa_rethrow_primary_exception 00018180 - 01 FUN _wrap___cxa_rethrow_primary_exception 000c73b0 - 10 STSYM __ZL44substitution___cxa_rethrow_primary_exception 00018180 (__TEXT,__text) non-external (was a private external) _wrap___cxa_rethrow_primary_exception 000c73b0 (__DATA,__interpose) non-external __ZL44substitution___cxa_rethrow_primary_exception (undefined) external ___cxa_rethrow_primary_exception (dynamically looked up)
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #6 from Iain Sandoe --- so I looked a little more at this. As Rainer points out, the libsupc++ implementation does not provide cxa_rethrow_primary_exception whereas the libc++-abi.dylib has it. For the macOS implementations of libstdc.6.dylib and libc++.dylib the symbols from libc++-abi are forwarded via the presented library. For the GCC implementation of libstdc++.dylib, the libsupc++ implementation is statically linked into the library. So - it seems that the build for libsanitiser is either resolving those symbols accidentally from the /usr/lib/libstdc++.dylib (or has been constructed to permit them to be unresolved at link time) - first case seems a build bug - second case they should be guarded by a check for presence at runtime (IIRC there are already an number of symbols that are guarded this way). Longer term: 1. we could add the cxa_rethrow_primary_exception to libsupc++ (will talk to Jonathan about the implications of this) 2. We could split libstdc++ from libsupc++ on Darwin and have the option to build the library on top of libc++abi (forwarding the symbols as the installed system library does). - this is analogous to the need to forward to the unwinder implementation which must always be provided by the shared /usr/lib/system impl on Darwin. It's not 100% clear at the moment if there's state in the abi lib that would mandate that action anyway. At the Cauldron I talked to Jonathan about potentially doing this split, and he didn't at that point have any objection.
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #5 from Iain Sandoe --- (In reply to r...@cebitec.uni-bielefeld.de from comment #4) > > --- Comment #3 from Dominique d'Humieres --- > >> Weird: can you check where the definition of > >> ___cxa_rethrow_primary_exception is coming from in your case? On my > >> 10.14.2 Beta1 system, I only found it in libc++abi.1.dylib. > > > > I see > > > > % nm x86_64-apple-darwin18.2.0/libsanitizer/asan/.libs/libasan.5.dylib | > > grep > > cxa_rethrow_primary_exception > > 000c73b0 s __ZL44substitution___cxa_rethrow_primary_exception > > U ___cxa_rethrow_primary_exception > > 00018170 t _wrap___cxa_rethrow_primary_exception > > Right, that's the reference from libasan. However, this needs to be > resolved from somewhere at runtime, and I don't see how this would happen. if you can invoke a failing test with DYLD_PRINT_LIBRARIES=1 DYLD_PRINT_BINDINGS=1 (+DYLD_LIBRARY_PATH if needed) .. then the binding printout should show you where it's trying to resolve it from. Looking at otool -lv will show you the load commands for the referenced libs. === The 10.6 fails are repeatable for me - good to hear 10.7/darwin11 is OK - I need to figure out when the lib transitioned from working => non-working (might be something associated with 64b code being run from a 32b kernel).
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #4 from ro at CeBiTec dot Uni-Bielefeld.DE --- > --- Comment #3 from Dominique d'Humieres --- >> Weird: can you check where the definition of >> ___cxa_rethrow_primary_exception is coming from in your case? On my >> 10.14.2 Beta1 system, I only found it in libc++abi.1.dylib. > > I see > > % nm x86_64-apple-darwin18.2.0/libsanitizer/asan/.libs/libasan.5.dylib | grep > cxa_rethrow_primary_exception > 000c73b0 s __ZL44substitution___cxa_rethrow_primary_exception > U ___cxa_rethrow_primary_exception > 00018170 t _wrap___cxa_rethrow_primary_exception Right, that's the reference from libasan. However, this needs to be resolved from somewhere at runtime, and I don't see how this would happen.
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #3 from Dominique d'Humieres --- > Weird: can you check where the definition of > ___cxa_rethrow_primary_exception is coming from in your case? On my > 10.14.2 Beta1 system, I only found it in libc++abi.1.dylib. I see % nm x86_64-apple-darwin18.2.0/libsanitizer/asan/.libs/libasan.5.dylib | grep cxa_rethrow_primary_exception 000c73b0 s __ZL44substitution___cxa_rethrow_primary_exception U ___cxa_rethrow_primary_exception 00018170 t _wrap___cxa_rethrow_primary_exception
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 --- Comment #2 from ro at CeBiTec dot Uni-Bielefeld.DE --- > --- Comment #1 from Dominique d'Humieres --- > WORKSFORME on x86_64-apple-darwin18.2.0 configured with > --with-sysroot=/Applications/Xcode-6.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk: > > PASS: c-c++-common/asan/alloca_big_alignment.c -O0 execution test > PASS: c-c++-common/asan/alloca_big_alignment.c -O0 output pattern test Weird: can you check where the definition of ___cxa_rethrow_primary_exception is coming from in your case? On my 10.14.2 Beta1 system, I only found it in libc++abi.1.dylib. > For x86_64-apple-darwin10.8 I have disabled ASAN (--disable-libsanitizer) due > to too many failures. On both x86_64-apple-darwin11.4.2 and i386-apple-darwin11.4.2 I had only and handful of asan failures before the merge.
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 Dominique d'Humieres changed: What|Removed |Added Status|UNCONFIRMED |WAITING Last reconfirmed||2018-11-05 Ever confirmed|0 |1 --- Comment #1 from Dominique d'Humieres --- WORKSFORME on x86_64-apple-darwin18.2.0 configured with --with-sysroot=/Applications/Xcode-6.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk: PASS: c-c++-common/asan/alloca_big_alignment.c -O0 execution test PASS: c-c++-common/asan/alloca_big_alignment.c -O0 output pattern test For x86_64-apple-darwin10.8 I have disabled ASAN (--disable-libsanitizer) due to too many failures.
[Bug sanitizer/87880] [9 regression] All macOS asan execution tests FAIL
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87880 Rainer Orth changed: What|Removed |Added Target Milestone|--- |9.0
