On Wed, Oct 30, 2024 at 5:30 PM Sundar Swamy <[email protected]> wrote:
> Thanks @samuel for the clarification. just curious and a naive question > that up until v25.3 we didn't have to compile protobuf with sanitizer.. we > started to see the TSAN failure with that mentioned stack only from v25.4 > release.?? > Not sure if there was a particular change between those two releases. But the ODR violation is undefined behavior, so "it's working" does not necessarily mean that it is correct. Undefined behavior might happen to work at some point and then fail under different conditions. > > On Wednesday, October 23, 2024 at 6:39:18 AM UTC-7 Samuel Benzaquen wrote: > >> On Tue, Aug 13, 2024 at 3:57 PM Sundar Swamy <[email protected]> wrote: >> >>> Platform: Mac OS,. 10.15 & Mac OS 11.x ( arm64) >>> Compiler: AppleClang 12.0.0.12000032 >>> Mac OSX SDK: 11 >>> I link libprotouf.a to my shared libary (dylib). The dylib is compiled >>> using -fsanitize=thread -O1 but protobuf is compiled as normal ( without >>> any sanitizer flags ) >>> >> >> This is the problem. >> Turning on sanitizers only in part of your binary causes ODR violations. >> https://en.cppreference.com/w/cpp/language/definition >> >> Many parts of protobuf have special logic for sanitizer support, >> including some that change the ABI of the library. >> Even if it didn't change the ABI, mixing sanitized and non-sanitized >> libraries will lead to false positives/negatives. Sanitizers that use >> shadow memory (ASan, MSan, TSan) need all the access to memory to properly >> update the shadow memory. >> For example, if you have part of your binary without TSan instrumentation >> it might miss a happens-before relationship and give false positives. >> >> >>> When i run in thread sanitizer, i get SEGV and here is a sample callstack >>> >>> [2024-08-13T17:26:21.490Z] ThreadSanitizer:DEADLYSIGNAL >>> [2024-08-13T17:26:21.490Z] ==1855==ERROR: ThreadSanitizer: SEGV on unknown >>> address 0x000000000000 (pc 0x0001021e9840 bp 0x7e8000186920 sp >>> 0x7e8000186900 T2629743) [2024-08-13T17:26:21.490Z] ==1855==The signal is >>> caused by a READ memory access. [2024-08-13T17:26:21.490Z] ==1855==Hint: >>> address points to the zero page. >>> ... >>> ... >>> >>> #0 >>> google::protobuf::internal::UntypedMapBase::TableEntryIsNonEmptyList(unsigned >>> int) const map.h:630 (mySample.9.1.2.17.dylib:x86_64+0xe7840) >>> >>> [2024-08-13T17:26:23.250Z] #1 >>> google::protobuf::internal::KeyMapBase<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> > >>> >::FindHelper(absl::lts_20240116::string_view, >>> absl::lts_20240116::container_internal::btree_iterator<absl::lts_20240116::container_internal::btree_node<absl::lts_20240116::container_internal::map_params<google::protobuf::internal::VariantKey, >>> google::protobuf::internal::NodeBase*, >>> std::__1::less<google::protobuf::internal::VariantKey>, >>> google::protobuf::internal::MapAllocator<std::__1::pair<google::protobuf::internal::VariantKey >>> const, google::protobuf::internal::NodeBase*> >, 256, false> >, >>> std::__1::pair<google::protobuf::internal::VariantKey const, >>> google::protobuf::internal::NodeBase*>&, >>> std::__1::pair<google::protobuf::internal::VariantKey const, >>> google::protobuf::internal::NodeBase*>*>*) const map.h:953 >>> (mySample.9.1.2.17.dylib:x86_64+0xe7594) >>> >>> [2024-08-13T17:26:23.251Z] #2 >>> std::__1::pair<google::protobuf::Map<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >::iterator, bool> >>> google::protobuf::Map<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >>> >::TryEmplaceInternal<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> > const, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >(std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> > const&&, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> >&&) map.h:1619 >>> (mySample.9.1.2.17.dylib:x86_64+0xea059) >>> >>> [2024-08-13T17:26:23.251Z] #3 >>> std::__1::pair<google::protobuf::Map<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >::iterator, bool> >>> google::protobuf::Map<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >>> >::ArenaAwareTryEmplace<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> > const, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >(std::__1::integral_constant<bool, false>, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > const&&, std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >&&) map.h:1692 >>> (mySample.9.1.2.17.dylib:x86_64+0xe9f25) >>> >>> [2024-08-13T17:26:23.251Z] #4 >>> std::__1::pair<google::protobuf::Map<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >::iterator, bool> >>> google::protobuf::Map<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >::try_emplace<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> > const, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >(std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> > const&&, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> >&&) map.h:1466 >>> (mySample.9.1.2.17.dylib:x86_64+0xe9e35) >>> >>> [2024-08-13T17:26:23.252Z] #5 >>> std::__1::pair<google::protobuf::Map<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >::iterator, bool> >>> google::protobuf::Map<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> >, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >>> >::insert<std::__1::pair<std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> > const, >>> std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > >, >>> 0>(std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, >>> std::__1::allocator<char> > const, std::__1::basic_string<char, >>> std::__1::char_traits<char>, std::__1::allocator<char> > >&&) map.h:1476 >>> (mySample.9.1.2.17.dylib:x86_64+0xb5643) >>> >>> …. >>> >>> …. >>> >>> … >>> >>> … >>> >>> >>> 2024-08-13T17:26:23.253Z] #18 >>> MySDK::CMySDKSocket::TestMonitorThread() MySDKSocket.cpp:191 >>> (mySample.9.1.2.17.dylib:x86_64+0x1a0b5f) >>> >>> [2024-08-13T17:26:23.253Z] #19 >>> decltype(*(std::__1::forward<MySDK::CMySDKSocket*>(fp0)).*fp()) >>> std::__1::__invoke<void (MySDK::CMySDKSocket::*)(), MySDK::CMySDKSocket*, >>> void>(void (MySDK::CMySDKSocket::*&&)(), MySDK::CMySDKSocket*&&) >>> type_traits:3486 (mySample.9.1.2.17.dylib:x86_64+0x1ac338) >>> >>> [2024-08-13T17:26:23.253Z] #20 void >>> std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, >>> std::__1::default_delete<std::__1::__thread_struct> >, void >>> (MySDK::CMySDKSocket::*)(), MySDK::CMySDKSocket*, >>> 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, >>> std::__1::default_delete<std::__1::__thread_struct> >, void >>> (MySDK::CMySDKSocket::*)(), MySDK::CMySDKSocket*>&, >>> std::__1::__tuple_indices<2ul>) thread:273 >>> (mySample.9.1.2.17.dylib:x86_64+0x1ac122) >>> >>> [2024-08-13T17:26:23.254Z] #21 void* >>> std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, >>> std::__1::default_delete<std::__1::__thread_struct> >, void >>> (MySDK::CMySDKSocket::*)(), MySDK::CMySDKSocket*> >(void*) thread:284 >>> (mySample.9.1.2.17.dylib:x86_64+0x1ab37b) >>> >>> [2024-08-13T17:26:23.254Z] #22 __tsan_thread_start_func <null>:3 >>> (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x2d89b) >>> >>> [2024-08-13T17:26:23.254Z] waitForEchoResponse: 19 >>> >>> #23 _pthread_start <null>:2 (libsystem_pthread.dylib:x86_64+0x6108) >>> >>> [2024-08-13T17:26:23.254Z] #24 thread_start <null>:2 >>> (libsystem_pthread.dylib:x86_64+0x1b8a) >>> >>> [2024-08-13T17:26:23.254Z] >>> >>> [2024-08-13T17:26:23.254Z] ==1855==Register values: >>> >>> [2024-08-13T17:26:23.254Z] rax = 0x000003ffffffffff rbx = >>> 0x0000000000000000 rcx = 0x0000000003390000 rdx = 0x0000600003390000 >>> >>> [2024-08-13T17:26:23.254Z] rdi = 0x0000000105730000 rsi = >>> 0x00000001021e9840 rbp = 0x00007e8000186920 rsp = 0x00007e8000186900 >>> >>> [2024-08-13T17:26:23.254Z] r8 = 0x0000000000000000 r9 = >>> 0x0000000000000000 r10 = 0x0000000000000000 r11 = 0x0000000000000000 >>> >>> [2024-08-13T17:26:23.254Z] r12 = 0x00007b3400006830 r13 = >>> 0x00007e8000186ab8 r14 = 0x0000000000000000 r15 = 0x0000000000000000 >>> >>> [2024-08-13T17:26:23.254Z] ThreadSanitizer can not provide additional >>> info. >>> >>> [2024-08-13T17:26:23.254Z] SUMMARY: ThreadSanitizer: SEGV map.h:630 in >>> google::protobuf::internal::UntypedMapBase::TableEntryIsNonEmptyList(unsigned >>> int) const >>> >>> [2024-08-13T17:26:23.254Z] ==1855==ABORTING >>> What could be causing this issue? we never had this issue with our >>> previous protobuf version compiled the same way 3.21.12 using the same >>> clang compiler version as universal binary. Any advise? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/protobuf/bcc31496-50bd-419d-bdd7-cac894796cfbn%40googlegroups.com >>> <https://groups.google.com/d/msgid/protobuf/bcc31496-50bd-419d-bdd7-cac894796cfbn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/protobuf/79e9fa14-872e-4efe-adbb-4264ee1b429bn%40googlegroups.com > <https://groups.google.com/d/msgid/protobuf/79e9fa14-872e-4efe-adbb-4264ee1b429bn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/protobuf/CAM9aRsJ6PYJ0qH%2BdYiUA0DS2sjVtS4nij7yP17a4V_JDOFHKwA%40mail.gmail.com.
