On Tue, 20 Jul 2021 at 19:48, Evgenii Stepanov <euge...@google.com> wrote:
>
>
> On Tue, Jul 20, 2021 at 1:17 AM Dmitry Vyukov <dvyu...@google.com> wrote:
>>
>> The error reported by buildbot is:
>>
>> =================================================================
>> ==26030==ERROR: AddressSanitizer: SEGV on unknown address 0xeaf50000
>> (pc 0xeadd3514 bp 0xc83e6c68 sp 0xc83e6c00 T-1)
>> ==26030==The signal is caused by a READ memory access.
>> #0 0xeadd3514 in GetThreadLocked
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_thread_registry.h:103:41
>> #1 0xeadd3514 in GetThreadContextByTidLocked
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:72:28
>> #2 0xeadd3514 in __asan::GetCurrentThread()
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:433:33
>> #3 0xead771b4 in GetFakeStack
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:188:19
>> #4 0xead771b4 in GetFakeStackFastAlways
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:204:10
>> #5 0xead771b4 in OnMallocAlways
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:220:19
>> #6 0xead771b4 in __asan_stack_malloc_always_0
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:256:1
>> #7 0xad341ab0 in SignalHandler(int, siginfo*, void*)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/test/asan/TestCases/Linux/uar_signals.cpp:19
>> #8 0xeab4174c (/apex/com.android.runtime/lib/bionic/libc.so+0x3374c)
>> #9 0xead51da8 in __internal_syscall
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_syscall_linux_arm.inc:109:3
>> #10 0xead51da8 in __sanitizer::internal_mmap(void*, unsigned long,
>> int, int, int, unsigned long long)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp:176:10
>> #11 0xead53d18 in __sanitizer::MmapNamed(void*, unsigned long,
>> int, int, char const*)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix.cpp:391:14
>> #12 0xead5b89c in MmapFixed
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:304:7
>> #13 0xead5b89c in MmapFixedNoReserve
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:318:10
>> #14 0xead5b89c in __sanitizer::MmapFixedSuperNoReserve(unsigned
>> long, unsigned long, char const*)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:328:12
>> #15 0xead59bb8 in __sanitizer::ReserveShadowMemoryRange(unsigned
>> long, unsigned long, char const*, bool)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_libcdep.cpp:151:25
>> #16 0xead76ca0 in PoisonAll
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:80:3
>> #17 0xead76ca0 in __asan::FakeStack::Destroy(int)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:66:3
>> #18 0xeadd2e68 in DeleteFakeStack
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.h:98:8
>> #19 0xeadd2e68 in __asan::AsanThread::Destroy()
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:116:5
>> #20 0xeab8ea7e in pthread_key_clean_all() pthread_key.cpp
>> #21 0xeab8e6ca in pthread_exit
>> (/apex/com.android.runtime/lib/bionic/libc.so+0x806ca)
>> #22 0xeab8e620 in __pthread_start(void*) pthread_create.cpp
>> #23 0xeab47902 in __start_thread clone.cpp
>>
>> The change dfd9808b6cea59ff075498ee7e6e57f2b5b3a798 did this:
>>
>> // Should be guarded by ThreadRegistryLock.
>> ThreadContextBase *GetThreadLocked(u32 tid) {
>> - DCHECK_LT(tid, n_contexts_);
>> - return threads_[tid];
>> + return threads_.empty() ? nullptr : threads_[tid];
>> }
>>
>> It started CHECK failing on Android with accessing threads_[0] when it
>> has 0 size.
>> Then we added this "threads_.empty() ? nullptr", but I think it only
>> masked the real problem.
>> The real problem is who/why is accessing threads_[0] before we
>> initialized the main thread. And I think this new crash answers this
>> -- a second thread calls GetThreadLocked(kMainThread) before we
>> initialized asan runtime and created the main thread.
>> I think we need to remove this "threads_.empty() ? nullptr" part and
>> instead ensure that runtime is initialized by the time of the call.
>
>
> Sounds right. I'm not sure how there can be a secondary thread while ASan
> runtime is not initialized, we'll need to look into that. Can't promise a
> quick resolution though.
Looking at the stack more closely, this happens in AsanThread::Destroy
after we destroyed the thread in ThreadRegistry:
void AsanThread::Destroy() {
...
(asanThreadRegistry().FinishThread(tid) == ThreadStatusRunning);
...
DeleteFakeStack(tid);
Maybe that's the problem.
>> On Tue, 20 Jul 2021 at 09:23, <llvm.buildmas...@lab.llvm.org> wrote:
>> >
>> > The Buildbot has detected a failed build on builder
>> > sanitizer-x86_64-linux-android while building compiler-rt.
>> >
>> > Full details are available at:
>> > https://lab.llvm.org/buildbot#builders/77/builds/7937
>> >
>> > Worker for this Build: sanitizer-buildbot6
>> > Blamelist:
>> > Dmitry Vyukov <dvyu...@google.com>,
>> > Johannes Doerfert <johan...@jdoerfert.de>
>> >
>> > BUILD FAILED: failed 'python
>> > ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py'
>> >
>> > Step 2 (annotate) failure: 'python
>> > ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py'
>> > python
>> > ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py
>> > in dir
>> > /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build
>> > (timeout 1200 secs)
>> > watching logfiles {}
>> > argv: [b'python',
>> > b'../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py']
>> > environment:
>> > BUILDBOT_BLAMELIST=[]
>> > BUILDBOT_BRANCH=main
>> > BUILDBOT_BUILDERNAME=sanitizer-x86_64-linux-android
>> > BUILDBOT_BUILDNUMBER=7937
>> > BUILDBOT_CLOBBER=
>> > BUILDBOT_GOT_REVISION=b899cd8edcb824c4e4f999ef254209060d1ab646
>> > BUILDBOT_REVISION=adb55d7c326559e99f1eeb3234cdfaaf3c9010de
>> >
>> > BUILDBOT_SCHEDULER=main:clang,compiler-rt,libcxx,libcxxabi,libunwind,lld,llvm
>> > HOME=/var/lib/buildbot
>> > INVOCATION_ID=c05ecc968f624671b2e2ffd0f558eeaa
>> > JOURNAL_STREAM=9:2763
>> > LANG=en_US.UTF-8
>> > LOGNAME=buildbot
>> > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
>> >
>> > PWD=/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build
>> > SHELL=/bin/bash
>> > TERM=dumb
>> > USER=buildbot
>> > using PTY: False
>> > builder name: sanitizer-x86_64-linux-android
>> > sanitizer-x86_64-linux-android runs: bash
>> > ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_android.sh
>> > + set -e
>> > + set -u
>> > +++ dirname
>> > ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_android.sh
>> > ++ cd ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers
>> > ++ pwd
>> > +
>> > HERE=/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers
>> > + .
>> > /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_functions.sh
>> > ++ echo @@@BUILD_STEP Info@@@
>> > Step 21 (run lit tests [arm/crosshatch-userdebug/RP1A.201105.002])
>> > failure: run lit tests [arm/crosshatch-userdebug/RP1A.201105.002] (failure)
>> > ...
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/asan_preload_test-2.cpp (44 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/asan_preload_test-3.cpp (45 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/coverage-missing.cpp (46 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android :: TestCases/Linux/cuda_test.cpp
>> > (47 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/dlopen-mixed-c-cxx.c (48 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/auto_memory_profile_test.cpp (49 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android :: TestCases/Linux/bzero.cpp (50
>> > of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/calloc-preload.c (51 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/function-sections-are-bad.cpp (52 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/init-order-dlopen.cpp (53 of 1460)
>> > XFAIL: AddressSanitizer-arm-android ::
>> > TestCases/Linux/globals-gc-sections-lld.cpp (54 of 1460)
>> > XFAIL: AddressSanitizer-arm-android ::
>> > TestCases/Linux/asan_dlopen_test.cpp (55 of 1460)
>> > PASS: AddressSanitizer-arm-android ::
>> > TestCases/Linux/init_fini_sections.cpp (56 of 1460)
>> > PASS: AddressSanitizer-arm-android ::
>> > TestCases/Linux/asan_default_suppressions.cpp (57 of 1460)
>> > PASS: AddressSanitizer-arm-android ::
>> > TestCases/Linux/global-overflow-bfd.cpp (58 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/interface_symbols_linux.cpp (59 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/kernel-area.cpp (60 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android :: TestCases/Linux/leak.cpp (61
>> > of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/leak_check_segv.cpp (62 of 1460)
>> > PASS: AddressSanitizer-arm-android ::
>> > TestCases/Linux/global-overflow-lld.cpp (63 of 1460)
>> > XFAIL: AddressSanitizer-arm-android ::
>> > TestCases/Linux/asan_rt_confict_test-1.cpp (64 of 1460)
>> > XFAIL: AddressSanitizer-arm-android ::
>> > TestCases/Linux/asan_rt_confict_test-2.cpp (65 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/malloc-in-qsort.cpp (66 of 1460)
>> > PASS: AddressSanitizer-arm-android :: TestCases/Linux/long-object-path.cpp
>> > (67 of 1460)
>> > PASS: AddressSanitizer-arm-android :: TestCases/Linux/longjmp_chk.c (68 of
>> > 1460)
>> > PASS: AddressSanitizer-arm-android :: TestCases/Linux/local_alias.cpp (69
>> > of 1460)
>> > PASS: AddressSanitizer-arm-android ::
>> > TestCases/Linux/aligned_delete_test.cpp (70 of 1460)
>> > XFAIL: AddressSanitizer-arm-android ::
>> > TestCases/Linux/interception_readdir_r_test.cpp (71 of 1460)
>> > PASS: AddressSanitizer-arm-android ::
>> > TestCases/Linux/initialization-bug-any-order.cpp (72 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/nohugepage_test.cpp (73 of 1460)
>> > PASS: AddressSanitizer-arm-android :: TestCases/Linux/mincore.cpp (74 of
>> > 1460)
>> > XFAIL: AddressSanitizer-arm-android ::
>> > TestCases/Linux/activation-options.cpp (75 of 1460)
>> > PASS: AddressSanitizer-arm-android :: TestCases/Linux/clone_test.cpp (76
>> > of 1460)
>> > PASS: AddressSanitizer-arm-android ::
>> > TestCases/Linux/new_delete_mismatch_global.cpp (77 of 1460)
>> > PASS: AddressSanitizer-arm-android :: TestCases/Linux/memmem_test.cpp (78
>> > of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/overflow-in-qsort.cpp (79 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/preinstalled_signal.cpp (80 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/print_memory_profile_test.cpp (81 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/printf-fortify-1.c (82 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/printf-fortify-2.c (83 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/printf-fortify-3.c (84 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/printf-fortify-4.c (85 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/printf-fortify-5.c (86 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/pthread_create_from_constructor.cpp (87 of 1460)
>> > PASS: AddressSanitizer-arm-android :: TestCases/Linux/clang_gcc_abi.cpp
>> > (88 of 1460)
>> > PASS: AddressSanitizer-arm-android ::
>> > TestCases/Linux/new_delete_mismatch_stack.cpp (89 of 1460)
>> > XFAIL: AddressSanitizer-arm-android :: TestCases/Linux/ptrace.cpp (90 of
>> > 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/read_binary_name_regtest.c (91 of 1460)
>> > UNSUPPORTED: AddressSanitizer-arm-android ::
>> > TestCases/Linux/recoverable-lsan.cpp (92 of 1460)
>> >
>> > Sincerely,
>> > LLVM Buildbot
>> >
--
You received this message because you are subscribed to the Google Groups
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to address-sanitizer+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/address-sanitizer/CACT4Y%2BaSY%3DtgWfufNp%3DLxyhiifqZf_5ZozZq5J0gN2P7uDQnxg%40mail.gmail.com.
