Re: [PATCH] mm: Revert x86_64 and arm64 ELF_ET_DYN_BASE base

['Dmitry Vyukov' via address-sanitizer]

On Sat, 17 Feb 2024 at 07:50, Kees Cook  wrote:
>
> *extreme thread[1] necromancy*
>
> On Mon, Aug 07, 2017 at 01:15:42PM -0700, Kees Cook wrote:
> > Moving the x86_64 and arm64 PIE base from 0x4000 to 0x0001
> > broke AddressSanitizer. This is a partial revert of:
> >
> >   commit eab09532d400 ("binfmt_elf: use ELF_ET_DYN_BASE only for PIE")
> >   commit 02445990a96e ("arm64: move ELF_ET_DYN_BASE to 4GB / 4MB")
> >
> > The AddressSanitizer tool has hard-coded expectations about where
> > executable mappings are loaded. The motivation for changing the PIE
> > base in the above commits was to avoid the Stack-Clash CVEs that
> > allowed executable mappings to get too close to heap and stack. This
> > was mainly a problem on 32-bit, but the 64-bit bases were moved too,
> > in an effort to proactively protect those systems (proofs of concept
> > do exist that show 64-bit collisions, but other recent changes to fix
> > stack accounting and setuid behaviors will minimize the impact).
>
> I happened to be looking at this again today, and wondered where things
> stood. It seems like ASan's mappings are documented here:
> https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#64-bit
>
> This implies that it would be safe to move the ELF_ET_DYN_BASE from
> 0x4000 down to 0x2000, since the shadow map ends at
> 0x10007fff7fff. (Well, anything above there would work, I was just
> picking a "round" number above it. We could just as well use
> 0x10008000, I think.)
>
> Is this correct? I'd like to open up some more room between mmap and
> stack...

Note that there is also TSAN and MSAN with their own mappings.
These are also different per-arch, e.g. TSAN/Linux/x86_64:
https://github.com/llvm/llvm-project/blob/d2a26a7bd5fc7cc5752337b7f4f999642feb37dc/compiler-rt/lib/tsan/rtl/tsan_platform.h#L48-L58

Search "linux/" in that file for other arches, e.g.:
https://github.com/llvm/llvm-project/blob/d2a26a7bd5fc7cc5752337b7f4f999642feb37dc/compiler-rt/lib/tsan/rtl/tsan_platform.h#L156-L165
https://github.com/llvm/llvm-project/blob/d2a26a7bd5fc7cc5752337b7f4f999642feb37dc/compiler-rt/lib/tsan/rtl/tsan_platform.h#L187-L196
https://github.com/llvm/llvm-project/blob/d2a26a7bd5fc7cc5752337b7f4f999642feb37dc/compiler-rt/lib/tsan/rtl/tsan_platform.h#L218-L227
https://github.com/llvm/llvm-project/blob/d2a26a7bd5fc7cc5752337b7f4f999642feb37dc/compiler-rt/lib/tsan/rtl/tsan_platform.h#L252-L263

And MSAN mappings:
https://github.com/llvm/llvm-project/blob/d2a26a7bd5fc7cc5752337b7f4f999642feb37dc/compiler-rt/lib/msan/msan.h#L44-L61


> Thanks!
>
> -Kees
>
> [1] https://lore.kernel.org/lkml/20170807201542.GA21271@beast/
>
> >
> > The new 32-bit PIE base is fine for ASan (since it matches the ET_EXEC
> > base), so only the 64-bit PIE base needs to be reverted to let x86 and
> > arm64 ASan binaries run again. Future changes to the 64-bit PIE base on
> > these architectures can be made optional once a more dynamic method for
> > dealing with AddressSanitizer is found. (e.g. always loading PIE into
> > the mmap region for marked binaries.)
> >
> > Reported-by: Kostya Serebryany 
> > Cc: sta...@vger.kernel.org
> > Signed-off-by: Kees Cook 
> > ---
> >  arch/arm64/include/asm/elf.h | 4 ++--
> >  arch/x86/include/asm/elf.h   | 4 ++--
> >  2 files changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
> > index acae781f7359..3288c2b36731 100644
> > --- a/arch/arm64/include/asm/elf.h
> > +++ b/arch/arm64/include/asm/elf.h
> > @@ -114,10 +114,10 @@
> >
> >  /*
> >   * This is the base location for PIE (ET_DYN with INTERP) loads. On
> > - * 64-bit, this is raised to 4GB to leave the entire 32-bit address
> > + * 64-bit, this is above 4GB to leave the entire 32-bit address
> >   * space open for things that want to use the area for 32-bit pointers.
> >   */
> > -#define ELF_ET_DYN_BASE  0x1UL
> > +#define ELF_ET_DYN_BASE  (2 * TASK_SIZE_64 / 3)
> >
> >  #ifndef __ASSEMBLY__
> >
> > diff --git a/arch/x86/include/asm/elf.h b/arch/x86/include/asm/elf.h
> > index 1c18d83d3f09..9aeb91935ce0 100644
> > --- a/arch/x86/include/asm/elf.h
> > +++ b/arch/x86/include/asm/elf.h
> > @@ -247,11 +247,11 @@ extern int force_personality32;
> >
> >  /*
> >   * This is the base location for PIE (ET_DYN with INTERP) loads. On
> > - * 64-bit, this is raised to 4GB to leave the entire 32-bit address
> > + * 64-bit, this is above 4GB to leave the entire 32-bit address
> >   * space open for things that want to use the area for 32-bit pointers.
> >   */
> >  #define ELF_ET_DYN_BASE  (mmap_is_ia32() ? 0x00040UL : \
> > -   0x1UL)
> > +   (TASK_SIZE / 3 * 2))
> >
> >  /* This yields a mask that user programs can use to figure out what
> > instruction set this CPU supports.  This could be done in user space,
> > --
> > 

Re: kpatch and kasan

['Dmitry Vyukov' via address-sanitizer]

On Wed, 18 Jan 2023 at 17:10, Joe Lawrence  wrote:
>
> On 1/18/23 10:21, Dmitry Vyukov wrote:
> > On Wed, 18 Jan 2023 at 14:45, Joe Lawrence  wrote:
> >>
> >> On 1/18/23 02:21, Dmitry Vyukov wrote:
> >>> On Tue, 17 Jan 2023 at 17:50, Kostya Serebryany  wrote:
> 
>  +kernel-dynamic-tools
> 
>  On Tue, Jan 17, 2023 at 6:32 AM Joe Lawrence  
>  wrote:
> >
> > Hi Kostya,
> >
> > I work on the kernel livepatching Kpatch project [1] and was hoping to
> > learn some info about compiler-generated (k)asan ELF sections.  If you
> > can point me to any references or folks who might entertain questions,
> > we would be much appreciated.
> >
> > The tl/dr; is that we would like to build kasan-enabled debug kernels
> > and then kpatches for them to help verify CVE mitigations.
> >
> > If you are unfamiliar with kpatch, it accepts an input .patch file,
> > builds a reference and patched kernel (with -ffunction-sections and
> > -fdata-sections) ... then performs a binary comparison between
> > reference/patched ELF sections.  New or changed ELF sections are
> > extracted into a new object file.  Boilerplate code is then added to
> > create a livepatch kernel module from that.
> >
> > The devil is in details, of course, so our kpatch-build tool needs to
> > know whether it should omit, copy, or re-generate an ELF section
> > depending on its purpose.  The kernel is rife with interesting sections
> > like para-virt instructions, jump labels, static call sites, etc.
> >
> > So, before trying to reverse engineer sections like .data..LASANLOC1 and
> > data..LASAN0 from the gcc source code, I was wondering if these were
> > documented somewhere?
> >
> >
> > Regards,
> >
> > [1] https://github.com/dynup/kpatch
> > --
> > Joe
> >>>
> >>> +kasan-dev
> >>>
> >>> Hi Joe,
> >>>
> >>> But why not just build a new KASAN kernel and re-test? This looks so
> >>> much simpler.
> >>>
> >>
> >> Hi Dmitry,
> >>
> >> Well yes, testing an ordinary (fixed) kernel build is much easier, however:
> >>
> >> 1 - Sometimes kpatches deviate from their kernel counterparts.  Examples
> >> include ABI changes, fixups in initialization code, etc.
> >
> > This does not prevent testing in a normal way, right? In fact I would
> > send the patch to the normal CI as the first thing.
> >
>
> Exactly.  At Red Hat, we typically wait for a corresponding kernel fix
> to pass tests before starting on our kpatch conversion (emergency CVEs
> aside) ... that way we're usually confident with the overall changes
> before we even start our work.
>
> In cases where the kernel fixes are verified via reproducer and KASAN
> enabled config, as long as our version is mostly 1:1 we can still be
> confident.  Giving our QA team a similar obvious verification with KASAN
> enabled kpatch would be bonus.

I meant the source patch used to create the kpatch, not some other patch.
Kpatch is also based on some normal source code patch, right? If so,
that exact patch used to create kpatch can be testing as a normal
patch, right?

Back to your actual question. I think sections like .data..LASANLOC1
and data..LASAN0 should be treated just as normal .data/.rodata
sections. git grep "ASANLOC" in llvm does not give me anything, but I
would assume these contain string descriptions used in KASAN reports.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/address-sanitizer/CACT4Y%2BZA_Up4Hn_qcTczuUh0RHdm0seUPGKxf-Eh09n34PcoXA%40mail.gmail.com.

Re: MTE enabled hardware

['Dmitry Vyukov' via address-sanitizer]

On Sat, 1 Oct 2022 at 14:25, 'Koschel, J. (Jakob)' via
address-sanitizer  wrote:
>
> Hey,
>
> it was worth a shot, after Dmitry suggested to try here ;)
> Regardless thanks for your reply, I guess I'll have to wait a bit more.

Just in case: you considered using qemu, right?

> > On 30. Sep 2022, at 01:26, Konstantin Serebryany 
> >  wrote:
> >
> > Hi Jakob!
> >
> > Thanks for your interest.
> > Unfortunately, we are not aware of any publicly available device with 
> > usable MTE.
> > Please pester your hardware suppliers directly, or wait :)
> >
> > --kcc
> >
> >
> >
> > On Thu, Sep 29, 2022 at 3:08 PM 'Jakob K.' via address-sanitizer 
> >  wrote:
> > Hi all,
> >
> > I'm currently looking into MTE and therefore inevitably also at the great 
> > work on MemTagSanitizer & KASAN_HW_TAGS.
> >
> > Since you guys are working on this: do you happen to know if there is any 
> > hardware out on the market you are using for testing MTE?
> >
> > I've been looking at the Samsung S22 with the exynos2200 which does support 
> > MTE. Unfortunately it looks like the bootloader (in connection with "Exynos 
> > Secure Monitor") is responsible to setup the DRAM locations to store the 
> > tags & this doesn't seem to be enabled. This results in MTE working *until* 
> > the tagged memory is evicted from the cache. Once the memory is brought 
> > back into the cache it forgot that it was tagged :/
> >
> > If you know of any hardware out there that you might have tested with MTE, 
> > I'd appreciated some pointers!
> >
> > Thank you!
> > Jakob
> >
> > --
> > You received this message because you are subscribed to the Google Groups 
> > "address-sanitizer" group.
> > To unsubscribe from this group and stop receiving emails from it, send an 
> > email to address-sanitizer+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit 
> > https://groups.google.com/d/msgid/address-sanitizer/01cba890-7360-4dc5-a45b-7265d14c4195n%40googlegroups.com.
> >
> > --
> > You received this message because you are subscribed to a topic in the 
> > Google Groups "address-sanitizer" group.
> > To unsubscribe from this topic, visit 
> > https://groups.google.com/d/topic/address-sanitizer/o-qhiTBFlBc/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to 
> > address-sanitizer+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit 
> > https://groups.google.com/d/msgid/address-sanitizer/CAGQ9bdxJM_eXgFQm_Wq%2BNyF-_p0XXVQChOGN08J6%3D%3Dok_8SPGw%40mail.gmail.com.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "address-sanitizer" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to address-sanitizer+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/address-sanitizer/62531A20-0374-4463-AFE8-0D3CB380A388%40vu.nl.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/address-sanitizer/CACT4Y%2BZGf%3Dyp61Sw_W6MPGe-KCd4Hy%2BOUnAUR0QWyQBzL5OSSg%40mail.gmail.com.

Re: Thread Sanitizer could not print proper stack trace.

['Dmitry Vyukov' via address-sanitizer]

On Wed, 22 Sept 2021 at 10:14, Dmitry Vyukov  wrote:
>
> +thread-sanitizer mailing list
>
> On Wed, 22 Sept 2021 at 10:10, Pankaj Kumar Thapa  
> wrote:
> >
> > Hello Dmitry,
> >
> > My name is Pankaj. I am from Bangalore, India.
> >
> > I am trying to clean up the code base using thread sanitizer. When I run 
> > TSAN, I get the below error. I am using the clang-9 compiler to compile the 
> > code in a dockerized environment.
> > Could you please give a hint or point out exactly why this issue is? With 
> > this log it is impossible for me to find exactly where the data race is. I 
> > am new to this and it would be great if you can help me out.
> >
> > Thank you in advance.
> >
> > Error output:
> >
> > ==199==WARNING: invalid path to external symbolizer!
> > ==199==WARNING: Failed to use and restart external symbolizer!

Hi Pankaj,

The error suggests there is something wrong with the symbolizer.
You need to have a llvm-symbolizer binary in the PATH for symbolization to work.

I see there are also some flag that may help:

COMMON_FLAG(
bool, symbolize, true,
"If set, use the online symbolizer from common sanitizer runtime to turn "
"virtual addresses to file/line locations.")
COMMON_FLAG(
const char *, external_symbolizer_path, nullptr,
"Path to external symbolizer. If empty, the tool will search $PATH for "
"the symbolizer.")
COMMON_FLAG(
bool, allow_addr2line, false,
"If set, allows online symbolizer to run addr2line binary to symbolize "
"stack traces (addr2line will only be used if llvm-symbolizer binary is "
"unavailable.")





> > ==
> > WARNING: ThreadSanitizer: data race (pid=199)
> >   Write of size 8 at 0x7b1c7e08 by main thread:
> > #0   (skvbc_replica+0xdcf3ce)
> > #1   (skvbc_replica+0xe84d23)
> > #2   (skvbc_replica+0xe84cd3)
> > #3   (skvbc_replica+0xe8397e)
> > #4   (skvbc_replica+0xe83fc8)
> > #5   (skvbc_replica+0xe23408)
> > #6   (skvbc_replica+0xf549a4)
> > #7   (skvbc_replica+0xf548f1)
> > #8   (skvbc_replica+0xf5485a)
> > #9   (skvbc_replica+0xf547d7)
> > #10   (skvbc_replica+0x16a268c)
> > #11   (skvbc_replica+0x1699e23)
> > #12   (skvbc_replica+0xf349d7)
> > #13   (skvbc_replica+0xe2b6c1)
> > #14   (skvbc_replica+0xe2b1b4)
> > #15   (skvbc_replica+0xe2ab8c)
> > #16   (skvbc_replica+0xe2a6ff)
> > #17   (skvbc_replica+0xe2a45d)
> > #18   (skvbc_replica+0xe2a2aa)
> > #19   (skvbc_replica+0xe29f25)
> > #20   (skvbc_replica+0xdd52e3)
> > #21   (skvbc_replica+0xdd1e7a)
> > #22   (skvbc_replica+0xdd2d5a)
> > #23   (libc.so.6+0x21bf6)
> >
> >   Previous atomic read of size 4 at 0x7b1c7e08 by thread T73:
> > #0   (skvbc_replica+0xd8a3bc)
> > #1   (skvbc_replica+0xddb7e8)
> > #2   (skvbc_replica+0xddb6ff)
> > #3   (skvbc_replica+0xddb603)
> > #4   (skvbc_replica+0xddb4d2)
> > #5   (skvbc_replica+0xddb458)
> > #6   (skvbc_replica+0xddb35e)
> > #7   (skvbc_replica+0x15b7106)
> > #8   (skvbc_replica+0x169ce95)
> > #9   (skvbc_replica+0x169cc3b)
> > #10   (skvbc_replica+0x169c9da)
> > #11   (skvbc_replica+0xf6bcb6)
> > #12   (skvbc_replica+0xfbbd84)
> > #13   (skvbc_replica+0xfbbca0)
> > #14   (skvbc_replica+0xf0)
> > #15   (skvbc_replica+0xfbbb58)
> > #16   (skvbc_replica+0xfbbaf8)
> > #17   (skvbc_replica+0xfbb8ff)
> > #18   (libstdc++.so.6+0xbd6de)
> >
> >   Thread T73 (tid=739, running) created by main thread at:
> > #0   (skvbc_replica+0xd4019b)
> > #1   (libstdc++.so.6+0xbd994)
> > #2   (skvbc_replica+0xfbadc1)
> > #3   (skvbc_replica+0x169c561)
> > #4   (skvbc_replica+0x1698b04)
> > #5   (skvbc_replica+0x1690331)
> > #6   (skvbc_replica+0x16909cd)
> > #7   (skvbc_replica+0xfc175f)
> > #8   (skvbc_replica+0xf32248)
> > #9   (skvbc_replica+0xe2b6c1)
> > #10   (skvbc_replica+0xe2b1b4)
> > #11   (skvbc_replica+0xe2ab8c)
> > #12   (skvbc_replica+0xe2a6ff)
> > #13   (skvbc_replica+0xe2a45d)
> > #14   (skvbc_replica+0xe2a2aa)
> > #15   (skvbc_replica+0xe29f25)
> > #16   (skvbc_replica+0xdd52e3)
> > #17   (skvbc_replica+0xdd1e7a)
> > #18   (skvbc_replica+0xdd2d5a)
> > #19   (libc.so.6+0x21bf6)
> >
> > SUMMARY: ThreadSanitizer: data race 
> > (/concord-bft/build/tests/simpleKVBC/TesterReplica/skvbc_replica+0xdcf3ce)
> > ==
> > ==
> > WARNING: ThreadSanitizer: data race (pid=199)
> >   Write of size 8 at 0x7b541eb0 by main thread:
> > #0   (skvbc_replica+0xf549b8)
> > #1   (skvbc_replica+0xf548f1)
> > #2   (skvbc_replica+0xf5485a)
> > #3   (skvbc_replica+0xf547d7)
> > #4   (skvbc_replica+0x16a268c)
> > #5   (skvbc_replica+0x1699e23)
> > #6   (skvbc_replica+0xf349d7)
> > #7   (skvbc_replica+0xe2b6c1)
> > #8   (skvbc_replica+0xe2b1b4)
> > #9   (skvbc_replica+0xe2ab8c)
> > #10   (skvbc_replica+0xe2a6ff)
> 

Re: Asan initialization broken on Android?

['Dmitry Vyukov' via address-sanitizer]

On Tue, 20 Jul 2021 at 19:48, Evgenii Stepanov  wrote:
>
>
> On Tue, Jul 20, 2021 at 1:17 AM Dmitry Vyukov  wrote:
>>
>> The error reported by buildbot is:
>>
>> =
>> ==26030==ERROR: AddressSanitizer: SEGV on unknown address 0xeaf5
>> (pc 0xeadd3514 bp 0xc83e6c68 sp 0xc83e6c00 T-1)
>> ==26030==The signal is caused by a READ memory access.
>> #0 0xeadd3514 in GetThreadLocked
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_thread_registry.h:103:41
>> #1 0xeadd3514 in GetThreadContextByTidLocked
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:72:28
>> #2 0xeadd3514 in __asan::GetCurrentThread()
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:433:33
>> #3 0xead771b4 in GetFakeStack
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:188:19
>> #4 0xead771b4 in GetFakeStackFastAlways
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:204:10
>> #5 0xead771b4 in OnMallocAlways
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:220:19
>> #6 0xead771b4 in __asan_stack_malloc_always_0
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:256:1
>> #7 0xad341ab0 in SignalHandler(int, siginfo*, void*)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/test/asan/TestCases/Linux/uar_signals.cpp:19
>> #8 0xeab4174c  (/apex/com.android.runtime/lib/bionic/libc.so+0x3374c)
>> #9 0xead51da8 in __internal_syscall
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_syscall_linux_arm.inc:109:3
>> #10 0xead51da8 in __sanitizer::internal_mmap(void*, unsigned long,
>> int, int, int, unsigned long long)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp:176:10
>> #11 0xead53d18 in __sanitizer::MmapNamed(void*, unsigned long,
>> int, int, char const*)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix.cpp:391:14
>> #12 0xead5b89c in MmapFixed
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:304:7
>> #13 0xead5b89c in MmapFixedNoReserve
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:318:10
>> #14 0xead5b89c in __sanitizer::MmapFixedSuperNoReserve(unsigned
>> long, unsigned long, char const*)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:328:12
>> #15 0xead59bb8 in __sanitizer::ReserveShadowMemoryRange(unsigned
>> long, unsigned long, char const*, bool)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_libcdep.cpp:151:25
>> #16 0xead76ca0 in PoisonAll
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:80:3
>> #17 0xead76ca0 in __asan::FakeStack::Destroy(int)
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:66:3
>> #18 0xeadd2e68 in DeleteFakeStack
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.h:98:8
>> #19 0xeadd2e68 in __asan::AsanThread::Destroy()
>> /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:116:5
>> #20 0xeab8ea7e in pthread_key_clean_all() pthread_key.cpp
>> #21 0xeab8e6ca in pthread_exit
>> (/apex/com.android.runtime/lib/bionic/libc.so+0x806ca)
>> #22 0xeab8e620 in __pthread_start(void*) pthread_create.cpp
>> #23 0xeab47902 in __start_thread clone.cpp
>>
>> The change dfd9808b6cea59ff075498ee7e6e57f2b5b3a798 did this:
>>
>>// Should be guarded by ThreadRegistryLock.
>>ThreadContextBase *GetThreadLocked(u32 tid) {
>> -DCHECK_LT(tid, n_contexts_);
>> -return threads_[tid];
>> +return threads_.empty() ? nullptr : threads_[tid];
>>}
>>
>> It started CHECK 

Asan initialization broken on Android?

['Dmitry Vyukov' via address-sanitizer]

The error reported by buildbot is:

=
==26030==ERROR: AddressSanitizer: SEGV on unknown address 0xeaf5
(pc 0xeadd3514 bp 0xc83e6c68 sp 0xc83e6c00 T-1)
==26030==The signal is caused by a READ memory access.
#0 0xeadd3514 in GetThreadLocked
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_thread_registry.h:103:41
#1 0xeadd3514 in GetThreadContextByTidLocked
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:72:28
#2 0xeadd3514 in __asan::GetCurrentThread()
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:433:33
#3 0xead771b4 in GetFakeStack
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:188:19
#4 0xead771b4 in GetFakeStackFastAlways
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:204:10
#5 0xead771b4 in OnMallocAlways
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:220:19
#6 0xead771b4 in __asan_stack_malloc_always_0
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:256:1
#7 0xad341ab0 in SignalHandler(int, siginfo*, void*)
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/test/asan/TestCases/Linux/uar_signals.cpp:19
#8 0xeab4174c  (/apex/com.android.runtime/lib/bionic/libc.so+0x3374c)
#9 0xead51da8 in __internal_syscall
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_syscall_linux_arm.inc:109:3
#10 0xead51da8 in __sanitizer::internal_mmap(void*, unsigned long,
int, int, int, unsigned long long)
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp:176:10
#11 0xead53d18 in __sanitizer::MmapNamed(void*, unsigned long,
int, int, char const*)
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix.cpp:391:14
#12 0xead5b89c in MmapFixed
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:304:7
#13 0xead5b89c in MmapFixedNoReserve
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:318:10
#14 0xead5b89c in __sanitizer::MmapFixedSuperNoReserve(unsigned
long, unsigned long, char const*)
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp:328:12
#15 0xead59bb8 in __sanitizer::ReserveShadowMemoryRange(unsigned
long, unsigned long, char const*, bool)
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_libcdep.cpp:151:25
#16 0xead76ca0 in PoisonAll
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:80:3
#17 0xead76ca0 in __asan::FakeStack::Destroy(int)
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_fake_stack.cpp:66:3
#18 0xeadd2e68 in DeleteFakeStack
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.h:98:8
#19 0xeadd2e68 in __asan::AsanThread::Destroy()
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:116:5
#20 0xeab8ea7e in pthread_key_clean_all() pthread_key.cpp
#21 0xeab8e6ca in pthread_exit
(/apex/com.android.runtime/lib/bionic/libc.so+0x806ca)
#22 0xeab8e620 in __pthread_start(void*) pthread_create.cpp
#23 0xeab47902 in __start_thread clone.cpp

The change dfd9808b6cea59ff075498ee7e6e57f2b5b3a798 did this:

   // Should be guarded by ThreadRegistryLock.
   ThreadContextBase *GetThreadLocked(u32 tid) {
-DCHECK_LT(tid, n_contexts_);
-return threads_[tid];
+return threads_.empty() ? nullptr : threads_[tid];
   }

It started CHECK failing on Android with accessing threads_[0] when it
has 0 size.
Then we added this "threads_.empty() ? nullptr", but I think it only
masked the real problem.
The real problem is who/why is accessing threads_[0] before we
initialized the main thread. And I think this new crash answers this
-- a second thread 

Re: Is there any reason to implement our malloc/free with exactly the same signature as library ones?

['Dmitry Vyukov' via address-sanitizer]

On Tue, Apr 13, 2021 at 11:55 AM Wei Chen  wrote:
>
> Hi all,
>
> I'm recently learning Address Sanitizer code and learned that the runtime 
> library is designed to replace the glibc malloc/free with our customized ones 
> so that we can insert red zone before and after the allocated region.
>
> From the code, I've learned that such a replacement is implemented by 
> changing the runtime library load priority. By writing our malloc/free with 
> exactly the same function signature as the library ones, and letting our 
> runtime library being loaded before glibc, our malloc/free function will be 
> invoked by the user program.
>
> However, we are not able to invoke printf library functions anymore, cause 
> printf depends on malloc, therefore, each time when printf is invoked when 
> implementing asan runtime library, it will call our malloc instead of library 
> malloc.
>
> Why not just implement our malloc and free to another name, say asan_malloc 
> and asan_free? In that case, we can replace the malloc invocation with 
> asan_malloc invocation during compilation, and no need to write a totally new 
> printf. Is there any reason to have the same name?

Hi Wei,

One reason to define them with the same name is that it allows to
intercept malloc/free calls in code compiled without ASAN (e.g. libc's
getline). We won't check accesses to the buffer inside of libc, but we
will check them in the user code that uses the buffer.
Another reason is probably more important: in some cases memory is
allocated in one library and freed in another (again - getline). In
that case we may get a fatal crash/memory corruption is malloc/free
are not matching.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/address-sanitizer/CACT4Y%2BZOzqECfZzYkr%2BXvLs7enq%2B%2BxxmrKoULAwA3LSza%3DNXwQ%40mail.gmail.com.

Re: TSan "Nested bug" intermittent failure

['Dmitry Vyukov' via address-sanitizer]

Maybe this symbolizer warning is just a red herring, hard to say.

I've tried to create very long identifiers using this modified tsan test:
https://gist.githubusercontent.com/dvyukov/a8f167c5e62349ede83db3b69e77533b/raw/d500a54803d04140a581705a948844fa69080891/gistfile1.txt
and giving compiler different values in -DXXX=NNN, e.g. 1000, 2000.
And also with -DLongLongLongLong=AAA.

It easily triggers the "WARNING: Symbolizer buffer too small", but no crashes.

I don't have any other ideas so far. Maybe just some memory corruption.



On Fri, Jul 17, 2020 at 10:48 AM Christian Holler  wrote:
>
> Sorry for the long delay here, but I finally found time to work on this
> now and managed to reproduce it with some debug patch applied to our
> compiler-rt.
>
> So far, the only information I have is by changing the output here:
>
> > @@ -530,12 +531,23 @@ bool SymbolizerProcess::ReadFromSymbolizer(char
> > *buffer, uptr max_length) {
> >  if (ReachedEndOfOutput(buffer, read_len))
> >break;
> >  if (read_len + 1 == max_length) {
> > -  Report("WARNING: Symbolizer buffer too small\n");
> > +  Report("WARNING: Symbolizer buffer too small (%zu, %zu,
> > %zu)\n", read_len, max_length, just_read);
> >read_len = 0;
> >break;
> >  }
> >}
>
> With that, I see that the WARNINGs look like this:
>
> [task 2020-07-16T19:57:55.124Z] 19:57:55 INFO - GECKO(1266) |
> ==1385==WARNING: Symbolizer buffer too small (16383, 16384, 4095)
> [task 2020-07-16T19:57:55.125Z] 19:57:55 INFO - GECKO(1266) |
> ==1385==WARNING: Symbolizer buffer too small (16383, 16384, 4094)
> [task 2020-07-16T19:57:55.126Z] 19:57:55 INFO - GECKO(1266) |
> ==1385==WARNING: Symbolizer buffer too small (16383, 16384, 16383)
> [task 2020-07-16T19:57:55.127Z] 19:57:55 INFO - GECKO(1266) |
> ThreadSanitizer:DEADLYSIGNAL
> [task 2020-07-16T19:57:55.127Z] 19:57:55 INFO - GECKO(1266) |
> ThreadSanitizer: nested bug in the same thread, aborting.
>
> We have the first warning, where `just_read` is something around
> 4094/4095 quite often.
>
> However, when the "nested bug" appears, it it *always* 16383 (max_length
> - 1).
>
> I've been trying to output the buffer, but I am having difficulties in
> doing so (not sure if this is a problem in our CI or a problem in my
> patch, I will keep trying).
>
> If you have any idea what might be happening around this particular edge
> case, that would be great.
>
> I also tried locally what you suggested and tested sanitizer symbolizing
> with huge templates, but I was not able to reproduce the bug at all.
>
>
> Cheers,
>
> Chris
>
>
> On 06.05.20 12:29, Dmitry Vyukov wrote:
> > On Wed, May 6, 2020 at 12:13 PM Christian Holler  
> > wrote:
> >> Hi,
> >>
> >> in our CI, we keep encountering this intermittent failure:
> >>
> >>> ==3406==WARNING: Symbolizer buffer too small
> >>> ==3406==WARNING: Symbolizer buffer too small
> >>> ThreadSanitizer:DEADLYSIGNAL
> >>> ThreadSanitizer: nested bug in the same thread, aborting.
> >> We are tracking this issue at
> >> https://bugzilla.mozilla.org/show_bug.cgi?id=1615608
> >>
> >> Any advice on how to debug/fix this problem? Also, if this has been
> >> addressed in a newer Clang version, would you mind pointing me at the
> >> fix, so we can backport it? We are still using Clang 9 in CI for now.
> > Hi Christian,
> >
> > There seems to be some correlation between these "Symbolizer buffer
> > too small" warnings and subsequent hard crash, right?
> > The bug probably affects all sanitizers because the code is all common.
> > I wonder if we have a bug on that path. I suspect it may have never been 
> > tested.
> >
> > I am not aware of any fixes in that area (though, it's not that I was
> > looking at all fixes).
> >
> > I see several reasonable next steps:
> > 1. Add a test with an extremely large function name in a report
> > (should be doable with some C++ recursive template magic). Wonder how
> > compilers handle function name >16K
> > Just to check if we have some stupid bug on that path.
> >
> > 2. Extend the error message to dump max_length, read_len, input
> > command and what was read from symbolizer so far.
> > It may provide some insight into what happens.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/address-sanitizer/CACT4Y%2BZWRO7SXkDQaAFojL%3D5%2BMrGOpHrZ9sorhRAt03Ys4o-ZQ%40mail.gmail.com.

Re: KLSAN

['Dmitry Vyukov' via address-sanitizer]

On Fri, Jun 12, 2020 at 5:09 PM Maxime Villard  wrote:
>
> Hi,
> I wanted to know if you had plans on possible KLSAN LLVM instrumentation
> to develop in order to detect kernel memory leaks automatically.
>
> I see this as an important need on both Linux and NetBSD. Memory leaks are
> the last class of bugs we don't have automatic detection for; some other
> subsystems exist, such as KMEMLEAK, but they have serious limitations.
>
> Some time ago I thought about an approach based on DFSan. On the compiler
> side:
>   - A 1:1 shadow, where each 8 bytes of kernel memory is backed by 8 bytes
> of shadow.
>   - Memory transfers / loads / stores go through dedicated __lsan_memcpy()
> and other functions, to transfer the shadow metadata (similar to MSan).
>   - Args / returns are stored in a TLS obtained by a call to the
> __lsan_get_context_state() function (similar to MSan).
> On the kernel side:
>   - The return value of kmem_alloc()/malloc()/etc is tagged in the shadow.
> The tag is basically just a pointer to a structure that contains a
> refcount to the allocated area.
>   - The tag naturally gets propagated in the shadow via loads/stores/etc.
>   - Given that each load/store/etc goes through __lsan_memcpy()/etc, the
> kernel has the opportunity to see when a pointer to mallocated memory
> gets overwritten (that is, when its tag gets overwritten).
>   - When such a pointer gets overwritten, the kernel decrements the
> refcount pointed to by the tag of the to-be-overwritten pointer. If
> the refcount reaches zero, and the area is still allocated, we have a
> leak.
>
> Obviously, there would have to be some tricks on the LLVM side, to not
> have cases where a mallocated pointer has no tag but is in a register.
> But overall it seems like a good KLSAN basis to me.
>
> Dmitry, is KLSAN something your instrumentation team would be interested
> in considering? I would tend to see that as extremely beneficial to Linux
> and other OSes.
>
> I would be interested in working on that with other people.
>
> Regards,
> Maxime

+address-sanitizer

Hi Maxime,

We do not have any plans to work on something like this in the near
future (1 year at least).

For the Linux kernel we use KMEMLEAK to find leaks. While it has some
implementation issues, it works and finds real leaks and these
implementation issues seem to be fixable.

Regarding the approach you described, frankly it looks
over-complicated to me and I would expect a very long tail of various
corner and special cases, at least for the Linux kernel. E.g.
non-instrumented asm, unions, pointers with high/low bits used for
something else, various special execution contexts and context
switches, etc. I am not even sure it will be possible to get a fully
working implementation without false positives and crashes.
The memory scanning approach used by both LSAN and KMEMLEAK looks much
more simple and robust to me. I can imagine a compiler providing some
aid for that, like register/memory slot lifetime markers, some type
info, but all of that only fights false negatives, so I would consider
it as an optional addition only.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/address-sanitizer/CACT4Y%2BZKe_vt4tymHAFJ3O2OTKn177KEoWGa%3D6-6WLGS8sfW5A%40mail.gmail.com.

Re: TSan "Nested bug" intermittent failure

['Dmitry Vyukov' via address-sanitizer]

On Wed, May 6, 2020 at 12:13 PM Christian Holler  wrote:
>
> Hi,
>
> in our CI, we keep encountering this intermittent failure:
>
> > ==3406==WARNING: Symbolizer buffer too small
> > ==3406==WARNING: Symbolizer buffer too small
> > ThreadSanitizer:DEADLYSIGNAL
> > ThreadSanitizer: nested bug in the same thread, aborting.
>
> We are tracking this issue at
> https://bugzilla.mozilla.org/show_bug.cgi?id=1615608
>
> Any advice on how to debug/fix this problem? Also, if this has been
> addressed in a newer Clang version, would you mind pointing me at the
> fix, so we can backport it? We are still using Clang 9 in CI for now.

Hi Christian,

There seems to be some correlation between these "Symbolizer buffer
too small" warnings and subsequent hard crash, right?
The bug probably affects all sanitizers because the code is all common.
I wonder if we have a bug on that path. I suspect it may have never been tested.

I am not aware of any fixes in that area (though, it's not that I was
looking at all fixes).

I see several reasonable next steps:
1. Add a test with an extremely large function name in a report
(should be doable with some C++ recursive template magic). Wonder how
compilers handle function name >16K
Just to check if we have some stupid bug on that path.

2. Extend the error message to dump max_length, read_len, input
command and what was read from symbolizer so far.
It may provide some insight into what happens.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/address-sanitizer/CACT4Y%2BYOkb_mOLLXCH%3D9XQSzCtSs%3DR%3DuNPtrKyRzgaWHNU0P8A%40mail.gmail.com.

Re: [PATCH] D78111: tsan: fixes to ThreadClock::releaseStoreAcquire

['Dmitry Vyukov' via address-sanitizer]

On Thu, Apr 16, 2020 at 11:08 PM Daniel Fava  wrote:
>
> Hi Dmitry,
>
> Thanks for the help.  I’ve now committed the changes.
>
> I do have a question about workflow, maybe you could point me in the right 
> direction.
>
> To get the patch in, I first cloned a fresh copy of the repo, applied the 
> patch, committed and pushed.  Is that a reasonable way of doing this?
>
> I had originally I tried merging the changes, but origin rejected my push.
>
> What do you usually do?  Do you use rebase?  (I’m not very familiar with 
> rebase and didn’t want to risk breaking something.)
>
> Thanks again,
>
> Daniel

+sanitizer mailing lists

Hi Daniel,

Yes, I usually apply a patch, commit and push.
However, if it's mine change, I usually already have it committed,
then I rebase with git and then push.

I've also heard people mention some tool called Arcanist to work with
review changes:
https://llvm.org/docs/Phabricator.html#requesting-a-review-via-the-command-line
but I never used it personally.

> Begin forwarded message:
>
> From: Dmitry Vyukov via Phabricator 
> Subject: [PATCH] D78111: tsan: fixes to ThreadClock::releaseStoreAcquire
> Date: 16 April 2020 at 11:49:28 CEST
> To: daniel.f...@gmail.com
> Reply-To: reviews+d78111+public+3d0320cd01ab0...@reviews.llvm.org
>
> dvyukov accepted this revision.
> dvyukov added a comment.
> This revision is now accepted and ready to land.
>
> You can now land this yourself, right?
>
>
> CHANGES SINCE LAST ACTION
>  https://reviews.llvm.org/D78111/new/
>
> https://reviews.llvm.org/D78111
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/address-sanitizer/CACT4Y%2BaNiF7nEMZ%2BgQg7mbZj-0TPP%3DzL34SQ%2B-LCkvs17Qgejg%40mail.gmail.com.

Re: __asan_default_options not called

['Dmitry Vyukov' via address-sanitizer]

On Tue, Dec 18, 2018 at 1:15 PM  wrote:
>
> We have added __asan_default_option() so that asan canpickup default options 
> from it.
>
> However, those options are not applied. To see if __asan_default_options is 
> called, I added a log but even log is not printed which means 
> __asan_default_options is not invoked at all.
>
> const char *__asan_default_options() {
>   fprintf(stderr, "__asan_default_options called\n");
>   return 
> "verbosity=1:disable_coredump=0::unmap_shadow_on_exit=1:abort_on_error=1:log_path=/cores/logs.txt";
> }
>
> Any idea?

extern "C"?

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [compiler-rt] Use of ESR context in AArch64 sigframe

['Dmitry Vyukov' via address-sanitizer]

On Wed, Mar 7, 2018 at 4:39 PM, Andrey Ryabinin  wrote:
> On 03/06/2018 08:58 PM, Will Deacon wrote:
>> Hi all,
>>
>> As part of some recent work to harden the Kernel Address Space Layout
>> Randomisation (KASLR) implementation in arm64 Linux, I've proposed a
>> patch for the kernel which omits the ESR context from the signal frame
>> if the faulting virtual address is outside the range of addresses which
>> can be mapped by userspace.
>>
>> http://lists.infradead.org/pipermail/linux-arm-kernel/2018-March/563837.html
>>
>> Looking around, it seems that AddressSanitizer is using this information
>> in compiler-rt in order to distinguish the faulting access type between
>> READ, WRITE or UNKNOWN. With this change, all attempted accesses to kernel
>> memory from userspace will be reported as UNKNOWN.
>>
>> Is this likely to cause a problem?
>
> I guess this shouldn't be a bid deal.
> AFAICS compiler-rt uses this information only in diagnostic message.

+address-sanitizer mailing list

Hi,

These diagnostic messages are then parsed and analyzed, and access
type is used at least during automatic security pre-assessment. Being
capable to read arbitrary memory is different from being able to write
arbitrary memory. Though, I don't know how we treat UNKNOWN. If it's
the same as WRITE, then it's probably fine.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Suggestion] Runtime sanitizer, or minimal sanitized binary

['Dmitry Vyukov' via address-sanitizer]

On Wed, Mar 7, 2018 at 1:48 PM,   wrote:
>
>>
>> > :: for user-space binary It is similar to memcheck of valgrind.
>>
>> How will it be different from memcheck? ASAN is already similar to
>> memcheck, just faster. So why not just use ASAN?
>
>   I have to do investigate more to answer this question...

It's kind of an important question.

>> > Perform the wrong shift. ==> I think It can be a improvement point. ==>
>> > exit process at this point by various ways. It can be helpful to block a
>> > chance to exploit.
>>
>> I think there is already an option for this. All sanitizers are
>> capable of terminating the process on first bug.
>
>
>  Yap, I found it. -fno-sanitize-recover may be what you said.

There also must be a runtime flag.

>> > :: for loadable kernel module AFAIK, Runtime-sanitization hasn't tried
>> > yet on LKM. (right??)
>>
>> KASAN can perfectly work on kernel modules, if they are instrumented.
>> Potentially one could instrument only a single module (but not kernel
>> code code) and enable KASAN runtime.
>> The same can be achieved with runtime binary instrumentation too, but
>> will be much more complex. Is it what you mean?
>
>
>  Yes, I mean runtime binary instrumentation. Is it already implemented in
> KASAN??
>  I agree with your opinion that is more complex.
>  So, I wonder what you think whether it is needed or not.

I can only say that we are not doing it because (1) it's too complex,
(2) you can't protect something for which you don't have source code
anyway (consider that the code is not doing what it's claiming to be
doing but instead sends all information about your credit cards to
internet without any heap corruptions).


>  Use-case what I consider,,
>  When a user want to run untrusted app or LKM,
>  runtime sanitization can add mitigation for the case, without isolation
> technique.
>  (But I'm not sure It's really needed.)

Well, if it's untrusted and you don't have source code then you can't
protect it (or from it), see above.


>  (* one more question *)
>  If linux kernel is built with KASAN, UBSAN, ...
>  Can kernel-module be built with those sanitizer?? without any additional
> effort of user??

Yes. This is happening by default.


>> > :: How to minimize instrumentation?? Only Functions influenced by user
>> > input can be instrumented. In the case of kernel driver, It is not 
>> > difficult
>> > to extract function list to be instrmented.
>>
>> How do you want o extract this list of functions? Or you mean that
>> user supplies the list?
>
>
>For linux kernel,
>1) Select system call function, exported kernel function as starter.
>2) Do taint-analysis or measuring-coverage  from the starter.
>I think these kind of techniques can be leveraged to extract list of
> functions.
>3) Exclude functions that are not selected of 1), 2).  (I don't know yet
> how this can be implemented.. Can it be implemented by gcc-plugin??)
>==> Output :  Minimal sanitized linux kernel.   Can this kernel be
> suitable for product environment??  I don't know yet..
>
>For kernel driver,
>1) Extract common condition of starter function of kernel driver.
>--> Starter function is generally be connected with function pointer.
> e.g)  struct file_operations->ioctl() ==> kernel_driver_ioctl()
>--> I have to find more condition..
>2), 3) are same to "For linux kernel"
>==> Output :  Minimal sanitized kernel driver.
>
>** User-supplied list can be an easiest way as I think.


This looks too complex to be practical.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Suggestion] Runtime sanitizer, or minimal sanitized binary

['Dmitry Vyukov' via address-sanitizer]

On Wed, Mar 7, 2018 at 7:30 AM,   wrote:
>
> Hi,
>
> I have a suggestion about sanitizer.
> Even though what I have a high-level concept only,
> I want to get some feedback from here.
>
> Especially, I wanna know what you think about whether this solution is
> needed or not.
> or If It is already exist, let me know.
> (If below content have something wrong, please let me know.)
>
> For details of my idea, please check below. (README.md)
> - https://github.com/jinb-park/rsan/
>
> I don't develop anything yet, but I have a plan to develop my idea as PoC.

Hi Jinbum,

Could you clarify few aspects?

> :: for user-space binary It is similar to memcheck of valgrind.

How will it be different from memcheck? ASAN is already similar to
memcheck, just faster. So why not just use ASAN?

> Perform the wrong shift. ==> I think It can be a improvement point. ==> exit 
> process at this point by various ways. It can be helpful to block a chance to 
> exploit.

I think there is already an option for this. All sanitizers are
capable of terminating the process on first bug.

> :: for loadable kernel module AFAIK, Runtime-sanitization hasn't tried yet on 
> LKM. (right??)

KASAN can perfectly work on kernel modules, if they are instrumented.
Potentially one could instrument only a single module (but not kernel
code code) and enable KASAN runtime.
The same can be achieved with runtime binary instrumentation too, but
will be much more complex. Is it what you mean?

> :: How to minimize instrumentation?? Only Functions influenced by user input 
> can be instrumented. In the case of kernel driver, It is not difficult to 
> extract function list to be instrmented.

How do you want o extract this list of functions? Or you mean that
user supplies the list?

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [EXTERNAL]: Issues booting ASAN enabled build even with "mm: Revert x86_64 and arm64 ELF_ET_DYN_BASE base" applied

['Dmitry Vyukov' via address-sanitizer]

Hi,

I am missing some context.
How can this be reproduced? What is the user-space program? Is it
built with gcc or clang? What version? Does it require some special
kernel? What exactly?



On Wed, Oct 25, 2017 at 7:17 PM, Olav Haugan  wrote:
> Hi Kees/Kostya/Dmitry,
>
> Have you been able to reproduce this?
>
> Thanks,
>
> Olav
>
>>
>> Hi Kees,
>>
>> Thanks for looking into it. Yeah, I think the 32 bit base is still an issue.
>>
>> >
>> > I see this in the log:
>> >
>> > 01-01 00:06:46.743   658   658 I : ==658==ASan shadow was
>> > supposed to be located in the [0x-0x1fff] range.
>> >
>> > I've CCed Kostya and Dmitry who should be able to help track this
>> > down. The upstream revert left the 32-bit base alone since they'd said
>> > it wasn't a problem. Perhaps it still is in some way?
>> >
>> > -Kees
>> >
>> > On Fri, Sep 29, 2017 at 1:28 AM, Olav Haugan  wrote:
>> > > Hi Kees,
>> > >
>> > > We are having issue booting our devices due to the "binfmt_elf: use
>> > ELF_ET_DYN_BASE only for PIE" and "arm64: move ELF_ET_DYN_BASE to 4GB
>> /
>> > 4MB" patches. We tried applying the partial revert of these with [1] but 
>> > we are
>> > still running into issues. It appears that 32 bit processes are having an 
>> > issue.
>> > Logcat attached.
>> > >
>> > > Have you seen this issue yet? The only solution we have so far is to do a
>> > complete revert of above mentioned changes.
>> > >
>> > > [1] https://patchwork.kernel.org/patch/9886105/
>> > >
>> > > Thanks,
>> > >
>> > > --
>> > > Olav
>> > >
>> >
>> >
>> >
>> > --
>> > Kees Cook
>> > Pixel Security

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: sanitizer_posix.cc:131 "(("unable to unmap" && 0)) != (0)" (0x0, 0x0)

['Dmitry Vyukov' via address-sanitizer]

On Thu, Oct 19, 2017 at 10:56 PM,   wrote:
> Hi Martin,
> I am having the same issue too, it seems to be a real issue in the address
> sanitizer: https://github.com/apache/trafficserver/issues/1498.
>
> In my case I am running a C++ component on RHEL 6.7, compiled with GCC 6.2.1
> and using address sanitizer (-fsanitize=address -fno-omit-frame-pointer -O1
> -g)
> I get an error stack very similar to yours:
> ==14657==ERROR: AddressSanitizer failed to deallocate 0x4000 (16384) bytes
> at address 0x629001220200
> ==14657==AddressSanitizer CHECK failed:
> ../../../../libsanitizer/sanitizer_common/sanitizer_posix.cc:131 "(("unable
> to unmap" && 0)) != (0)" (0x0, 0x0)
> ==14657==ERROR: AddressSanitizer failed to deallocate 0x4000 (16384) bytes
> at address 0x62900121b200
> ==14657==AddressSanitizer CHECK failed:
> ../../../../libsanitizer/sanitizer_common/sanitizer_posix.cc:131 "(("unable
> to unmap" && 0)) != (0)" (0x0, 0x0)
> #0 0x33b18cb2cd  (/usr/lib64/libasan.so.3+0x33b18cb2cd)
> #1 0x33b18d0f13 in __sanitizer::CheckFailed(char const*, int, char
> const*, unsigned long long, unsigned long long)
> (/usr/lib64/libasan.so.3+0x33b18d0f13)
> #2 0x33b18dacaf  (/usr/lib64/libasan.so.3+0x33b18dacaf)
> #3 0x33b18db970  (/usr/lib64/libasan.so.3+0x33b18db970)
> #0 0x33b18cb2cd  (/usr/lib64/libasan.so.3+0x33b18cb2cd)
> #4 0x33b18ce578  (/usr/lib64/libasan.so.3+0x33b18ce578)
> #5 0x3c8ae07ae8 in start_thread (/lib64/libpthread.so.0+0x3c8ae07ae8)
> #6 0x3c8aae896c in __clone (/lib64/libc.so.6+0x3c8aae896c)
>
> Let me know if you have any clue on how to overcome this issue.
>
> Also, agree with your sentiment about the sanitizers, they are awesome!
>
> Marco
>
>
> On Wednesday, March 15, 2017 at 5:19:01 PM UTC-7, Martin Quinson wrote:
>>
>> Hello,
>>
>> we are great fans of the sanitizers, they catched quite a lot of bugs
>> in our codebase. We are really thankful for that.
>>
>> I come to you because they are however getting mad at our code, and
>> most of the remaining errors appearing seem to be somewhere in your
>> code, because we trigger an assertion failure in sanitizer_posix.
>>
>> (gcc 6.3.0, Debian testing)
>>
>> +==20715==ERROR: AddressSanitizer failed to deallocate 0x2000 (8192) bytes
>> at address 0x7f1096171a20
>> +==20715==AddressSanitizer CHECK failed:
>> ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix.cc:131
>> "(("unable to unmap" && 0)) != (0)" (0x0, 0x0)
>> +#0 0x7f1096243ebd  (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xcaebd)
>> +#1 0x7f1096249b03 in __sanitizer::CheckFailed(char const*, int, char
>> const*, unsigned long long, unsigned long long)
>> (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xd0b03)
>> +#2 0x7f109625378f  (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xda78f)
>> +#3 0x7f1096254430  (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xdb430)
>> +#4 0x7f1096247168  (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xce168)
>> +#5 0x7f10946cf4e8 in __nptl_deallocate_tsd.part.4
>> (/lib/x86_64-linux-gnu/libpthread.so.0+0x64e8)
>> +#6 0x7f10946d05d7 in start_thread
>> (/lib/x86_64-linux-gnu/libpthread.so.0+0x75d7)
>> +#7 0x7f10944139be in __clone
>> (/lib/x86_64-linux-gnu/libc.so.6+0xe89be)
>>
>> Here is an example:
>>
>> https://ci.inria.fr/simgrid/view/Tous/job/SimGrid-DynamicAnalysis-Sanitizer-Address/label=simgrid-debian8-64-dynamic-analysis-2/lastCompletedBuild/testReport/%28root%29/TestSuite/msg_plugin_hostload_thread/
>>
>> The configure and build log (gcc 6.3.0, Debian 8):
>>
>> https://ci.inria.fr/simgrid/view/Tous/job/SimGrid-DynamicAnalysis-Sanitizer-Address/label=simgrid-debian8-64-dynamic-analysis-2/lastCompletedBuild/console
>>
>> The source code is available:
>> https://github.com/simgrid/simgrid
>>
>> The script used to build on the CI server is also available:
>> https://github.com/simgrid/simgrid/blob/master/tools/jenkins/Sanitizers.sh
>>
>> In short:
>> cmake -Denable_documentation=OFF -Denable_lua=ON -Denable_java=OFF \
>>   -Denable_compile_optimizations=ON -Denable_compile_warnings=ON \
>>   -Denable_jedule=ON -Denable_mallocators=OFF \
>>   -Denable_smpi=ON -Denable_smpi_MPICH3_testsuite=ON
>> -Denable_model-checking=OFF \
>>   -Denable_memcheck=OFF -Denable_memcheck_xml=OFF
>> -Denable_smpi_ISP_testsuite=ON -Denable_coverage=OFF\
>>   -Denable_fortran=OFF -Denable_address_sanitizer=ON \
>>   -Denable_undefined_sanitizer=OFF -Denable_thread_sanitizer=OFF
>> make -j4
>> examples/msg/plugin-hostload/plugin-hostload
>> examples/platforms/energy_platform.xml --cfg=contexts/factory:thread
>>
>> Attached: strace output of the offending command. Note that the strace
>> was captured on my laptop, not on the server. I tried to reproduce the
>> same conditions, but I'm not 100% sure.
>>
>> If you need more info, please just ask.
>>
>> Thanks again for this magical tool,
>> Mt.


Hi,

Somehow this was forgotten. Sorry.
This seems to be all about AddressSanitizer, so 

Re: Stack Variables and Global Variables

['Dmitry Vyukov' via address-sanitizer]

You can play with these flags to reduce heap overheads:

ASAN_FLAG(
bool, poison_heap, true,
"Poison (or not) the heap memory on [de]allocation. Zero value is
useful "
"for benchmarking the allocator or instrumentator.")
ASAN_FLAG(
int, max_malloc_fill_size, 0x1000,  // By default, fill only the first
4K.
"ASan allocator flag. max_malloc_fill_size is the maximal amount of "
"bytes that will be filled with malloc_fill_byte on malloc.")
ASAN_FLAG(int, max_redzone, 2048,
  "Maximal size (in bytes) of redzones around heap objects.")
ASAN_FLAG(int, quarantine_size_mb, -1,
  "Size (in Mb) of quarantine used to detect use-after-free "
  "errors. Lower value may reduce memory usage but increase the "
  "chance of false negatives.")
ASAN_FLAG(int, thread_local_quarantine_size_kb, -1,
  "Size (in Kb) of thread local quarantine used to detect "
  "use-after-free errors. Lower value may reduce memory usage but "
  "increase the chance of false negatives. It is not advised to go "
  "lower than 64Kb, otherwise frequent transfers to global
quarantine "
  "might affect performance.")






On Thu, Jun 22, 2017 at 11:53 AM, Alexander Potapenko <
ramosian.gli...@gmail.com> wrote:

> This is more or less expected.
> As Kuba said, there's no distinction between a check for a heap overflow
> or global overflow - we just check that every memory access is valid.
>
> On Thu, Jun 22, 2017 at 9:08 AM, Aayushi Agrawal 
> wrote:
>
>> Yes I tried but its coming out to be almost same.
>>
>> Slowdown is almost 2 times.
>>
>> On Wed, Jun 21, 2017 at 9:28 PM, Kuba Mracek  wrote:
>>
>>> How big is the slowdown?  Did you try using ASan with higher
>>> optimization levels (-O1, -O2, -O3)?
>>>
>>> At the point where ASan instruments memory reads and writes, it doesn't
>>> know whether the destination is a stack variable, a global variable or a
>>> pointer to the heap.  So I don't think there's a way of selectively only
>>> instrumenting stack and/or globals.
>>>
>>> Kuba
>>>
>>> On 21 Jun 2017, at 04:11, Aayushi Agrawal 
>>> wrote:
>>>
>>> Hello
>>>
>>> I used " -fsanitise=address " as one of the option for compiling my
>>> program with clang.
>>> But it slows down the execution of the program.
>>>
>>> Can anybody please tell me that is there a way to only check the stack
>>> variables overflow, global variables overflow and avoid all other checks
>>> using ASAN which could help in improving the execution speed?
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "address-sanitizer" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to address-sanitizer+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>>
>>
>> --
>> Aayushi Agrawal
>> B.Tech(Final Year)
>> Computer Science Engineering Undergraduate
>> LNMIIT,JAIPUR
>> Contact - 09649357639
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "address-sanitizer" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to address-sanitizer+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "address-sanitizer" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to address-sanitizer+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Looking for source code of ASan

['Dmitry Vyukov' via address-sanitizer]

On Wed, May 24, 2017 at 8:28 PM, PhD - Eyasu Getahun Chekole
 wrote:
> Dear all,
>
> Could you please help me pointing out location of the source code?
>
> Dear Dmitry,
>
> You replied me with empty message last time :).

Pretty sure my email contained "The source code is in gcc already
(libsanitizer/asan), just build gcc".

> Best regards,
>
> Eyasu.
>
> 
> From: Dmitry Vyukov 
> Sent: Thursday, May 4, 2017 11:47:00 PM
> To: PhD - Eyasu Getahun Chekole
> Cc: k...@google.com; bruen...@google.com; gli...@google.com; address-sanitizer
> Subject: Re: Looking for source code of ASan
>
> On Thu, May 4, 2017 at 5:11 PM, PhD - Eyasu Getahun Chekole
>  wrote:
>> Dear all,
>>
>> I think the source code of ASan is moved to the LLVM compiler 
>> infrastructure. But I would want to get the source code and build it to work 
>> with gcc, not with clang. I couldn't find a pointer where to get the source 
>> code to do that. Could you please point me where to get it?
>
> Hi Eyasu,
>
> The source code is in gcc already (libsanitizer/asan), just build gcc.
> The above message may contain confidential and/or proprietary information 
> that is exempt from disclosure under applicable law and is intended for 
> receipt and use solely by the addressee(s) named above. If you are not the 
> intended recipient, you are hereby notified that any use, dissemination, 
> distribution, or copying of this message, or any attachment, is strictly 
> prohibited. If you have received this email in error, please inform the 
> sender immediately by reply e-mail or telephone, reversing the charge if 
> necessary. Please delete the message thereafter. Thank you.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Looking for source code of ASan

['Dmitry Vyukov' via address-sanitizer]

On Thu, May 4, 2017 at 5:11 PM, PhD - Eyasu Getahun Chekole
 wrote:
> Dear all,
>
> I think the source code of ASan is moved to the LLVM compiler infrastructure. 
> But I would want to get the source code and build it to work with gcc, not 
> with clang. I couldn't find a pointer where to get the source code to do 
> that. Could you please point me where to get it?

Hi Eyasu,

The source code is in gcc already (libsanitizer/asan), just build gcc.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ASan on OpenBSD

['Dmitry Vyukov' via address-sanitizer]

On Mon, May 1, 2017 at 12:28 PM, Mark Kettenis  wrote:
>> From: Dmitry Vyukov 
>> Date: Mon, 1 May 2017 10:43:26 +0200
>>
>> On Mon, May 1, 2017 at 8:51 AM, Greg Steuck  wrote:
>> > I naively tried to build something with -fsanitize=address using llvm-4.0
>> > port available on OpenBSD 6.1-amd64. I was immediately greeted with:
>> >   clang-4.0: error: unsupported option '-fsanitize=address' for target
>> >   'amd64-unknown-openbsd6.1'
>> >
>> > How deep a rat hole does one have to go to port ASan to a new
>> > flavour of BSD? Is OpenBSD going to be particularly painful with
>> > its special malloc and advanced ASLR? Is anybody working on this?
>
> Our focus is currently still on integrating llvm/clang/lld into
> OpenBSD.  As far as I know nobody is working on this yet, but it
> sounds like something we'd certainly be interested in having.
>
>> Hi,
>>
>> I can think of 2 major areas re porting to a new OS:
>> 1. Function interception. Presumably our current scheme just works on
>> OpenBSD (as it works on Linux and FreeBSD).
>> 2. Shadow memory mapping. We need to mmap a multi-TB range in process
>> address space. Kernel need to support such huge mappings at all and
>> with reasonable performance. How aggressive is ASLR? Is it possible to
>> turn it off for a process (that may be the simplest option)? What's so
>> special about malloc? Note that asan replaces standard malloc
>> entirely.
>
> The ASLR is pretty aggressive and there is (quite deliberately) no
> option to turn it off.  Our malloc has some pretty nift bug detection
> strategies built-in such as guard pages, aggressive unmapping of freed
> memory, aggressive radomization, etc.  Many of these features are
> turned on by default.  But replacing malloc with a different
> implementation is still possible in OpenBSD.
>
> Currently mapping a mult-TB range doesn't work in OpenBSD.  But I
> think it could be made to work if most of that range will be mapped
> with PROT_NONE.

It needs to be mapped with PROT_READ|PROT_WRITE and MAP_NORESERVE. We
don't have code that later would detect which parts of the mapping
actually become active and need to be remapped from PROT_NONE to
PROT_RW, we rely on lazy, on-demand paging in.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: GetPreviousInstructionPc question

['Dmitry Vyukov' via address-sanitizer]

On Thu, Apr 20, 2017 at 12:00 PM, Yuri Gribov <tetra2...@gmail.com> wrote:
> On Thu, Apr 20, 2017 at 10:53 AM, 'Dmitry Vyukov' via
> address-sanitizer <address-sanitizer@googlegroups.com> wrote:
>> On Thu, Apr 20, 2017 at 11:50 AM, Yuri Gribov <tetra2...@gmail.com> wrote:
>>> On Thu, Apr 20, 2017 at 10:20 AM, 'Dmitry Vyukov' via
>>> address-sanitizer <address-sanitizer@googlegroups.com> wrote:
>>>> On Thu, Apr 20, 2017 at 11:11 AM, evgeny777 <evgeny.levi...@gmail.com> 
>>>> wrote:
>>>>> Thanks for clarifying it, Dmitry.
>>>>>
>>>>> Here is piece of report I get:
>>>>>
>>>>> ==18244==ERROR: AddressSanitizer: heap-buffer-overflow on address
>>>>> 0x6020001a at pc 0x005a9cad bp 0x7ffc10528760 sp 0x7ffc10528740
>>>>> WRITE of size 1 at 0x6020001a thread T0
>>>>> #0 0x5a9cac  (/home/evgeny/work/linker_scripts/asan/asan+0x5a9cac)
>>>>> #1 0x7f310488082f  (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
>>>>> #2 0x419498  (/home/evgeny/work/linker_scripts/asan/asan+0x419498)
>>>>>
>>>>> 
>>>>>
>>>>> Below is the piece of disassembly of main :
>>>>>
>>>>> .
>>>>> 0x5a9ca8 <+136>: callq  0x56d9d0  ;
>>>>> ::__asan_report_store1(__sanitizer::uptr) at asan_rtl.cc:136
>>>>> 0x5a9cad <+141>: xorl   %eax, %eax
>>>>> .
>>>>>
>>>>> As you may noticed 0x5a9cac == (0x5a9cad - 1)
>>>>
>>>>
>>>> I think tsan prints unmodified PC and we should do the same in asan.
>>>> This also reliefs us from figuring out correct instruction length on
>>>> ARM/thumb/etc as nobody sees the modified PC.
>>>
>>> Hm, the unmodified PC will make symbolized stacktraces less readable.
>>> What's the problem with "-1"? Addr2line and other bintools work fine
>>> with it.
>>
>> I literally mean "print unmodified PC" (as a hex value). I am not
>> proposing to change how symbolization works.
>
> My understanding is that symbolization code symbolizes "pc-1" as well.
> If we keep symbolization code unchanged and only change printed hex
> value, this will probly cause offline symbolizer to behave differently
> from internal symbolizer.


Okay, I don't think this is worth any more of our time.
If there is a spot improvement to GetPreviousInstructionPc for arm,
that will be fine to do.



> Actually the situation is even more interesting: on ARM we do pc &
> (~1) (not pc-1) to cancel out Thumb bit.
>
>>>>> On Thursday, April 20, 2017 at 12:01:25 PM UTC+3, Dmitry Vyukov wrote:
>>>>>>
>>>>>> On Thu, Apr 20, 2017 at 10:44 AM, evgeny777 <evgeny@gmail.com> wrote:
>>>>>> > I noticed that GetPreviousInstructionPc() function returns 'pc - 1' for
>>>>>> > both
>>>>>> > arm32 and arm64.
>>>>>> > This causes odd addresses to appear in stack traces, which is nonsense,
>>>>>> > as
>>>>>> > both arm32/64 instructions
>>>>>> > have 4 byte size and alignment.
>>>>>> >
>>>>>> > The x86 and x86_64 cases are even more confusing, because instruction
>>>>>> > length
>>>>>> > is not constant. What exactly this 'pc - 1' is expected to return?
>>>>>> >
>>>>>> > But even if one is able to get previous instruction address correctly 
>>>>>> > he
>>>>>> > may
>>>>>> > still get confusing results. In case some instruction triggers
>>>>>> > hardware exception, its address will go to ASAN stack trace (via
>>>>>> > SlowUnwindStackWithContext). Returning address of previous instruction
>>>>>> > in such case can be extremely confusing.
>>>>>> >
>>>>>> > Is there any point in using this function?
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Yes, there is a very bold point in using this function.
>>>>>> Typically top frame PC is obtained with __builtin_return_address,
>>>>>> which means that it points to the next instruction after the call. And
>>>>>> we need to obtain debug info associated with the call instruction. To
>>>>>

Re: GetPreviousInstructionPc question

['Dmitry Vyukov' via address-sanitizer]

On Thu, Apr 20, 2017 at 11:50 AM, Yuri Gribov <tetra2...@gmail.com> wrote:
> On Thu, Apr 20, 2017 at 10:20 AM, 'Dmitry Vyukov' via
> address-sanitizer <address-sanitizer@googlegroups.com> wrote:
>> On Thu, Apr 20, 2017 at 11:11 AM, evgeny777 <evgeny.levi...@gmail.com> wrote:
>>> Thanks for clarifying it, Dmitry.
>>>
>>> Here is piece of report I get:
>>>
>>> ==18244==ERROR: AddressSanitizer: heap-buffer-overflow on address
>>> 0x6020001a at pc 0x005a9cad bp 0x7ffc10528760 sp 0x7ffc10528740
>>> WRITE of size 1 at 0x6020001a thread T0
>>> #0 0x5a9cac  (/home/evgeny/work/linker_scripts/asan/asan+0x5a9cac)
>>> #1 0x7f310488082f  (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
>>> #2 0x419498  (/home/evgeny/work/linker_scripts/asan/asan+0x419498)
>>>
>>> 
>>>
>>> Below is the piece of disassembly of main :
>>>
>>> .
>>> 0x5a9ca8 <+136>: callq  0x56d9d0  ;
>>> ::__asan_report_store1(__sanitizer::uptr) at asan_rtl.cc:136
>>> 0x5a9cad <+141>: xorl   %eax, %eax
>>> .
>>>
>>> As you may noticed 0x5a9cac == (0x5a9cad - 1)
>>
>>
>> I think tsan prints unmodified PC and we should do the same in asan.
>> This also reliefs us from figuring out correct instruction length on
>> ARM/thumb/etc as nobody sees the modified PC.
>
> Hm, the unmodified PC will make symbolized stacktraces less readable.
> What's the problem with "-1"? Addr2line and other bintools work fine
> with it.

I literally mean "print unmodified PC" (as a hex value). I am not
proposing to change how symbolization works.

>>> On Thursday, April 20, 2017 at 12:01:25 PM UTC+3, Dmitry Vyukov wrote:
>>>>
>>>> On Thu, Apr 20, 2017 at 10:44 AM, evgeny777 <evgeny@gmail.com> wrote:
>>>> > I noticed that GetPreviousInstructionPc() function returns 'pc - 1' for
>>>> > both
>>>> > arm32 and arm64.
>>>> > This causes odd addresses to appear in stack traces, which is nonsense,
>>>> > as
>>>> > both arm32/64 instructions
>>>> > have 4 byte size and alignment.
>>>> >
>>>> > The x86 and x86_64 cases are even more confusing, because instruction
>>>> > length
>>>> > is not constant. What exactly this 'pc - 1' is expected to return?
>>>> >
>>>> > But even if one is able to get previous instruction address correctly he
>>>> > may
>>>> > still get confusing results. In case some instruction triggers
>>>> > hardware exception, its address will go to ASAN stack trace (via
>>>> > SlowUnwindStackWithContext). Returning address of previous instruction
>>>> > in such case can be extremely confusing.
>>>> >
>>>> > Is there any point in using this function?
>>>>
>>>> Hi,
>>>>
>>>> Yes, there is a very bold point in using this function.
>>>> Typically top frame PC is obtained with __builtin_return_address,
>>>> which means that it points to the next instruction after the call. And
>>>> we need to obtain debug info associated with the call instruction. To
>>>> achieve that we subtract 1 from PC. All symbolization code that we've
>>>> seen is fine with PC pointing into a middle of an instruction.
>>>>
>>>> Now, if we print pc-1 in reports (do we?), then it's a bug. We need to
>>>> print unaltered PC in reports.
>>>>
>>>> Re hardware exceptions. This needs to be fixed. A trivial change would
>>>> be to add 1 to PCs pointing to faulting instruction. Then
>>>> GetPreviousInstructionPc will offset this and we get correct debug
>>>> info. However, then we will print incorrect PC in report. So a proper
>>>> fix would be to augment all stack traces with a flag saying if top PC
>>>> needs to be adjusted during symbolization or not.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "address-sanitizer" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to address-sanitizer+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "address-sanitizer" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to address-sanitizer+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "address-sanitizer" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to address-sanitizer+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: GetPreviousInstructionPc question

['Dmitry Vyukov' via address-sanitizer]

On Thu, Apr 20, 2017 at 11:11 AM, evgeny777  wrote:
> Thanks for clarifying it, Dmitry.
>
> Here is piece of report I get:
>
> ==18244==ERROR: AddressSanitizer: heap-buffer-overflow on address
> 0x6020001a at pc 0x005a9cad bp 0x7ffc10528760 sp 0x7ffc10528740
> WRITE of size 1 at 0x6020001a thread T0
> #0 0x5a9cac  (/home/evgeny/work/linker_scripts/asan/asan+0x5a9cac)
> #1 0x7f310488082f  (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
> #2 0x419498  (/home/evgeny/work/linker_scripts/asan/asan+0x419498)
>
> 
>
> Below is the piece of disassembly of main :
>
> .
> 0x5a9ca8 <+136>: callq  0x56d9d0  ;
> ::__asan_report_store1(__sanitizer::uptr) at asan_rtl.cc:136
> 0x5a9cad <+141>: xorl   %eax, %eax
> .
>
> As you may noticed 0x5a9cac == (0x5a9cad - 1)


I think tsan prints unmodified PC and we should do the same in asan.
This also reliefs us from figuring out correct instruction length on
ARM/thumb/etc as nobody sees the modified PC.




> On Thursday, April 20, 2017 at 12:01:25 PM UTC+3, Dmitry Vyukov wrote:
>>
>> On Thu, Apr 20, 2017 at 10:44 AM, evgeny777  wrote:
>> > I noticed that GetPreviousInstructionPc() function returns 'pc - 1' for
>> > both
>> > arm32 and arm64.
>> > This causes odd addresses to appear in stack traces, which is nonsense,
>> > as
>> > both arm32/64 instructions
>> > have 4 byte size and alignment.
>> >
>> > The x86 and x86_64 cases are even more confusing, because instruction
>> > length
>> > is not constant. What exactly this 'pc - 1' is expected to return?
>> >
>> > But even if one is able to get previous instruction address correctly he
>> > may
>> > still get confusing results. In case some instruction triggers
>> > hardware exception, its address will go to ASAN stack trace (via
>> > SlowUnwindStackWithContext). Returning address of previous instruction
>> > in such case can be extremely confusing.
>> >
>> > Is there any point in using this function?
>>
>> Hi,
>>
>> Yes, there is a very bold point in using this function.
>> Typically top frame PC is obtained with __builtin_return_address,
>> which means that it points to the next instruction after the call. And
>> we need to obtain debug info associated with the call instruction. To
>> achieve that we subtract 1 from PC. All symbolization code that we've
>> seen is fine with PC pointing into a middle of an instruction.
>>
>> Now, if we print pc-1 in reports (do we?), then it's a bug. We need to
>> print unaltered PC in reports.
>>
>> Re hardware exceptions. This needs to be fixed. A trivial change would
>> be to add 1 to PCs pointing to faulting instruction. Then
>> GetPreviousInstructionPc will offset this and we get correct debug
>> info. However, then we will print incorrect PC in report. So a proper
>> fix would be to augment all stack traces with a flag saying if top PC
>> needs to be adjusted during symbolization or not.
>
> --
> You received this message because you are subscribed to the Google Groups
> "address-sanitizer" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to address-sanitizer+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: GetPreviousInstructionPc question

['Dmitry Vyukov' via address-sanitizer]

On Thu, Apr 20, 2017 at 10:44 AM, evgeny777  wrote:
> I noticed that GetPreviousInstructionPc() function returns 'pc - 1' for both
> arm32 and arm64.
> This causes odd addresses to appear in stack traces, which is nonsense, as
> both arm32/64 instructions
> have 4 byte size and alignment.
>
> The x86 and x86_64 cases are even more confusing, because instruction length
> is not constant. What exactly this 'pc - 1' is expected to return?
>
> But even if one is able to get previous instruction address correctly he may
> still get confusing results. In case some instruction triggers
> hardware exception, its address will go to ASAN stack trace (via
> SlowUnwindStackWithContext). Returning address of previous instruction
> in such case can be extremely confusing.
>
> Is there any point in using this function?

Hi,

Yes, there is a very bold point in using this function.
Typically top frame PC is obtained with __builtin_return_address,
which means that it points to the next instruction after the call. And
we need to obtain debug info associated with the call instruction. To
achieve that we subtract 1 from PC. All symbolization code that we've
seen is fine with PC pointing into a middle of an instruction.

Now, if we print pc-1 in reports (do we?), then it's a bug. We need to
print unaltered PC in reports.

Re hardware exceptions. This needs to be fixed. A trivial change would
be to add 1 to PCs pointing to faulting instruction. Then
GetPreviousInstructionPc will offset this and we get correct debug
info. However, then we will print incorrect PC in report. So a proper
fix would be to augment all stack traces with a flag saying if top PC
needs to be adjusted during symbolization or not.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About Asan ASAN_MEMORY_ACCESS_CALLBACK_BODY macro

['Dmitry Vyukov' via address-sanitizer]

Yes, all negative shadow values mean "all 8 bytes are bad".


On Tue, Nov 29, 2016 at 5:17 AM, steven shi  wrote:
> OK, I think I understand now. The (s8) is the key. It force cast the 0xf9
> value to a signed negative char which is less than 3. Thank you anyway!
>
> --
> You received this message because you are subscribed to the Google Groups
> "address-sanitizer" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to address-sanitizer+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: About Asan ASAN_MEMORY_ACCESS_CALLBACK_BODY macro

['Dmitry Vyukov' via address-sanitizer]

On Mon, Nov 28, 2016 at 3:06 PM, steven shi  wrote:
> Hello,
> I'm porting the Asan RT lib to my firmware, and I meet a issue to block my
> shadow memory checking work correctly. I have to update the Asan core logic
> macro to let it works in my side, and I hope some expert could help me
> understand below code correctly.
>
> LLVM Asan use below macro to check whether the addr shadow memory  is
> poisoned or not. My issue is about the line 166 condition: (size >=
> SHADOW_GRANULARITY ||  ...), and  I have to update it as (s >=
> SHADOW_GRANULARITY ||  ...). I think the "size" is just from hard code
> INTERFACE function name definition (e.g. ASAN_MEMORY_ACCESS_CALLBACK(load,
> false, 16)), and its values are one of {1,2,4,8,16}. So, for __asan_load8
> and __asan_load16 with 8 granularity, the "size" will be 8 and 16, and (size
>>= SHADOW_GRANULARITY) will always be true, isn't it wrong? My understanding
> is we need check the addr shadow memory value here to make sure the addr's
> shadow memory value is less than its covered SHADOW_GRANULARITY bound.
> Appreciate any clarification about this code.

> I think the "size" is just from hard code INTERFACE function name definition 
> (e.g. ASAN_MEMORY_ACCESS_CALLBACK(load, false, 16)), and its values are one 
> of {1,2,4,8,16}

That's true and correct.

If access size is 8 or 16, then we just compare shadow value s with 0.
If it's not zero, we report bug.
For accesses of size 1, 2, 4 we need more complex check: s != 0 &&
((s8)((addr & (SHADOW_GRANULARITY - 1)) + size - 1)) >= (s8)s)




> http://llvm.org/svn/llvm-project/compiler-rt/trunk/lib/asan/asan_rtl.cc
> line 161:
> #define ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, exp_arg,
> fatal) \
> uptr sp = MEM_TO_SHADOW(addr);
> \
> uptr s = size <= SHADOW_GRANULARITY ? *reinterpret_cast(sp)
> \
> : *reinterpret_cast(sp);
> \
> if (UNLIKELY(s)) {
> \
>   if (UNLIKELY(size >= SHADOW_GRANULARITY ||
> \
>((s8)((addr & (SHADOW_GRANULARITY - 1)) + size - 1)) >=
> \
>(s8)s)) {
> \
> if (__asan_test_only_reported_buggy_pointer) {
> \
>   *__asan_test_only_reported_buggy_pointer = addr;
> \
> } else {
> \
>   GET_CALLER_PC_BP_SP;
> \
>   ReportGenericError(pc, bp, sp, addr, is_write, size, exp_arg,
> \
>   fatal);
> \
> }
> \
>   }
> \
> }
>
> My updated version:
>
> #define ASAN_MEMORY_ACCESS_CALLBACK_BODY(type, is_write, size, exp_arg,
> fatal) \
> uptr sp = MEM_TO_SHADOW(addr);
> \
> uptr s = size <= SHADOW_GRANULARITY ? *reinterpret_cast(sp)
> \
> : *reinterpret_cast(sp);
> \
> if (UNLIKELY(s)) {
> \
>   if (UNLIKELY(s>= SHADOW_GRANULARITY ||   \
>((s8)((addr & (SHADOW_GRANULARITY - 1)) + size - 1)) >=
> \
>(s8)s)) {
> \
> if (__asan_test_only_reported_buggy_pointer) {
> \
>   *__asan_test_only_reported_buggy_pointer = addr;
> \
> } else {
> \
>   GET_CALLER_PC_BP_SP;
> \
>   ReportGenericError(pc, bp, sp, addr, is_write, size, exp_arg,
> \
>   fatal);
> \
> }
> \
>   }
> \
> }
>
> --
> You received this message because you are subscribed to the Google Groups
> "address-sanitizer" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to address-sanitizer+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Does LLVM Asan with -fsanitize=kernel-address support check glabal and stack buffer issues?

['Dmitry Vyukov' via address-sanitizer]

On Mon, Nov 7, 2016 at 12:06 AM, steven shi  wrote:
> Hi Vyukov,
> Do you mean directly add "-mllvm -debug" flags in clang build option? It
> doesn't work for me. E.g. I build below global buffer overflow example with
> "-mllvm -debug" flags, the clang build fail with "Unknown command". Please
> advise.
>
> https://github.com/google/sanitizers/wiki/AddressSanitizerExampleGlobalOutOfBounds
>
> $ clang AddressSanitizerExampleGlobalOutOfBounds.c -O0 -g -fno-common
> -fsanitize=address -mllvm -asan-stack=1 -mllvm -asan-globals=1 -mllvm
> -asan-instrumentation-with-call-threshold=0 -mllvm -debug
> clang (LLVM option parsing): Unknown command line argument '-debug'.  Try:
> 'clang (LLVM option parsing) -help'
> clang (LLVM option parsing): Did you mean '-help'?
>
> $ clang --version
> clang version 4.0.0 (trunk 281885)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
> InstalledDir: /usr/local/bin


I suspect that if you configure llvm with cmake with
CMAKE_BUILD_TYPE=Debug and LLVM_ENABLE_ASSERTIONS=ON, this flag will
appear. See  
https://github.com/google/sanitizers/wiki/AddressSanitizerHowToBuild
on how to pass these configs.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: kAsan runtime address check

['Dmitry Vyukov' via address-sanitizer]

On Tue, Aug 30, 2016 at 8:42 AM, WookHyun Han  wrote:
> I'm looking address sanitizer instrumentation and found that UseCalls is
> enabled when CompileKernel is enabled
>
> When UseCalls is enabled, instrumentation inserts function calls which check
> address at runtime in front of memory instructions
>
> I just wonder why CompileKernel uses runtime address check,is it related to
> binary size of kernel?


+kasan-dev

Hi,

KASAN has 2 modes: inline instrumentation and outline instrumentation.
Outline instrumentation is present mostly for historical reasons: it
was simpler to implement KASAN with outline instrumentation and
outline instrumentation support appeared in slightly earlier version
of gcc.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: configure scripts, pthread detection and asan

['Dmitry Vyukov' via address-sanitizer]

On Wed, Jan 20, 2016 at 2:22 PM, Hanno Böck  wrote:
> Hi,
>
> I stumbled upon an issue that I'm not sure what the best fix is and
> wanted to ask people here for opinions.
>
> There are a couple of packages that have a check in the configure
> script that will check for the presence of the pthread_create function.
> The check works as far as I understand as follows:
> * check whether pthread_create is available without a flag
> * check whether pthread_create is available with -lpthread
>
> Now here's the problem: libasan contains a pthread_create function.
> Therefore the configure scripts will assume that pthread is available.
> However libasan doesn't contain the full pthread functionality, other
> functions like pthread_mutex_create aren't available.
>
> Therefore:
> * configure script thinks we don't need -lpthread
> * application uses pthread functionality that's not in libasan
> * compilation fails
>
>
> I have first encountered this in sqlite, upstream has now fixed this by
> checking for a different pthread function (patch came from me):
> https://www.sqlite.org/cgi/src/info/9c3a0dc2e8bf202c
>
> Now I wonder - is this a good solution? I know of at least three other
> packages (poppler, openldap, p11-kit) that have the same issue and
> wonder if I should send similar patches or if there's a better / more
> clean solution to this.


FWIW you can add -lpthread whenever you add -fsanitize=address.

-- 
You received this message because you are subscribed to the Google Groups 
"address-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Blacklist file

['Dmitry Vyukov' via address-sanitizer]

Wasn't sure where I need to file the issue due to c.g.c closure. Now filed:
https://code.google.com/p/address-sanitizer/issues/detail?id=404


On Thu, Aug 13, 2015 at 3:03 PM, Ilya Mirsky ilya.mir...@gmail.com wrote:
 I agree, it would be best.
 Is there a ticket I could follow?

 On Wed, Aug 12, 2015 at 4:52 PM, 'Dmitry Vyukov' via thread-sanitizer
 thread-saniti...@googlegroups.com wrote:

 +address-sanitizer

 Thanks for the report!
 I guess it is the same for asan as well.
 I think we need to promote usage of relative paths (absolute paths are
 not portable between machines), and then match then as suffix of
 compiled source files.






 On Wed, Aug 12, 2015 at 1:24 PM, Ilya Mirsky ilya.mir...@gmail.com
 wrote:
  Hi,
 
  I discovered a weird behaviour with a blacklist file- it doesn't work if
  the
  the blacklisted path is absolute and the build string uses relative
  paths,
  or vice versa. Is it a bug, a feature, or am I missing something here?
  The
  following 4 scenarios demonstrate this behaviour.
 
  Clang version: 3.7
  OS: Ubuntu 12.04, Linux kernel 3.5.0-36
 
  Thanks,
  Ilya
 
  1. Relative blacklisting and relative build paths: blacklisting works
 
  mirski@mirski-u12:~/src$ cat demos/tsan/blacklist.tsan
  src:demos/tsan/*
 
  mirski@mirski-u12:~/src$ clang -g -fsanitize=thread
  -fsanitize-blacklist=demos/tsan/blacklist.tsan demos/tsan/race.c
  mirski@mirski-u12:~/src$ TSAN_OPTIONS= ./a.out
  Global=42
  mirski@mirski-u12:~/src$
 
  2. Relative blacklisting and absolute build paths: blacklisting doesn't
  work
 
  mirski@mirski-u12:~/src$ cat demos/tsan/blacklist.tsan
  src:demos/tsan/*
 
  mirski@mirski-u12:~/src$ clang -g -fsanitize=thread
  -fsanitize-blacklist=demos/tsan/blacklist.tsan
  /home/mirski/src/demos/tsan/race.c
  mirski@mirski-u12:~/src$ TSAN_OPTIONS= ./a.out
  ==
  WARNING: ThreadSanitizer: data race (pid=31138)
Read of size 4 at 0x014ad4e0 by main thread:
  #0 main /home/mirski/src/demos/tsan/race.c:22:27
  (a.out+0x004a2eb8)
 
Previous write of size 4 at 0x014ad4e0 by thread T1:
  #0 Thread1 /home/mirski/src/demos/tsan/race.c:8:12
  (a.out+0x004a2e11)
 
Location is global 'Global' of size 4 at 0x014ad4e0
  (a.out+0x014ad4e0)
 
Thread T1 (tid=31140, running) created by main thread at:
  #0 pthread_create
 
  /export/local1/mirski/llvm/release_37/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:849
  (a.out+0x00448c83)
  #1 main /home/mirski/src/demos/tsan/race.c:16:5
  (a.out+0x004a2e78)
 
  SUMMARY: ThreadSanitizer: data race
  /home/mirski/src/demos/tsan/race.c:22:27
  in main
  ==
  Global=42
  ThreadSanitizer: reported 1 warnings
 
  3. Absolute blacklisting and absolute build paths: blacklisting works
 
  mirski@mirski-u12:~/src$ cat /home/mirski/src/demos/tsan/blacklist.tsan
  src:/home/mirski/src/demos/tsan/*
 
  mirski@mirski-u12:~/src$ clang -g -fsanitize=thread
  -fsanitize-blacklist=demos/tsan/blacklist.tsan
  /home/mirski/src/demos/tsan/race.c
  mirski@mirski-u12:~/src$ TSAN_OPTIONS= ./a.out
  Global=42
 
  4. Absolute blacklisting and relative build paths: blacklisting doesn't
  work
 
  mirski@mirski-u12:~/src$ cat /home/mirski/src/demos/tsan/blacklist.tsan
  src:/home/mirski/src/demos/tsan/*
 
  mirski@mirski-u12:~/src$ clang -g -fsanitize=thread
  -fsanitize-blacklist=demos/tsan/blacklist.tsan demos/tsan/race.c
  mirski@mirski-u12:~/src$ TSAN_OPTIONS= ./a.out
  ==
  WARNING: ThreadSanitizer: data race (pid=32632)
Read of size 4 at 0x014ad4e0 by main thread:
  #0 main /home/mirski/src/demos/tsan/race.c:22:27
  (a.out+0x004a2eb8)
 
Previous write of size 4 at 0x014ad4e0 by thread T1:
  #0 Thread1 /home/mirski/src/demos/tsan/race.c:8:12
  (a.out+0x004a2e11)
 
Location is global 'Global' of size 4 at 0x014ad4e0
  (a.out+0x014ad4e0)
 
Thread T1 (tid=32634, running) created by main thread at:
  #0 pthread_create
 
  /export/local1/mirski/llvm/release_37/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:849
  (a.out+0x00448c83)
  #1 main /home/mirski/src/demos/tsan/race.c:16:5
  (a.out+0x004a2e78)
 
  SUMMARY: ThreadSanitizer: data race
  /home/mirski/src/demos/tsan/race.c:22:27
  in main
  ==
  Global=42
  ThreadSanitizer: reported 1 warnings
 
  --
  You received this message because you are subscribed to the Google
  Groups
  thread-sanitizer group.
  To unsubscribe from this group and stop receiving emails from it, send
  an
  email to thread-sanitizer+unsubscr...@googlegroups.com.
  For more options, visit https://groups.google.com/d/optout.

 --
 You received this message because you are subscribed to the Google Groups
 thread-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to thread-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


 --
 You received 

Re: Blacklist file

['Dmitry Vyukov' via address-sanitizer]

+address-sanitizer

Thanks for the report!
I guess it is the same for asan as well.
I think we need to promote usage of relative paths (absolute paths are
not portable between machines), and then match then as suffix of
compiled source files.






On Wed, Aug 12, 2015 at 1:24 PM, Ilya Mirsky ilya.mir...@gmail.com wrote:
 Hi,

 I discovered a weird behaviour with a blacklist file- it doesn't work if the
 the blacklisted path is absolute and the build string uses relative paths,
 or vice versa. Is it a bug, a feature, or am I missing something here? The
 following 4 scenarios demonstrate this behaviour.

 Clang version: 3.7
 OS: Ubuntu 12.04, Linux kernel 3.5.0-36

 Thanks,
 Ilya

 1. Relative blacklisting and relative build paths: blacklisting works

 mirski@mirski-u12:~/src$ cat demos/tsan/blacklist.tsan
 src:demos/tsan/*

 mirski@mirski-u12:~/src$ clang -g -fsanitize=thread
 -fsanitize-blacklist=demos/tsan/blacklist.tsan demos/tsan/race.c
 mirski@mirski-u12:~/src$ TSAN_OPTIONS= ./a.out
 Global=42
 mirski@mirski-u12:~/src$

 2. Relative blacklisting and absolute build paths: blacklisting doesn't work

 mirski@mirski-u12:~/src$ cat demos/tsan/blacklist.tsan
 src:demos/tsan/*

 mirski@mirski-u12:~/src$ clang -g -fsanitize=thread
 -fsanitize-blacklist=demos/tsan/blacklist.tsan
 /home/mirski/src/demos/tsan/race.c
 mirski@mirski-u12:~/src$ TSAN_OPTIONS= ./a.out
 ==
 WARNING: ThreadSanitizer: data race (pid=31138)
   Read of size 4 at 0x014ad4e0 by main thread:
 #0 main /home/mirski/src/demos/tsan/race.c:22:27 (a.out+0x004a2eb8)

   Previous write of size 4 at 0x014ad4e0 by thread T1:
 #0 Thread1 /home/mirski/src/demos/tsan/race.c:8:12
 (a.out+0x004a2e11)

   Location is global 'Global' of size 4 at 0x014ad4e0
 (a.out+0x014ad4e0)

   Thread T1 (tid=31140, running) created by main thread at:
 #0 pthread_create
 /export/local1/mirski/llvm/release_37/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:849
 (a.out+0x00448c83)
 #1 main /home/mirski/src/demos/tsan/race.c:16:5 (a.out+0x004a2e78)

 SUMMARY: ThreadSanitizer: data race /home/mirski/src/demos/tsan/race.c:22:27
 in main
 ==
 Global=42
 ThreadSanitizer: reported 1 warnings

 3. Absolute blacklisting and absolute build paths: blacklisting works

 mirski@mirski-u12:~/src$ cat /home/mirski/src/demos/tsan/blacklist.tsan
 src:/home/mirski/src/demos/tsan/*

 mirski@mirski-u12:~/src$ clang -g -fsanitize=thread
 -fsanitize-blacklist=demos/tsan/blacklist.tsan
 /home/mirski/src/demos/tsan/race.c
 mirski@mirski-u12:~/src$ TSAN_OPTIONS= ./a.out
 Global=42

 4. Absolute blacklisting and relative build paths: blacklisting doesn't work

 mirski@mirski-u12:~/src$ cat /home/mirski/src/demos/tsan/blacklist.tsan
 src:/home/mirski/src/demos/tsan/*

 mirski@mirski-u12:~/src$ clang -g -fsanitize=thread
 -fsanitize-blacklist=demos/tsan/blacklist.tsan demos/tsan/race.c
 mirski@mirski-u12:~/src$ TSAN_OPTIONS= ./a.out
 ==
 WARNING: ThreadSanitizer: data race (pid=32632)
   Read of size 4 at 0x014ad4e0 by main thread:
 #0 main /home/mirski/src/demos/tsan/race.c:22:27 (a.out+0x004a2eb8)

   Previous write of size 4 at 0x014ad4e0 by thread T1:
 #0 Thread1 /home/mirski/src/demos/tsan/race.c:8:12
 (a.out+0x004a2e11)

   Location is global 'Global' of size 4 at 0x014ad4e0
 (a.out+0x014ad4e0)

   Thread T1 (tid=32634, running) created by main thread at:
 #0 pthread_create
 /export/local1/mirski/llvm/release_37/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:849
 (a.out+0x00448c83)
 #1 main /home/mirski/src/demos/tsan/race.c:16:5 (a.out+0x004a2e78)

 SUMMARY: ThreadSanitizer: data race /home/mirski/src/demos/tsan/race.c:22:27
 in main
 ==
 Global=42
 ThreadSanitizer: reported 1 warnings

 --
 You received this message because you are subscribed to the Google Groups
 thread-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to thread-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: stack-buffer-overflow on addressable memory

['Dmitry Vyukov' via address-sanitizer]

On Mon, Mar 2, 2015 at 12:44 PM, Martin Ri mric...@googlemail.com wrote:
 Hi,

 has anyone seen reports like the one below? A stack-buffer-overflow is
 reported, but the memory is addressable.
 Notes: I have seen similar instances reported as unknown-crash. Looking at
 the code in asan-report.cc, this is the initial error code, and the real
 error is determined by looking at the shadow. So it seems this could be some
 data race: When the shadow is initially checked as part of the instrumented
 code, the memory is marked as invalid and error reporting is triggered. But
 shortly afterwards it gets marked as valid. However, I can't imagine how
 that could happen: The object lives on the stack of thread 0, this thread is
 still running, and the frame should still be valid (which is also what ASan
 reports). The reported offset 44 also matches the accessed object in
 BinarySemaphore::open().

 The involved code uses futexes, and I suspect this somehow causes the
 problem. But I'm not sure whether it is a problem of the codebase (at least
 we haven't seen anything like it in practice), or a limitation/bug of ASan.

 ==39533==ERROR: AddressSanitizer: stack-buffer-overflow on address
 0x7fff656ad0ec at pc 0x7f1a103a4d6b bp 0x7f17e0181e10 sp 0x7f17e0181e08
 READ of size 4 at 0x7fff656ad0ec thread T182 (Pool)
 #0 0x7f1a103a4d6a in
 Synchronization::BinarySemaphore::open(Execution::Context)
 #1 0x7f1a5fc64e25 in TrexSync::Event::signal()
 #2 0x7f1a47ecbdf3 in TrexSync::EventGuard::signal()
 #3 0x7f1a47ec83c2 in TrexThreads::PoolThread::run()
 #4 0x7f1a47ec6a12 in TrexThreads::PoolThread::run(void*)
 #5 0x7f1a100a8a80 in Execution::Thread::staticMainImp(void**)
 #6 0x7f1a100b37ff in Execution::Thread::staticMain(void*)

 Address 0x7fff656ad0ec is located in stack of thread T0 at offset 44 in
 frame
 #0 0x7f1a47ed09df in
 TrexThreads::ThreadManager::create(TrexThreads::Thread*, void*,
 TrexThreads::_CreateFlags, int)

   This frame has 1 object(s):
 [32, 160) 'aCreationInfo' == Memory access at offset 44 is inside this
 variable
 HINT: this may be a false positive if your program uses some custom stack
 unwind mechanism or swapcontext
   (longjmp and C++ exceptions *are* supported)
 SUMMARY: AddressSanitizer: stack-buffer-overflow
 Shadow bytes around the buggy address:
   0x10006cacd9c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x10006cacd9d0: 00 00 00 00 f1 f1 f1 f1 04 f2 04 f2 04 f2 04 f2
   0x10006cacd9e0: 04 f2 04 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2
   0x10006cacd9f0: 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2
   0x10006cacda00: 04 f2 04 f2 00 f3 f3 f3 00 00 00 00 00 00 00 00
 =0x10006cacda10: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00[00]00 00
   0x10006cacda20: 00 00 00 00 00 00 00 00 00 00 00 00 f3 f3 f3 f3
   0x10006cacda30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   0x10006cacda40: 00 00 00 00 f1 f1 f1 f1 04 f2 04 f2 04 f2 04 f2
   0x10006cacda50: 04 f2 00 00 f2 f2 00 00 f3 f3 f3 f3 00 00 00 00
   0x10006cacda60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 Shadow byte legend (one shadow byte represents 8 application bytes):
   Addressable:   00


It can be a racy use-after-return. When asan detects the bug the
shadow is poisoned, but when it reports it, another thread already
unpoisoned the memory.
Since the stack trace contains words like Thread and BinarySemaphore,
I would look for a data race first. Try to run under ThreadSanitizer
(-fsanitize=thread).

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Not exporting __interceptor_* symbols?

['Dmitry Vyukov' via address-sanitizer]

On Thu, Jan 22, 2015 at 1:01 PM, 'Evgeniy Stepanov' via
address-sanitizer address-sanitizer@googlegroups.com wrote:
 Hi,

 at the moment we add __interceptor_xxx (for all intercepted functions)
 to the runtime library dynamic list, along with xxx themselves. This
   (a) seems to confuse the symbolizer sometimes (it would say that the
 memory was allocated in __interceptor_malloc which is suboptimal)
   (b) slows down program startup (I don't have any numbers though)

 I don't see why we need to export __interceptor_xxx. It does not look
 like they are called by that name anywhere. Disabling this (a
 one-liner in gen_dynamic_list.py) does not break any tests.

 Am I missing anything?


We do call interceptors by that name in few places in our internal code base.

Also, if you remove them, then symbolizer won't be able to print
anything at all, which does not look better.
Btw, tsan strips __interceptor_ prefixes. Which is actually another
use of __interceptor_ symbols.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Not exporting __interceptor_* symbols?

['Dmitry Vyukov' via address-sanitizer]

On Thu, Jan 22, 2015 at 1:39 PM, 'Evgeniy Stepanov' via
address-sanitizer address-sanitizer@googlegroups.com wrote:
 On Thu, Jan 22, 2015 at 1:12 PM, 'Dmitry Vyukov' via address-sanitizer
 address-sanitizer@googlegroups.com wrote:
 On Thu, Jan 22, 2015 at 1:01 PM, 'Evgeniy Stepanov' via
 address-sanitizer address-sanitizer@googlegroups.com wrote:
 Hi,

 at the moment we add __interceptor_xxx (for all intercepted functions)
 to the runtime library dynamic list, along with xxx themselves. This
   (a) seems to confuse the symbolizer sometimes (it would say that the
 memory was allocated in __interceptor_malloc which is suboptimal)
   (b) slows down program startup (I don't have any numbers though)

 I don't see why we need to export __interceptor_xxx. It does not look
 like they are called by that name anywhere. Disabling this (a
 one-liner in gen_dynamic_list.py) does not break any tests.

 Am I missing anything?


 We do call interceptors by that name in few places in our internal code base.

 Yeah, sometimes we do need to know that we are calling the
 interceptor, and not the libc function.


 Also, if you remove them, then symbolizer won't be able to print
 anything at all, which does not look better.

 No, we have 2 exported symbols: xxx and __interceptor_xxx, the
 proposal was to remove the second one.


User can override any interceptor. Then the only name of an
interceptor is __interceptor_xxx.


 Btw, tsan strips __interceptor_ prefixes. Which is actually another
 use of __interceptor_ symbols.

 --
 You received this message because you are subscribed to the Google Groups 
 address-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to address-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

 --
 You received this message because you are subscribed to the Google Groups 
 address-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to address-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: kSpaceBeg for tsan

['Dmitry Vyukov' via address-sanitizer]

On Tue, Jan 20, 2015 at 2:35 PM, Venkataramanan Kumar
venkataramanan.ku...@linaro.org wrote:
 Hi all,

 While trying to use tsan on Aarch64, I am getting mmap failure.

 __sanitizer::internal_mmap (addr=0x7d00, length=1099511627776, prot=0,
 flags=16434, fd=-1, offset=0)

  p syscall(222, 0x7d, 1099511627776, 0, 16434, -1,0,0)
 $34 = -1


 // SizeClassAllocator64 -- allocator for 64-bit address space.
 //
 // Space: a portion of address space of kSpaceSize bytes starting at
 // a fixed address (kSpaceBeg). Both constants are powers of two and
 // kSpaceBeg is kSpaceSize-aligned.
 // At the beginning the entire space is mprotect-ed, then small parts of it
 // are mapped on demand.
 //


 The kSpaceBeg seem to be  0x7d by default and kSpaceSize seem
 to be 1099511627776.

 I am trying to understand where these default values are set?

+thread-saniti...@googlegroups.com
address-sanitizer@googlegroups.com to bcc

Hi Venkat,

All memory ranges are specified here:

http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/tsan/rtl/tsan_platform.h?view=markup

In particular, you are looking for kHeapMemBeg and kHeapMemEnd constants.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Off list - TSAN internal_sys calls

['Dmitry Vyukov' via address-sanitizer]

On Mon, Jan 19, 2015 at 11:17 AM, Venkataramanan Kumar
venkataramanan.ku...@linaro.org wrote:
 Hi  Dmitry,

 I am seeing some  internal_syscalls like internel_clone, internal_mmap
 written for x86_64 and I am trying to duplicate them for Aarch64.

 Have you been seeing this done for  MIPS port in LLVM?

 regards,
 Venkat,


+address-sanitizer group

Hi Venkat,

I am sure that asan works on several arches other than x86_64. So
there must be some support for such ports.
Can't find internel_clone function, where is it defined?

For simpler syscalls we do:

#include sys/syscall.h

uptr internal_read(fd_t fd, void *buf, uptr count) {
  sptr res;
  HANDLE_EINTR(res, (sptr)internal_syscall(SYSCALL(read), fd, (uptr)buf,
   count));
  return res;
}

I think it should work for aarch64 as well.

Do you see any concrete issues, compilation errors or difficulties with porting?

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Keep global variables metadata in redzone

['Dmitry Vyukov' via address-sanitizer]

What are the savings? We remove the metadata but add 32 bytes to
redzones. Seems roughly equivalent.

The bss is a good point. It can be unacceptable to lots of
applications that has multi-megabyte objects in bss. Which is I would
say very frequent.


On Wed, Dec 24, 2014 at 11:41 AM, Yury Gribov y.gri...@samsung.com wrote:
 Hi all,

 Has anyone considered moving global variables metadata (struct
 __asan_global) to redzone? I expect this to save quite some RAM.

 This would require us to
 (1) extend redzone on x64 from 32 to 64 bytes (sizeof(__asan_global) ==
 sizeof(uptr) * 7 == 56 bytes)
 (2) add one more pointer 'next' to keep address of next __asan_global in
 current module
 (3) remove pointer to module name from __asan_global and keep it at the end
 of linked list instead

 Note that (1) will only be needed for globals which are larger than 4 bytes
 (we'll be able to reuse 32-byte padding). We could squeeze up even more
 space by moving has_dynamic_init bit to lower bits of
 __asan_global.size_with_redzone.

 Best regards,
 Yury

 --
 You received this message because you are subscribed to the Google Groups
 address-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to address-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Selectively disable AddressSanitizer for some memory regions

['Dmitry Vyukov' via address-sanitizer]

On Tue, Dec 23, 2014 at 1:25 PM, Jonas Wagner jonas.wag...@epfl.ch wrote:
 Hi,

 First of all, why would you want to do such thing?
 Do you have a particular use case?


 It's a justified question. We do have use cases for this. For example,
 consider a library that is formally proven to be free from memory errors. We
 can already build this library without ASan, so that it won't have checks.
 Yet it is still linked with the ASan memory allocator, so its objects have
 red zones, compete for quarantine space, etc.

Memory-safety is not a property of a library.
Is libc's memcpy safe? Maybe yes, maybe no, depending on the pointers
passed from outside.
Is std::string safe? Yes. Unless c_str() pointer is leaked to an
unsafe C function.

A library can be safe iff it does not accept any pointers from outside
nor leak any pointer to outside. I have not seen any such libraries.
Can you provide any examples? For now it looks like a practically
uninteresting case to support.



 There are other questions to clarify, like how to separate the unsafe memory
 space from the regular program memory. But I think we could solve these in
 time when they arrive. For now, I'm looking mostly for ideas about how
 unprotected objects could coexist with ASan.

 Cheers,
 Jonas

 --
 You received this message because you are subscribed to the Google Groups
 address-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to address-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Selectively disable AddressSanitizer for some memory regions

['Dmitry Vyukov' via address-sanitizer]

OK, if it is not we who are supporting it :)

Yes, you can create a separate instance of memory allocator for unsafe
objects. Disable addition of redzones for such unsafe allocations.
Asan allocator occupies a continuous region of virtual memory. So in
free you can just look at address value to understand that it comes
from the unsafe allocator (no need for any flags), in such case you
just free it directly to the unsafe allocator. Looks pretty simple.
Large allocations are handled slightly differently -- there is no
continuous region of virtual memory for them. So probably you can just
disable addition of redzones, poisoning and quarantine for them. But
otherwise use the same large object allocator. This will require a
header to store the safe/unsafe flag for large objects.




On Tue, Dec 23, 2014 at 2:30 PM, Jonas Wagner jonas.wag...@epfl.ch wrote:
 Hi,

 Memory-safety is not a property of a library.


 I can see your concerns. They are valid, but I still like the idea of being
 selective. For the moment, just believe me that I do have use cases :)

 For now it looks like a practically
 uninteresting case to support.


 Note that I'm not requesting a new ASan feature that would need support.
 Instead, I have a (crazy, maybe) idea of something that could be built using
 ASan. I don't know yet how to build it... But you, the ASan experts,  have
 valuable knowledge about this, and I'd love to hear your thoughts.

 Cheers,
 Jonas

 --
 You received this message because you are subscribed to the Google Groups
 address-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to address-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [PINGv2][PATCH] Ignore alignment by option

['Dmitry Vyukov' via address-sanitizer]

On Thu, Dec 4, 2014 at 4:48 PM, Yury Gribov y.gri...@samsung.com wrote:
 On 12/04/2014 03:47 PM, Dmitry Vyukov wrote:

 size_in_bytes = -1 instrumentation is too slow to be the default in
 kernel.

 If we want to pursue this, I propose a different scheme.
 Handle 8+ byte accesses as 1/2/4 accesses. No changes to 1/2/4 access
 handling.
 Currently when we allocate, say, 17-byte object we store 0 0 1 into
 shadow. An 8-byte access starting at offset 15 won't be detected,
 because the corresponding shadow value is 0. Instead we start storing
 0 9 1 into shadow. Then the first shadow != 0 check will fail, and the
 precise size check will catch the OOB access.
 Make this scheme the default for kernel (no additional flags).

 This scheme has the following advantages:
 - load shadow only once (as opposed to the current size_in_bytes = -1
 check that loads shadow twice)
 - less code in instrumentation
 - accesses to beginning and middle of the object are not slowed down
 (shadow still contains 0, so fast-path works); only accesses to the
 very last bytes of the object are penalized.


 Makes sense.  The scheme actually looks bullet-proof, why Asan team
 preferred current (fast but imprecise) algorithm?

 BTW I think we'll want this option in userspace so well so we'll probably
 need to update libasan.


We've discussed this scheme, but nobody has shown that it's important enough.
It bloats binary (we do have issues with binary sizes) and slows down
execution a bit. And if it is non-default mode, then it adds more
flags (which is bad) and adds more configurations to test.

For this to happen somebody needs to do research on (1) binary size
increase, (2) slowdown, (3) number of additional bugs it finds (we can
run it over extensive code base that is currently asan-clean).

Here is the issue with some notes:
https://code.google.com/p/address-sanitizer/issues/detail?id=100

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: AddressSanitizer's allocator

['Dmitry Vyukov' via address-sanitizer]

On Thu, Sep 25, 2014 at 7:44 AM, Jonas Wagner jonas.wag...@epfl.ch wrote:
 Dear AddressSanitizer developers,

 I'm thinking about ways to optimize the performance of ASan's allocator.
 There are a few benchmarks where a large fraction of the overhead comes from
 the allocator and the quarantine queue, rather than the checks themselves
 (e.g., gcc from SPEC2006).

 When I looked at the allocator, I was surprised that it is implemented
 inside ASan's runtime library (or rather, in sanitizer_common). This is
 unlike other intercepted functions such as strcpy, which forward to the
 implementation from libc. What is the reason for this?

 Would it be possible to implement asan_malloc as a decorator on top of libc
 malloc? Or on top of an existing implementation such as tcmalloc? This seems
 desirable to me because these are highly tuned. It might also simplify the
 sanitizer codebase.

 I'm sure this case has been considered. What are the reasons for the current
 design?

 Besides this question, I wonder if there are other ways of optimizing the
 allocator or the quarantine mechanism. If you can think of any (relatively)
 low-hanging fruit, I'd be motivated to give it a try.


Hi Jonas,

When I measured performance of sanitizer allocator against tcmalloc on
a large real application that calls malloc a lot, sanitizer allocator
was 10% faster. You are free to do you own measurements. Maybe you
will discover some inefficiency in the allocator that you can fix.

One historical reason for using own allocator is that we needed the
meta information that is not covered by shadow, so it can't be a
simple prefix/postfix of the memory block itself. We don't use the
meta info anymore.

Even if we switch to tcmalloc, we still need quarantine on top of it.
And quarantine kills cache locality. You remove that overhead simply
by switching to a different allocator.

One of the low hanging fruits is disabling allocator stats for asan.
The stats are required for tsan, but asan keeps own stats. So it's
possible to implement a NoOpStats class for asan, and parametrize the
allocator with that class. I don't know of any other low hanging
fruits.

Also note that the allocator must scale well for high core counts.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: AddressSanitizer and issue suppression

['Dmitry Vyukov' via address-sanitizer]

+tetra2005


On Wed, Sep 24, 2014 at 4:31 PM, Kuba Brecka kuba.bre...@gmail.com wrote:
 Hi,

 I'm trying to explore the possibilities of extending ASan to be able to
 continue execution after an error is found and a report printed out. I
 understand that the fact that ASan is currently aborting/exiting on a report
 is totally intentional and that there are some good reasons for it, e.g.
 that it's not safe to continue because the memory is corrupted, or that the
 UnreachableInst/doesNotReturn play an important role for optimizations.

 However, I believe that there may be some valid reasons to allow continuing
 program execution, like when there's a bug in a system library. This can
 easily happen even when this library itself is not instrumented, due to
 wrappers/interceptors affecting system libraries as well. So doing something
 like...

 a = malloc(15);
 memset(a, 0, 16);

 ...in a system library would get caught by ASan, and it's definitely a bug.
 On the other hand, in this specific case, without ASan, this alone would
 (almost certainly) never crash or cause *real* memory corruption, since
 malloc allocates 16 bytes anyway. We want to learn about the bug (to be able
 to fix it), but I think we also want to be able to continue using ASan
 before a new version of the library is shipped with an OS update.

 I am mainly interested in wrappers/interceptors only, because the reports
 invoked by instrumentation cannot happen in a library function (since it's
 not instrumented) and if a bug in a library triggers a report to be produced
 in user's instrumented code, he can blacklist that function or use
 __attribute__((no_sanitize_address)). It seems to me it should be possible
 to modify the error reporting (in the wrappers only) not to be fatal, and if
 that decision whether to abort or not (let's say via a suppression list) is
 done only in the case when a poisoned memory access is detected, it
 shouldn't have any significant performance hit.

 I noticed that there is an issue suppression mechanism in ThreadSanitizer
 and I'm aware that the circumstances for this feature in TSan are different.
 However, I'd still like to discuss the possibilities and opinions on this
 topic.

 Thank you for any feedback!

 Kuba

 --
 You received this message because you are subscribed to the Google Groups
 address-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to address-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: GNU coreutils addition to found bugs?

['Dmitry Vyukov' via address-sanitizer]

done, thanks
https://code.google.com/p/address-sanitizer/wiki/FoundBugs?ts=1407754815updated=FoundBugs#GNU_Coreutils


On Sun, Aug 10, 2014 at 3:49 PM, Pádraig Brady pixelb...@gmail.com wrote:
 I wonder could
 http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commit;h=f940fece
 be added to https://code.google.com/p/address-sanitizer/wiki/FoundBugs

 --
 You received this message because you are subscribed to the Google Groups
 address-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to address-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Asan working on a simple test but not in a full project

['Dmitry Vyukov' via address-sanitizer]

You are welcome!


On Wed, Jul 23, 2014 at 11:46 AM, Jonathan
mkvtoolnix.build.jon...@gmail.com wrote:


 Le mardi 22 juillet 2014 23:15:07 UTC+2, Jonathan a écrit :



 Le mardi 22 juillet 2014 21:25:05 UTC+2, Dmitry Vyukov a écrit :

 On Tue, Jul 22, 2014 at 11:10 PM, Jonathan
 mkvtoolnix@gmail.com wrote:
  Hello,
 
  I'm currently experiencing something strange and I can't explain it.
  I've compliled a ToT (approx. of today) LLMV/Clang/compiler-rt using
  libcxx
   libcxxabi on Linux (Debian 6.0) the shipped compiler is GCC 4.4 (so
  no
  C++11 support). All compilations are done in 32-bit.
 
  When I run the target check-asan and all I only get 2 failed test
  (something
  about the size of ThreadSize being bigger) so I assume the compiler 
  asan
  are in a working state.
 
  So I made a litte test (simple main with char[] on the stack and
  strcpy() a
  bigger array in it) as expected I get an Asan report.
 
  But if I copy the same code in my bigger project, there is no report
  from
  Asan but I do get in another tool of the project report about ODR
  violations.
  If I set verbosity in ASAN_OPTIONS it loads correctly and I get a few
  warning that the lib is not being able to intercept __c99_printf and a
  few
  other functions that are basicaly related to printf.
 
  If I use nm on one of my binary I can see the __asan_init  other
  asan
  functions.
 
  So is there a way to check if the code is instrumented for asan ? What
  could
  be the problem here ?
 
  Thanks in advance for your help,
 


 It's possible that your buggy code is so simple that compiler
 optimizes it away. E.g. if your projects is compiled with -O2, but
 your test is -O0. Try to add some volatiles and prints.



 Will try that but the project is compiled with -O1. Also with an older
 clang+asan I know there is a problem in a third library that isn't reported
 with version. (about overlap memory with memcpy() )


 After retrying with printf() it works as expected, so the code was optimized
 away as you said, sorry for the trouble.
 Thanks for the help.

 --
 You received this message because you are subscribed to the Google Groups
 address-sanitizer group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to address-sanitizer+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Reverse poisoning

['Dmitry Vyukov' via address-sanitizer]

On Wed, Jul 16, 2014 at 7:50 PM, Konstantin Serebryany
konstantin.s.serebry...@gmail.com wrote:



 On Wed, Jul 16, 2014 at 7:42 PM, Tomek Bury tomek.b...@gmail.com wrote:

 Hi all,

 The default mode for asan is to start clean and temporarily poison some
 areas. Wouldn't the opposite be more robust? I mean, start with entire
 memory poisoned and temporarily un-poison small bits of memory for their
 life time.


 There are pieces of memory that come from outside of asan's control (e.g.
 memory mmaped via a pure mmap syscall) -- they have to be clean.
 If we can be sure that we control all allocated memory -- then yes, there is
 such option.


Also I am not sure what it will give us. For malloc it already works
in the best possible way. For other memory it will only verify asan's
guessing of what memory is OK to access. This can become a constant
pain point w/o any benefits.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Reverse poisoning

['Dmitry Vyukov' via address-sanitizer]

On Wed, Jul 16, 2014 at 9:36 PM, Yuri Gribov tetra2...@gmail.com wrote:
 On Wed, Jul 16, 2014 at 7:59 PM, 'Dmitry Vyukov' via address-sanitizer
 address-sanitizer@googlegroups.com wrote:
 Also I am not sure what it will give us. For malloc it already works
 in the best possible way. For other memory it will only verify asan's
 guessing of what memory is OK to access. This can become a constant
 pain point w/o any benefits.

 My colleagues from kernel team pointed out that forced poisoning of
 stack frames could serve as a poor man's UAR detector (especially if
 frames could be interspersed with randomly sized poisoned areas).

This is somewhat harder to do in user-space, because we don't
necessary know when a stack is mapped, when it's unmapped and what's
its size (on linux, windows and mac).

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Stack instrumentation for kernel.

['Dmitry Vyukov' via address-sanitizer]

On Tue, May 27, 2014 at 1:37 PM, Yuri Gribov tetra2...@gmail.com wrote:
 On Mon, May 26, 2014 at 7:57 PM, 'Dmitry Vyukov' via address-sanitizer
 address-sanitizer@googlegroups.com wrote:
 What exactly do you want to write? We need to coordinate.

 From compiler's side we'll probably start experimenting with stack
 instrumentation. Provided that we stick with function calls, I like
 the frame metadata approach most (i.e. insert calls to
 __kasan_instrument_stack/__kasan_uninstrument_stack at start/end of
 function).

Hi Yuri,

Regarding your instrumentation patch in the gcc mailing list.
I was thinking about evolution of kasan instrumentation in gcc. Let's
submit it as is for now to reduce traction in the gcc community. But
sooner or later we will want inline instrumentation for kasan, and it
won't work with this -fsanitize=address
-asan-instrumentation-with-call-threshold=-1 scheme (the compiler
won't be able to understand what type of instrumentation user wants).
Also we now have -fsanitize=kernel-address in our makefiles, and we
will need to replace it with -fsanitize=address
-asan-instrumentation-with-call-threshold=-1; and later when we have
inline instrumentation we will need to replace it once again.

So I propose to add -fsanitize=kernel-address when the first patch is
committed. Now it will just enable -fsanitize=address
-asan-instrumentation-with-call-threshold=-1 internally. But later we
will be able to change instrumentation for kernel under the hood w/o
disturbing users.

As for __kasan vs __asan prefixes, I don't care too much. __asan will
work for us.

Sounds good?

Also, does gcc asan pass emit ctors into every translation unit? It
was not working for us a year ago in kernel. Do you plan to disable
them in kernel instrumentation mode?

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Stack instrumentation for kernel.

['Dmitry Vyukov' via address-sanitizer]

On Fri, Jun 6, 2014 at 8:29 PM, Yuri Gribov tetra2...@gmail.com wrote:
 Dmitry,

 I propose to add -fsanitize=kernel-address when the first patch is
 committed. Now it will just enable -fsanitize=address
 -asan-instrumentation-with-call-threshold=-1 internally. But later we
 will be able to change instrumentation for kernel under the hood w/o
 disturbing users.

 Yup, sounds right. Note that this will (unfortunately) be
 --param asan-instrumentation-with-call-threshold=0 (not -1)
 because GCC developers don't want to allow negative parameters.

If it all happens under the covers of -fsanitize=kernel-address, it
does not matter much, right?

 As for __kasan vs __asan prefixes, I don't care too much.
 __asan will work for us.

 Ok. Or we could throw in a some __attribute__((alias))'es.

We can (and probably will) throw them temporary during transition. But
I don't want to leave them forever. It looks ugly.

 Also, does gcc asan pass emit ctors into every translation unit? It
 was not working for us a year ago in kernel. Do you plan to disable
 them in kernel instrumentation mode?

 We had some discussion with Andrey but haven't yet fleshed anything out.
 For now we will disable them with --param asan-globals=0.


Then this is another reason to introduce -fsanitize=kernel-address
earlier rather than later.

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Stack instrumentation for kernel.

['Dmitry Vyukov' via address-sanitizer]

On Mon, May 26, 2014 at 2:18 PM, Yuri Gribov tetra2...@gmail.com wrote:
 Kostya,

 I'm wondering, have you thought about implementing stack instrumentation for
 linux kernel?
 Of course we did! This is in our near term plans.

 When do you think we can expect this feature?

Difficult to say. Several months maybe.

 We first need to prepare a reasonable patch to GCC that implements
 kasan instrumentation (in progress).

 Could you provide some details about Kasan instrumentation plans? Are
 you going to keep function calls or do inline instrumentation similar
 to userspace Asan?


Eventually we want to switch to inlined instrumentation similar to
user-space asan.
But we need to figure out how it will look like, it's much easier to
experiment with runtime calls.
The instrumentation is not the hotspot ATM.


 Then we are going to extend it to support stack overflow instrumentation.

 Can we help on compiler/kernel side?


We will be sending gcc patches soon. Review and +1's are welcome.

Regarding kernel, we need to put together a TODO list -- there is lots of work.
Do you able to build kernel with asan and boot it? Just verifying it
and pointing any weak places in our docs would be a good start.
Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
address-sanitizer group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to address-sanitizer+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.