Re: WITH_ASAN= vs. vfork: ASAN can not deal with vfork in a reliable manor, so what to do to avoid vfork fairly generally, including for kyua?
On 2022-Jan-15, at 15:25, Mark Millard wrote:
> ASAN is documented to not be able to deal reliably with vfork use
> (inaccurate results, result mixing interpretations of the old
> and new contexts, and so on). [There may be other routines
> sufficiently analogous to vfork to have the same sorts of issues.
> I'll write only of vfork explicitly.]
>
> It turns out that using SH_DISABLE_VFORK= with kyua runs
> helps avoid a bunch of junk failure ASAN reports that involve
> /bin/sh using vfork. So I've been recently using the likes of:
>
> env SH_DISABLE_VFORK= ASAN_OPTIONS=detect_container_overflow=0 \
> kyua test -k /usr/tests/Kyuafile
>
> in my kyua runs. But that need not cover all the vfork use in
> the kyua runs --or in general system operation.
>
>
>
> There is more that could be done. For example . . .
>
> contrib/llvm-project/compiler-rt/lib/asan/asan_interceptors.h
> currently has:
>
> #if SANITIZER_LINUX &&\
>(defined(__arm__) || defined(__aarch64__) || defined(__i386__) || \
> defined(__x86_64__) || SANITIZER_RISCV64)
> # define ASAN_INTERCEPT_VFORK 1
> #else
> # define ASAN_INTERCEPT_VFORK 0
> #endif
>
> The "# define ASAN_INTERCEPT_VFORK 1" causes interception to
> use fork for the vfork implementation, instead of an actual
> vfork.
>
> It looks to me like asan_interceptors.h for FreeBSD should
> also be using:
>
> # define ASAN_INTERCEPT_VFORK 1
>
> but it currently is not.
Well, looking around there does not seem to be a FreeBSD-supporting
implementation to go with use of "# define ASAN_INTERCEPT_VFORK 1".
So there would be more to it than just causing the #define to happen.
Thus, there does not seem to be a quick way for me to see what would
result in the kyua run from more avoiding of vfork use (beyond what
use of SH_DISABLE_VFORK= did).
> There is even the possibility that a WITH_ASAN= buildworld could
> build a world that uses fork behavior for vfork and never does an
> actual vfork (by behavior). Basically restricting itself to things
> that fit with ASAN use.
>
>
>
> I tracked this issue down via a report in a kyua run that
> referenced:
>
> QUOTE
> ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7fffa580 at
> pc 0x000801e0f8ed bp 0x7fff9bf0 sp 0x7fff9be8
> WRITE of size 8 at 0x7fffa580 thread T0
> . . .
> Address 0x7fffa580 is located in stack of thread T0 at offset 0 in frame
>#0 0x7fffa5ef ()
>
> This frame has 1 object(s):
>[32, 288) 'buf' (line 180)
> HINT: this may be a false positive if your program uses some custom stack
> unwind mechanism, swapcontext or vfork
> (longjmp and C++ exceptions *are* supported)
> END QUOTE
>
> The "'buf' (line 180)" turned out to be from the declaration in
> bin/sh/exec.c 's tryexec :
>
> static void
> tryexec(char *cmd, char **argv, char **envp)
> {
>int e, in;
>ssize_t n;
>char buf[256];
>
>execve(cmd, argv, envp);
>e = errno;
>if (e == ENOEXEC) {
>INTOFF;
>in = open(cmd, O_RDONLY | O_NONBLOCK);
>if (in != -1) {
>n = pread(in, buf, sizeof buf, 0);
>close(in);
>if (n > 0 && isbinary(buf, n)) {
>errno = ENOEXEC;
>return;
>}
>}
>*argv = cmd;
>*--argv = __DECONST(char *, _PATH_BSHELL);
>execve(_PATH_BSHELL, argv, envp);
>}
>errno = e;
> }
>
> So I could finally see/validate that an example
> stack-buffer-* report was tied to vfork handling
> (analyzing other code related to the tryexec use).
>
> The report happens well after tryexec did its
> execve. The information about buf is just old,
> no longer accurate information, leading to a
> false report:
>
>
> ==87961==ERROR: AddressSanitizer: stack-buffer-underflow on address
> 0x7fffa580 at pc 0x000801e0f8ed bp 0x7fff9bf0 sp 0x7fff9be8
> WRITE of size 8 at 0x7fffa580 thread T0
>#0 0x801e0f8ec in bintime2timespec
> /usr/obj/BUILDs/main-amd64-nodbg-clang-alt/usr/main-src/amd64.amd64/tmp/usr/include/sys/time.h:285:14
>#1 0x801e0f8ec in __vdso_clock_gettime
> /usr/main-src/lib/libc/sys/__vdso_gettimeofday.c:195:2
>#2 0x801e0e1a0 in clock_gettime
> /usr/main-src/lib/libc/sys/clock_gettime.c:48:11
>#3 0x10d54da in clock_gettime
> /usr/main-src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:2189:13
>#4 0x11234f5 in __sanitizer::MonotonicNanoTime()
> /usr/main-src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:860:3
>#5 0x10ba02c in
> __sanitizer::SizeClassAllocator64<__asan::AP64<__sanitizer::LocalAddressSpaceView>
> >::PopulateFreeArray(__sanitizer::AllocatorStats*, unsigned long,
> __sanitizer::SizeClassAllocator
> 64<__as
UBSAN report from kyua run in WITH_UBSAN= based world (via chroot): /bin/sh 's waitcmdloop does NULL+0 undefined behavior
# /bin/sh /usr/tests/bin/sh/builtins/wait6.0
/usr/main-src/bin/sh/jobs.c:590:35: runtime error: applying zero offset to null
pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
/usr/main-src/bin/sh/jobs.c:590:35 in
/usr/main-src/bin/sh/jobs.c:601:22: runtime error: applying zero offset to null
pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
/usr/main-src/bin/sh/jobs.c:601:22 in
So:
# lldb /bin/sh /usr/tests/bin/sh/builtins/wait6.0
(lldb) target create "/bin/sh"
Current executable set to '/bin/sh' (x86_64).
(lldb) settings set -- target.run-args "/usr/tests/bin/sh/builtins/wait6.0"
(lldb) run
Process 66125 launched: '/bin/sh' (x86_64)
Process 66125 stopped
* thread #1, name = 'sh', stop reason = Nullptr with offset
frame #0: 0x01135850 sh`::__ubsan_on_report() at
ubsan_monitor.cpp:39
36 }
37
38 SANITIZER_WEAK_DEFAULT_IMPL
-> 39 void __ubsan::__ubsan_on_report(void) {}
40
41 void __ubsan::__ubsan_get_current_report_data(const char **OutIssueKind,
42 const char **OutMessage,
(lldb) bt
* thread #1, name = 'sh', stop reason = Nullptr with offset
* frame #0: 0x01135850 sh`::__ubsan_on_report() at
ubsan_monitor.cpp:39
frame #1: 0x01130011
sh`__ubsan::Diag::~Diag(this=0x7fffcc60) at ubsan_diag.cpp:354:29
frame #2: 0x01134f44
sh`handlePointerOverflowImpl(Data=, Base=,
Result=, Opts=(FromUnrecoverableHandler = false, pc = 18263566, bp
= 140737488343328)) at ubsan_diag.h:0:21
frame #3: 0x01134a7a
sh`::__ubsan_handle_pointer_overflow(Data=, Base=,
Result=) at ubsan_handlers.cpp:815:3
frame #4: 0x0116ae0e sh`waitcmdloop(job=0x) at
jobs.c:590:35
frame #5: 0x0114528a sh`evalcommand(cmd=, flags=0,
backcmd=0x) at eval.c:1107:16
frame #6: 0x0113eeb8 sh`evaltree(n=0x615000d8,
flags=) at eval.c:289:4
frame #7: 0x0117a317 sh`cmdloop(top=) at main.c:228:4
frame #8: 0x01179789 sh`main(argc=2, argv=) at
main.c:175:3
frame #9: 0x010b35dd sh`_start(ap=,
cleanup=) at crt1_c.c:73:7
(lldb) thread info -s
thread #1: tid = 101020, 0x01135850 sh`::__ubsan_on_report() at
ubsan_monitor.cpp:39, name = 'sh', stop reason = Nullptr with offset
{
"col": 35,
"description": "nullptr-with-offset",
"filename": "/usr/main-src/bin/sh/jobs.c",
"instrumentation_class": "UndefinedBehaviorSanitizer",
"line": 590,
"memory_address": 0,
"summary": "Applying zero offset to null pointer",
"tid": 101020,
"trace": []
}
(lldb) up 4
frame #4: 0x0116ae0e sh`waitcmdloop(job=0x) at
jobs.c:590:35
587 return retval;
588 }
589 } else {
-> 590 for (jp = jobtab ; jp < jobtab + njobs; jp++)
591 if (jp->used && jp->state == JOBDONE) {
592 if (! iflag || ! jp->changed)
593 freejob(jp);
(lldb) c
Process 66125 resuming
/usr/main-src/bin/sh/jobs.c:590:35: runtime error: applying zero offset to null
pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
/usr/main-src/bin/sh/jobs.c:590:35 in
Process 66125 stopped
* thread #1, name = 'sh', stop reason = Nullptr with offset
frame #0: 0x01135850 sh`::__ubsan_on_report() at
ubsan_monitor.cpp:39
36 }
37
38 SANITIZER_WEAK_DEFAULT_IMPL
-> 39 void __ubsan::__ubsan_on_report(void) {}
40
41 void __ubsan::__ubsan_get_current_report_data(const char **OutIssueKind,
42 const char **OutMessage,
(lldb) bt
* thread #1, name = 'sh', stop reason = Nullptr with offset
* frame #0: 0x01135850 sh`::__ubsan_on_report() at
ubsan_monitor.cpp:39
frame #1: 0x01130011
sh`__ubsan::Diag::~Diag(this=0x7fffcc60) at ubsan_diag.cpp:354:29
frame #2: 0x01134f44
sh`handlePointerOverflowImpl(Data=, Base=,
Result=, Opts=(FromUnrecoverableHandler = false, pc = 1826, bp
= 140737488343328)) at ubsan_diag.h:0:21
frame #3: 0x01134a7a
sh`::__ubsan_handle_pointer_overflow(Data=, Base=,
Result=) at ubsan_handlers.cpp:815:3
frame #4: 0x0116b17c sh`waitcmdloop(job=0x) at
jobs.c:601:22
frame #5: 0x0114528a sh`evalcommand(cmd=, flags=0,
backcmd=0x) at eval.c:1107:16
frame #6: 0x0113eeb8 sh`evaltree(n=0x615000d8,
flags=) at eval.c:289:4
frame #7: 0x0117a317 sh`cmdloop(top=) at main.c:228:4
frame #8: 0x01179789 sh`main(argc=2, argv=) at
main.c:175:3
frame #9: 0x010b35dd sh`_start(ap=,
cleanup=) at crt1_c.c:73:7
(lldb) thread info -s
thread #1: tid = 101020, 0x01135850 sh`::__ubsan_on_report() at
ubsan
