[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Paul Floyd]

https://bugs.kde.org/show_bug.cgi?id=208531

Paul Floyd  changed:

   What|Removed |Added

 Status|CONFIRMED   |RESOLVED
 Resolution|--- |FIXED

--- Comment #38 from Paul Floyd  ---
That's it. The Beastie has landed. Champagne tonight.

FreeBSD is now part of the official Valgrind code. When 3.18 is out I'll update
the FreeBSD port (devel/valgrind) to point to the offical tarball.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Paul Floyd]

https://bugs.kde.org/show_bug.cgi?id=208531

Paul Floyd  changed:

   What|Removed |Added

   Assignee|jsew...@acm.org |pjfl...@wanadoo.fr

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Ed Maste]

https://bugs.kde.org/show_bug.cgi?id=208531

Ed Maste  changed:

   What|Removed |Added

   See Also||https://bugs.kde.org/show_b
   ||ug.cgi?id=433437

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Paul Floyd]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #37 from Paul Floyd  ---
clang/x86 none/tests/sigsusp no longer hangs :-)

That 'only' leaves a couple of segfaults and client requests with clang as
serious problems on FreeBSD 12.1.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Paul Floyd]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #36 from Paul Floyd  ---
Gory details, part 3. Last for the moment.

Good news for any FreeBSD users compiling with GCC on amd64:

== 735 tests, 7 stderr failures, 1 stdout failure, 0 stderrB failures, 0
stdoutB failures, 0 post failures ==

I won't dwell on that since clang is the priority.

The full git diff --stat output is too scary to copy in its entirety. Here is
the last line

 498 files changed, 107880 insertions(+), 1377 deletions(-)

I will work on removing as many gratuitous differences as possible, but at the
same time I'm adding regression tests, so I'm afraid this is just going to get
bigger.

Finally, syscall and test coverage. I reckon that FreeBSD 12.1 has 363 syscalls
(not counting obsolete ones). 75 of these are covered by generic syscalls, 287
specific to FreeBSD. 43 of these are unimplemented. There are tests for at
least 77 of these syscalls, plus a full scalar test.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Paul Floyd]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #35 from Paul Floyd  ---
Gory details part 2.

x86/clang, this is the worst case.

== 668 tests, 32 stderr failures, 2 stdout failures, 3 stderrB failures, 1
stdoutB failure, 0 post failures ==

Mostly the same as amd64/clang

none/tests/sigsusp
Testcase hangs :-( We try to clear the VGKILL bit in the syswrap, but for some
reason that is failing (perhaps because the mask is const).

none/tests/manythreads
Fails after 8192 threads have run. Don't know why yet.

gdbserver_tests/nlpasssigalrm
SIGSEGV in host. Problem with correct construction of sigframe.

none/tests/pth_self_kill_15_other
drd/tests/sigalrm
More SIGSEGVs in host.

drd/tests/concurrent_close (4 cases in total)
drd/tests/annotate*
Extra conflicting loads

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Paul Floyd]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #34 from Paul Floyd  ---
Some gory details, part 1. In the FreeBSD world, normally there are 2 active
release streams (currently 11 and 12) and one development stream (currently
13). The system compiler is clang, with GCC as an optional install. Whilst
other architectures are supported by FreeBSD, amd64 and x86 are the only two
that work with Valgrind. x86 (i386 in the FreeBSD world) is still fairly widely
used.

My primary build/test work is all on FreeBSD 12.1. I started with amd64/GCC but
 now I'm concentrating on amd64/clang. I also build and test on x86/clang and
x86/gcc. I occasionally do builds on FreeBSD 11.3.

Here are some regtest results that I'm currently getting, and some analysis of
the failures.

More analysis here https://github.com/paulfloyd/freebsd_valgrind/issues

amd64/clang
== 725 tests, 22 stderr failures, 1 stdout failure, 1 stderrB failure, 0
stdoutB failures, 0 post failures ==


memcheck/tests/leak* (4 fails)
There's a discrepancy with clang between the client request report and the
final report. WIP.

drd/tests/omp (3 fails)
Many errors, not fully analysed. clang support for OpenMP seems fairly weak.

memcheck/tests/varinfo6 (+1 similar)
drd/tests/tc04_free_lock
Issue with getting varinfo, error line number printed as 0

drd/tests/dlopen
Hits an assert setting pthreadid, possibly related to the special handling that
was required for dlopen on Solaris.

memcheck/tests/descr_belowsp
Error message doesn't mention guard page. Needs different guard page detection
heuristics?

memcheck/tests/gone_abrt_xml
helgrind/tests/tc22_exit_w_lock 
There's a hack for stacktraces without function call prologs that doesn't work
in these 2 cases.

gdbserver_tests/mcinfcallWSRU
Line number diffs in stderrB. Looks harmless.

memcheck/tests/amd64/insn-pmovmskb
Looks like clang optimization won the day with this testcase. Missing error
messages and changes in order.

memcheck/tests/clientperm
Looks like clang optimization of branches.

helgrind/tests/tls_threads (+ drd)
Hard problem. On Linux this relies on spying on libc variables for tls. That
doesn't work with non-GCC FreeBSD libc.

helgrind/tests/pth_cond_destroy_busy
One missing error message

helgrind/tests/tc22_exit_w_lock 

drd/tests/std_list
A load of locale errors.

drd/tests/atomic_var
Missing varinfo details

none/tests/amd64/ssse3_misaligned
SIGSEGV in host :-(

none/tests/rlimit_nofile
Only fails when I'm on KDE because of an extra krunner socket.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Paul Floyd]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #33 from Paul Floyd  ---
Over 10 years since this item was opened and it's my turn to take a stab at
getting it done.

Here a couple of sample results (best and worst)

GCC on amd64 configured with -O0
== 714 tests, 19 stderr failures, 5 stdout failures, 1 stderrB failure, 3
stdoutB failures, 0 post failures ==

clang on x86
== 648 tests, 69 stderr failures, 4 stdout failures, 4 stderrB failures, 4
stdoutB failures, 1 post failure ==


Stuff that's been fixed
~~~
FreeBSD 11.1 added a ~64MB MAP_GUARD region below the user stack which was
breaking the address space manager on x86.
For x86 on amd64 there was a failure arising from Valgrind not fully
synthesizing auxv for the guest combined with a bug in rtld. That took a long
time to hunt down.
x86 sigreturn was broken.
Debug output never worked on x86 and caused crashes on amd64.
There were several redir errors in drd and helgrind.
Added ~ 20 syscalls.


Remaining problems
~~
The number one remaining issue is with signal delivery, and it's worse on x86.

Some of the clang failures that I've looked at are due to the codegen. In a
couple of cases I saw that clang used cmov opcodes rather then test/jmp used by
GCC. I need to look ore at things like varinfo differences.

The default suppression file is pretty noisy with plain old libc functions. If
I run 'ps' under memcheck then I get a bunch of 'Conditional jumps' in strlen
and strvis. Qt apps could also do with some attention.


Not enough regtests yet. I've only added 7 I think. I definitely need to add a
'scalar' test.

Other stuff
~~~
There are a few other issues, but I might be able to live with them as known
limitations.

I've tried to clean out the diffs in common source files as much as possible,
but there's still some work to do. I have been fairly heavy handed in modifying
test scripts that use sed, and should probably make .in files and a configure
time AC_CONF_SED.

Next steps
~~
Before any talk of patches or code reviews:

I'd like to deal with the signal issues on x86, but that does not look easy.

The issues with clang need to be all analyzed, since this is the system
compiler.

Test, test, test.

My github repo is here
https://github.com/paulfloyd/freebsd_valgrind

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Paul Floyd]

https://bugs.kde.org/show_bug.cgi?id=208531

Paul Floyd  changed:

   What|Removed |Added

 CC||pa...@free.fr

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Ed Maste]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #32 from Ed Maste  ---
Rebased patch set in
https://github.com/FreeBSDFoundation/valgrind/tree/freebsd.20190307

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Ed Maste]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #31 from Ed Maste  ---
I've fixed up a few more issues, and started working on curating the changes
into more reasonably reviewable pieces.

Current WIP is in the "freebsd.20190103" branch - I'll try to keep the patch
set rebased on upstream and will start a new branch when rebasing.

https://github.com/FreeBSDFoundation/valgrind/tree/freebsd.20190103

Two questions:

- link_tool_exe_freebsd.in is a copy of older link_tool_exe_linux.in. Our
toolchain is GNU-compatible and link_tool_exe_linux.in will probably work out
of the box on FreeBSD. For cases like this should we share the script (perhaps
renaming it)?

- tests/filter_libc starts with
#! /usr/bin/perl -w
which does not work on FreeBSD (perl is /usr/local/bin/perl).

Should this be renamed to tests/filter_libc.in and start with #! @PERL@ ?

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Yuri]

https://bugs.kde.org/show_bug.cgi?id=208531

Yuri  changed:

   What|Removed |Added

 CC|y...@tsoft.com  |

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Ed Maste]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #30 from Ed Maste  ---
I believe the specific points raised in previous comments have been addressed.
On the mailing list Philippe Waroquiers reported that many files based on a
Linux version still have the old filename in header comments etc., will address
those soon.

Regtest results as of today:
 == 599 tests, 203 stderr failures, 68 stdout failures, 0 stderrB
failures, 0 stdoutB failures, 0 post failures ==

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Ed Maste]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #29 from Ed Maste  ---
(In reply to Bart Van Assche from comment #16)
> -check_PROGRAMS = clreq simwork threads
> +check_PROGRAMS = clreq simwork
> +
> +if !VGCONF_PLATFORMS_INCLUDE_X86_FREEBSD
> +check_PROGRAMS += threads
> +endif

This is no longer in the patchset. tsan_unittest is disabled on FreeBSD in the
patch though:

+if !VGCONF_OS_IS_FREEBSD
+check_PROGRAMS += tsan_unittest
 endif

> And why has the change below been made ? As far as I can see $(EGREP)
> doesn't occur anywhere.
> 
> --- //depot/vendor/valgrind/configure.in.orig
> +++ //depot/vendor/valgrind/configure.in
> @@ -39,6 +39,7 @@
>  # AC_SUBST([OBJCFLAGS])
>  #])
>  AC_PROG_RANLIB
> +AC_PROG_EGREP

Does it get used implicitly by AC_EGREP_CPP perhaps? I also see a ${EGREP} use
in setting xpg_present.

> And why has AC_PATH_PROG(SED, sed) been added ? AC_PROG_SED is already
> present in configure.in.

This one has been removed.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Ed Maste]

https://bugs.kde.org/show_bug.cgi?id=208531

--- Comment #28 from Ed Maste  ---
Current work is in progress at https://github.com/freebsdfoundation/valgrind.
The most up-to-date branch is freebsd.rebase,
https://github.com/freebsdfoundation/valgrind/tree/freebsd.rebase.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Ed Maste]

https://bugs.kde.org/show_bug.cgi?id=208531

Ed Maste  changed:

   What|Removed |Added

 CC||y...@tsoft.com

--- Comment #27 from Ed Maste  ---
*** Bug 368873 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 208531] [PATCH]: FreeBSD support for valgrind

[Ivo Raisr]

https://bugs.kde.org/show_bug.cgi?id=208531

Ivo Raisr  changed:

   What|Removed |Added

 CC||iv...@ivosh.net

--- Comment #26 from Ivo Raisr  ---
Stanislav, Ed et al,
Has there been any progress recently on fixing the remaining issues on FreeBSD
port [1]? I see the last commit dates 2016-01-13, more than 1,5 years ago.
Perhaps the port has moved somewhere else?

[1] https://bitbucket.org/stass/valgrind-freebsd/

-- 
You are receiving this mail because:
You are watching all bug changes.