Re: [Qemu-devel] [PATCH v3 01/15] tests: add fp-test, a floating point test suite
On Wed, Apr 11, 2018 at 02:20:49 +0100, Alex Bennée wrote: > Emilio G. Cotawrites: > So with the attached patch and my proposed cross build we can now get: > > 02:15:54 [alex@zen:~/l/q/qemu.git] softfloat-fixes-for-2.12-v1 ± find . > -iname "fp-test" | xargs file > ./ppc64-linux-user/tests/fp-test: ELF 64-bit LSB executable, 64-bit > PowerPC or cisco 7500, version 1 (GNU/Linux), statically linked, for > GNU/Linux 3.2.0, not stripped (snip) > But it did mean having to hack about a little, mainly to get rid of > glib. That will let us build fp-test using a cross-compiler. My initial thinking was that since we'd end up testing on a real host (with "-t host" mode), cross-compiling wouldn't be necessary since we could just compile natively on said host. But since we seem to be moving towards supporting cross-compilers, it takes little effort to cross-compile fp-test as well. The main hurdle is to remove the glib dependence as you pointed out. I just wrote a few patches to do this: $ git log --oneline -5 --reverse 48e802b osdep: disable glib-compat.h include with QEMU_NO_GLIB d3c78c7 softfloat: do not include glib headers 744a9c4 tests/tcg/Makefile: define _GNU_SOURCE 661c0e2 tests/fp: fixup e057d45 tests/tcg/Makefile: fp-test build fixup The main difference with your attached patch is that we remove ifdef's from fp-test.c while keeping the osdep.h include. You can fetch the patches from https://github.com/cota/qemu/tree/softfloat-fixes-for-2.12-v1 [BTW the name of the branch is just to keep your original branch name; I'm in now way intending for this to be part of 2.12 :>] Thanks, Emilio
Re: [Qemu-devel] [PATCH v3 01/15] tests: add fp-test, a floating point test suite
Alex Bennéewrites: > Emilio G. Cota writes: > >> This will allow us to run correctness tests against our >> FP implementation. The test can be run in two modes (called >> "testers"): host and soft. With the former we check the results >> and FP flags on the host machine against the model. >> With the latter we check QEMU's fpu primitives against the >> model. Note that in soft mode we are not instantiating any >> particular CPU (hence the HW_POISON_H hack to avoid macro poisoning); >> for that we need to run the test in host mode under QEMU. > > > So with the attached patch and my proposed cross build we can now get: > > --- a/tests/tcg/Makefile > +++ b/tests/tcg/Makefile > @@ -24,6 +24,9 @@ > VPATH = $(SRC_PATH)/tests/tcg/multiarch > TEST_SRCS = $(wildcard $(SRC_PATH)/tests/tcg/multiarch/*.c) > > +VPATH += $(SRC_PATH)/tests/fp > +TEST_SRCS += $(wildcard $(SRC_PATH)/tests/fp/*.c) > + > VPATH += $(SRC_PATH)/tests/tcg/$(ARCH) > TEST_SRCS += $(wildcard $(SRC_PATH)/tests/tcg/$(ARCH)/*.c) It also needs: fp-test: LDFLAGS+=-lm -- Alex Bennée
Re: [Qemu-devel] [PATCH v3 01/15] tests: add fp-test, a floating point test suite
Emilio G. Cotawrites: > This will allow us to run correctness tests against our > FP implementation. The test can be run in two modes (called > "testers"): host and soft. With the former we check the results > and FP flags on the host machine against the model. > With the latter we check QEMU's fpu primitives against the > model. Note that in soft mode we are not instantiating any > particular CPU (hence the HW_POISON_H hack to avoid macro poisoning); > for that we need to run the test in host mode under QEMU. So with the attached patch and my proposed cross build we can now get: 02:15:54 [alex@zen:~/l/q/qemu.git] softfloat-fixes-for-2.12-v1 ± find . -iname "fp-test" | xargs file ./ppc64-linux-user/tests/fp-test: ELF 64-bit LSB executable, 64-bit PowerPC or cisco 7500, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, not stripped ./armeb-linux-user/tests/fp-test: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, not stripped ./aarch64-linux-user/tests/fp-test:ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, for GNU/Linux 3.7.0, not stripped ./i386-linux-user/tests/fp-test: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, not stripped ./arm-linux-user/tests/fp-test:ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, not stripped ./s390x-linux-user/tests/fp-test: ELF 64-bit MSB executable, IBM S/390, version 1 (SYSV), statically linked, for GNU/Linux 3.2.0, not stripped ./aarch64_be-linux-user/tests/fp-test: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, for GNU/Linux 3.7.0, not stripped ./tests/fp-test/fp-test: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped But it did mean having to hack about a little, mainly to get rid of glib. --8<---cut here---start->8--- >From 04ed0d9f58f34aa51b9a8284514aab6e36a702b4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alex=20Benn=C3=A9e?= Date: Wed, 11 Apr 2018 01:35:52 +0100 Subject: [PATCH] tests/tcg: add fp-test to per-guest tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The fp-test code was originally designed to be able to include softfloat. However cross-compiling QEMU based code is harder than it needs to be so hide softfloat stuff behind USE_SOFTFLOAT. We also need to tweak: - manually include what we need - g_assert -> assert() - use libc hsearch instead of g_hash_table Signed-off-by: Alex Bennée --- tests/fp/fp-test.c | 148 ++--- tests/tcg/Makefile | 3 ++ 2 files changed, 121 insertions(+), 30 deletions(-) diff --git a/tests/fp/fp-test.c b/tests/fp/fp-test.c index 27637c4617..4cee2a918c 100644 --- a/tests/fp/fp-test.c +++ b/tests/fp/fp-test.c @@ -6,12 +6,72 @@ * License: GNU GPL, version 2 or later. * See the COPYING file in the top-level directory. */ -#ifndef HW_POISON_H -#error Must define HW_POISON_H to work around TARGET_* poisoning -#endif + +/* If HW_POISON_H isn't defined then we aren't building against qemu's + * softfloat */ +#ifdef HW_POISON_H #include "qemu/osdep.h" #include "fpu/softfloat.h" +#define USE_SOFTFLOAT 1 + +#else /* else define what QEMU would have given us */ + +#define _GNU_SOURCE + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +/* See include/fpu/softfloat-types.h */ +enum { +float_tininess_after_rounding = 0, +float_tininess_before_rounding = 1 +}; + +enum { +float_round_nearest_even = 0, +float_round_down = 1, +float_round_up = 2, +float_round_to_zero = 3, +float_round_ties_away= 4, +float_round_to_odd = 5, +}; + +enum { +float_flag_invalid = 1, +float_flag_divbyzero = 4, +float_flag_overflow = 8, +float_flag_underflow = 16, +float_flag_inexact = 32, +float_flag_input_denormal = 64, +float_flag_output_denormal = 128 +}; + +/* See include/compiler.h */ +#ifndef likely +#if __GNUC__ < 3 +#define __builtin_expect(x, n) (x) +#endif + +#define likely(x) __builtin_expect(!!(x), 1) +#define unlikely(x) __builtin_expect(!!(x), 0) +#endif + +#endif + +#ifndef ARRAY_SIZE +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) +#endif #include #include @@ -116,16 +176,18 @@ struct tester { struct whitelist { char **lines; size_t n; -GHashTable *ht; +struct hsearch_data ht; }; static uint64_t test_stats[ERROR_MAX]; static struct whitelist whitelist; static uint8_t default_exceptions; static bool die_on_error = true; +#ifdef
[Qemu-devel] [PATCH v3 01/15] tests: add fp-test, a floating point test suite
This will allow us to run correctness tests against our FP implementation. The test can be run in two modes (called "testers"): host and soft. With the former we check the results and FP flags on the host machine against the model. With the latter we check QEMU's fpu primitives against the model. Note that in soft mode we are not instantiating any particular CPU (hence the HW_POISON_H hack to avoid macro poisoning); for that we need to run the test in host mode under QEMU. The input files are taken from IBM's FPGen test suite: https://www.research.ibm.com/haifa/projects/verification/fpgen/ I see no license file in there so I am just downloading them with wget. We might want to keep a copy on a qemu server though, in case IBM takes those files down in the future. The "IBM" syntax of those files (for now the only syntax supported in fp-test) is documented here: https://www.research.ibm.com/haifa/projects/verification/fpgen/papers/ieee-test-suite-v2.pdf Note that the syntax document has some inaccuracies; the appended parsing code works around some of those. The exception flag (-e) is important: many of the optimizations included in the following commits assume that the inexact flag is set, so "-e x" is necessary in order to test those code paths. The whitelist flag (-w) points to a file with test cases to be ignored. I have put some whitelist files online, but we should have them on a QEMU-related server. Thus, a typical of fp-test is as follows: $ cd qemu/build/tests/fp-test $ make -j && \ ./fp-test -t soft ibm/*.fptest \ -w whitelist.txt \ -e x If we want to test after-rounding tininess detection, then we need to pass "-a -w whitelist-tininess-after.txt" in addition to the above. (NB. we can pass "-w" as many times as we want.) The patch immediately after this one fixes a mismatch against the model in softfloat, but after that is applied the above should finish with a 0 return code, and print something like: All tests OK. Tests passed: 76572. Not handled: 51237, whitelisted: 2662 The tests pass on "host" mode on x86_64 and aarch64 machines, although note that for the x86_64 you need to pass -w whitelist-tininess-after.txt. Running on host mode under QEMU reports flag mismatches (e.g. for x86_64-linux-user), but that isn't too surprising given how little love the i386 frontend gets. Host mode under aarch64-linux-user passes OK. Flush-to-zero and flush-inputs-to-zero modes can be tested with the -z and -Z flags. Note however that the IBM input files are only IEEE-compliant, so for now I've tested these modes by diff'ing the reported errors against the model files. We should look into generating files for these non-standard modes to make testing these modes less painful. Signed-off-by: Emilio G. Cota--- configure |2 + tests/fp/fp-test.c | 1159 tests/Makefile.include |3 + tests/fp/.gitignore|3 + tests/fp/Makefile | 34 ++ 5 files changed, 1201 insertions(+) create mode 100644 tests/fp/fp-test.c create mode 100644 tests/fp/.gitignore create mode 100644 tests/fp/Makefile diff --git a/configure b/configure index f156805..07dc5da 100755 --- a/configure +++ b/configure @@ -7106,12 +7106,14 @@ fi # build tree in object directory in case the source is not in the current directory DIRS="tests tests/tcg tests/tcg/cris tests/tcg/lm32 tests/libqos tests/qapi-schema tests/tcg/xtensa tests/qemu-iotests tests/vm" +DIRS="$DIRS tests/fp" DIRS="$DIRS docs docs/interop fsdev scsi" DIRS="$DIRS pc-bios/optionrom pc-bios/spapr-rtas pc-bios/s390-ccw" DIRS="$DIRS roms/seabios roms/vgabios" FILES="Makefile tests/tcg/Makefile qdict-test-data.txt" FILES="$FILES tests/tcg/cris/Makefile tests/tcg/cris/.gdbinit" FILES="$FILES tests/tcg/lm32/Makefile tests/tcg/xtensa/Makefile po/Makefile" +FILES="$FILES tests/fp/Makefile" FILES="$FILES pc-bios/optionrom/Makefile pc-bios/keymaps" FILES="$FILES pc-bios/spapr-rtas/Makefile" FILES="$FILES pc-bios/s390-ccw/Makefile" diff --git a/tests/fp/fp-test.c b/tests/fp/fp-test.c new file mode 100644 index 000..27637c4 --- /dev/null +++ b/tests/fp/fp-test.c @@ -0,0 +1,1159 @@ +/* + * fp-test.c - Floating point test suite. + * + * Copyright (C) 2018, Emilio G. Cota + * + * License: GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ +#ifndef HW_POISON_H +#error Must define HW_POISON_H to work around TARGET_* poisoning +#endif + +#include "qemu/osdep.h" +#include "fpu/softfloat.h" + +#include +#include + +enum error { +ERROR_NONE, +ERROR_NOT_HANDLED, +ERROR_WHITELISTED, +ERROR_COMMENT, +ERROR_INPUT, +ERROR_RESULT, +ERROR_EXCEPTIONS, +ERROR_MAX, +}; + +enum input_fmt { +INPUT_FMT_IBM, +}; + +struct input { +const char * const name; +enum error (*test_line)(const char *line); +}; + +enum precision { +PREC_FLOAT, +PREC_DOUBLE,