On 02/02/2024 01:47, Paul Eggert wrote:
On 2/1/24 13:59, Pádraig Brady wrote:
bfloat16 looks like a truncated single precision IEEE,
so we should be able to just pad the extra 16 bits with zeros
when converting to single precision internally for processing.
Sounds good. This would mean od could work even the platform doesn't
support bfloat16_t, since od.c could fall back on the above code (though
I suppose it could be endianness-dependent).
I see that __bf16 is supported by released versions of gcc and clang,
so rather than add conversion complexity to the core od print loop,
the attached relies on compiler support for _Float16 and __bf16 types,
which compilers will expand to more targets going forward.
I tested the attached on:
gcc 13, clang 17 x86 (Both types supported)
gcc 7 aarch64 (Only -fH supported)
gcc 13 ppc(be) (Neither supported (both will be with GCC 14))
I'll commit this later.
Marking this bug as done.
thanks,
Pádraig
From 5c1349185e050b8bf8fb1d356c54170112d0e673 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <p...@draigbrady.com>
Date: Thu, 1 Feb 2024 17:59:51 +0000
Subject: [PATCH] od: support half precision floating point
Rely on compiler support for _Float16 and __bf16
to support -fH and -fB formats respectively.
I.e. IEEE 16 bit, and brain 16 bit floats respectively.
Modern GCC and LLVM compilers support both types.
clang-sect=half-precision-floating-point
https://gcc.gnu.org/onlinedocs/gcc/Half-Precision.html
https://clang.llvm.org/docs/LanguageExtensions.html#$clang-sect
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0192r4.html
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1467r9.html
This was tested on:
gcc 13, clang 17 x86 (Both types supported)
gcc 7 aarch64 (Only -fH supported)
gcc 13 ppc(be) (Neither supported. -fH will be with GCC 14)
* src/od.c: Support -tf2 or -tfH to print IEEE 16 bit floating point,
or -tfB to print Brain 16 bit floating point.
* configure.ac: Check for _Float16 and __bf16 types.
* doc/coreutils.texi (od invocation): Mention the new -f types.
* tests/od/od-float.sh: Add test cases.
* NEWS: Mention the new feature.
Addresses https://bugs.gnu.org/68871
---
NEWS | 3 ++
configure.ac | 3 ++
doc/coreutils.texi | 4 +++
src/od.c | 73 ++++++++++++++++++++++++++++++++++++++++++--
tests/od/od-float.sh | 21 +++++++++++++
5 files changed, 101 insertions(+), 3 deletions(-)
diff --git a/NEWS b/NEWS
index dc5d875dd..5b5befd2c 100644
--- a/NEWS
+++ b/NEWS
@@ -57,6 +57,9 @@ GNU coreutils NEWS -*- outline -*-
chgrp now accepts the --from=OWNER:GROUP option to restrict changes to files
with matching current OWNER and/or GROUP, as already supported by chown(1).
+ od now supports printing IEEE half precision floating point with -t fH,
+ or brain 16 bit floating point with -t fB, where supported by the compiler.
+
tail now supports following multiple processes, with repeated --pid options.
** Improvements
diff --git a/configure.ac b/configure.ac
index 48ab4ef53..64ff32a96 100644
--- a/configure.ac
+++ b/configure.ac
@@ -522,6 +522,9 @@ CFLAGS=$ac_save_CFLAGS
LDFLAGS=$ac_save_LDFLAGS
ac_c_werror_flag=$cu_save_c_werror_flag
+# Test compiler support for half precision floating point types (for od)
+AC_CHECK_TYPES([_Float16, __bf16])
+
ac_save_CFLAGS=$CFLAGS
CFLAGS="-mavx -mpclmul $CFLAGS"
AC_MSG_CHECKING([if pclmul intrinsic exists])
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index d13e0f834..c7a49de31 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -2127,6 +2127,10 @@ long
For floating point (@code{f}):
@table @asis
+@item B
+brain 16 bit float
+@item H
+half precision float
@item F
float
@item D
diff --git a/src/od.c b/src/od.c
index 75bed5e7d..1e2489367 100644
--- a/src/od.c
+++ b/src/od.c
@@ -19,6 +19,7 @@
#include <config.h>
#include <ctype.h>
+#include <float.h>
#include <stdio.h>
#include <getopt.h>
#include <sys/types.h>
@@ -49,6 +50,26 @@ typedef unsigned long long int unsigned_long_long_int;
typedef unsigned long int unsigned_long_long_int;
#endif
+#if HAVE__FLOAT16
+ /* Available since clang 6 (2018), and gcc 7 (2017). */
+ typedef _Float16 float16;
+#else
+# define HAVE__FLOAT16 0
+ /* This is just a place-holder to avoid a few '#if' directives.
+ In this case, the type isn't actually used. */
+ typedef float float16;
+#endif
+
+#if HAVE___BF16
+ /* Available since clang 11 (2020), and gcc 13 (2023). */
+ typedef __bf16 bfloat16;
+#else
+# define HAVE___BF16 0
+ /* This is just a place-holder to avoid a few '#if' directives.
+ In this case, the type isn't actually used. */
+ typedef float bfloat16;
+#endif
+
enum size_spec
{
NO_SIZE,
@@ -58,6 +79,7 @@ enum size_spec
LONG,
LONG_LONG,
/* FIXME: add INTMAX support, too */
+ FLOAT_HALF,
FLOAT_SINGLE,
FLOAT_DOUBLE,
FLOAT_LONG_DOUBLE,
@@ -71,6 +93,8 @@ enum output_format
OCTAL,
HEXADECIMAL,
FLOATING_POINT,
+ HFLOATING_POINT,
+ BFLOATING_POINT,
NAMED_CHARACTER,
CHARACTER
};
@@ -156,6 +180,11 @@ static const int width_bytes[] =
sizeof (int),
sizeof (long int),
sizeof (unsigned_long_long_int),
+#if HAVE___BF16
+ sizeof (bfloat16),
+#else
+ sizeof (float16),
+#endif
sizeof (float),
sizeof (double),
sizeof (long double)
@@ -400,8 +429,9 @@ TYPE is made up of one or more of these specifications:\n\
\n\
SIZE is a number. For TYPE in [doux], SIZE may also be C for\n\
sizeof(char), S for sizeof(short), I for sizeof(int) or L for\n\
-sizeof(long). If TYPE is f, SIZE may also be F for sizeof(float), D\n\
-for sizeof(double) or L for sizeof(long double).\n\
+sizeof(long). If TYPE is f, SIZE may also be B for Brain 16 bit,\n\
+H for Half precision float, F for sizeof(float), D for sizeof(double),\n\
+or L for sizeof(long double).\n\
"), stdout);
fputs (_("\
\n\
@@ -477,6 +507,8 @@ PRINT_TYPE (print_int, unsigned int)
PRINT_TYPE (print_long, unsigned long int)
PRINT_TYPE (print_long_long, unsigned_long_long_int)
+PRINT_FLOATTYPE (print_bfloat, bfloat16, ftoastr, FLT_BUFSIZE_BOUND)
+PRINT_FLOATTYPE (print_halffloat, float16, ftoastr, FLT_BUFSIZE_BOUND)
PRINT_FLOATTYPE (print_float, float, ftoastr, FLT_BUFSIZE_BOUND)
PRINT_FLOATTYPE (print_double, double, dtoastr, DBL_BUFSIZE_BOUND)
PRINT_FLOATTYPE (print_long_double, long double, ldtoastr, LDBL_BUFSIZE_BOUND)
@@ -773,6 +805,18 @@ decode_one_format (char const *s_orig, char const *s, char const **next,
++s;
switch (*s)
{
+ case 'B':
+ ++s;
+ fmt = BFLOATING_POINT;
+ size = sizeof (bfloat16);
+ break;
+
+ case 'H':
+ ++s;
+ fmt = HFLOATING_POINT;
+ size = sizeof (float16);
+ break;
+
case 'F':
++s;
size = sizeof (float);
@@ -802,7 +846,10 @@ decode_one_format (char const *s_orig, char const *s, char const **next,
else
{
if (size > MAX_FP_TYPE_SIZE
- || fp_type_size[size] == NO_SIZE)
+ || fp_type_size[size] == NO_SIZE
+ || (! HAVE__FLOAT16 && HAVE___BF16
+ && size == sizeof (bfloat16))
+ )
{
error (0, 0,
_("invalid type string %s;\n"
@@ -817,6 +864,15 @@ decode_one_format (char const *s_orig, char const *s, char const **next,
}
size_spec = fp_type_size[size];
+ if ((! HAVE__FLOAT16 && fmt == HFLOATING_POINT)
+ || (! HAVE___BF16 && fmt == BFLOATING_POINT))
+ {
+ error (0, 0,
+ _("this system doesn't provide a %s floating point type"),
+ quote (s_orig));
+ return false;
+ }
+
{
struct lconv const *locale = localeconv ();
size_t decimal_point_len =
@@ -824,6 +880,12 @@ decode_one_format (char const *s_orig, char const *s, char const **next,
switch (size_spec)
{
+ case FLOAT_HALF:
+ print_function = fmt == BFLOATING_POINT
+ ? print_bfloat : print_halffloat;
+ field_width = FLT_STRLEN_BOUND_L (decimal_point_len);
+ break;
+
case FLOAT_SINGLE:
print_function = print_float;
field_width = FLT_STRLEN_BOUND_L (decimal_point_len);
@@ -1598,6 +1660,11 @@ main (int argc, char **argv)
for (i = 0; i <= MAX_FP_TYPE_SIZE; i++)
fp_type_size[i] = NO_SIZE;
+#if HAVE__FLOAT16
+ fp_type_size[sizeof (float16)] = FLOAT_HALF;
+#elif HAVE___BF16
+ fp_type_size[sizeof (bfloat16)] = FLOAT_HALF;
+#endif
fp_type_size[sizeof (float)] = FLOAT_SINGLE;
/* The array entry for 'double' is filled in after that for 'long double'
so that if they are the same size, we avoid any overhead of
diff --git a/tests/od/od-float.sh b/tests/od/od-float.sh
index 35a6d1249..239b5f10e 100755
--- a/tests/od/od-float.sh
+++ b/tests/od/od-float.sh
@@ -69,4 +69,25 @@ set x $(printf 00000000ff000000 | tr 0f '\000\377' | od -t fL) || fail=1
#*) fail=1;;
#esac
+# Check Half precision IEEE 16 bit float
+if grep '^#define HAVE__FLOAT16 1' "$CONFIG_HEADER" >/dev/null; then
+ for fmt in '-tfH' '-tf2'; do
+ od_out=$(printf '\x3C\x00\x3C\x00' | od --endian=big -An $fmt | tr -d ' ')
+ test "$od_out" = '11' || fail=1
+ done
+else
+ echo "od: this system doesn't provide a 'fH' floating point type" > exp_err
+ returns_ 1 od -tfH /dev/null 2>err || fail=1
+ compare exp_err err || fail=1
+fi
+# Check Half precision Brain 16 bit float
+if grep '^#define HAVE___BF16 1' "$CONFIG_HEADER" >/dev/null; then
+ od_out=$(printf '\x3F\x80\x3F\x80' | od --endian=big -An -tfB | tr -d ' ')
+ test "$od_out" = '11' || fail=1
+else
+ echo "od: this system doesn't provide a 'fB' floating point type" > exp_err
+ returns_ 1 od -tfB /dev/null 2>err || fail=1
+ compare exp_err err || fail=1
+fi
+
Exit $fail
--
2.43.0
