On 07/10/2025 23:36, Pádraig Brady wrote:
On 12/09/2025 08:45, Johannes Schauer Marin Rodrigues wrote:
Hi,
the document "The International System of Units (SI) (9th ed)" states in
chapter 5.4.3:
The numerical value always precedes the unit and a space is always used to
separate the unit from the number. Thus the value of the quantity is the
product of the number and the unit. The space between the number and the unit
is regarded as a multiplication sign (just as a space between units implies
multiplication). The only exceptions to this rule are for the unit symbols
for degree, minute and second for plane angle, °, ′ and ″, respectively, for
which no space is left between the numerical value and the unit symbol.
https://web.archive.org/web/20230113090424/https://www.bipm.org/documents/20126/41483022/SI-Brochure-9.pdf/
it would be nice if numfmt would support outputting formatted numbers that
conform to this requirement. Currently, no space is placed between number and
unit suffix.
I tried to come up with a patch implementing this, which you can find attached.
To preserve backwards compatibility, I introduced a new option called
--space-separated. I made this option not specific to SI-conformance because it
also works with custom suffixes and it has no special handling if the suffix
happens to be a degree symbol.
Please keep me CC-ed in replies, as I'm not subscribed to the list.
Thank you for considering!
cheers, josch
Yes I think this feature is useful. I think the standards
also mention a preference to avoid line breaks between numbers,
and so NBSP might be good to support as an option also.
So we could have an optional argument like: --suffix-space[=CHAR]
Supporting:
--suffix-space # ASCII space
--suffix-space=$'\u00A0' # Non breaking space
--suffix-space=$'\u202F' # Narrow NBSP
This should ideally support such suffix spacing on _input_ numbers too,
though that's more complicated, especially with --field processing
and ASCII spacing.
I'll apply these 3 patches later.
The first supports parsing numbers with blanks before units.
The second is semi related an supports parsing numbers with grouping chars,
which is something that I've needed quite a bit, and has annoyed me.
The last is the main one to add --unit-separator to give more control.
Note I did write code to give --unit-separator higher precedence than
(default) field separators, but I removed it as it was overkill I think,
and added quite a bit of complication to the existing field parsing code.
cheers,
Padraig
From 2ef1db6175bb85ef573cd1cf6e332f9acbd5162a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email protected]>
Date: Tue, 14 Oct 2025 16:17:56 +0100
Subject: [PATCH 1/3] numfmt: support reading numbers with NBSP before unit
* src/numfmt.c (simple_strtod_human): Accept (multi-byte)
non-breaking space character between number and unit.
Note we restrict this to a single character between number
and unit, to allow less ambiguous parsing if multiple blanks
are used to delimit fields.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the improvement.
---
NEWS | 2 ++
src/numfmt.c | 18 ++++++++++++++----
tests/misc/numfmt.pl | 19 +++++++++++++++++++
3 files changed, 35 insertions(+), 4 deletions(-)
diff --git a/NEWS b/NEWS
index e6053a04b..40d443942 100644
--- a/NEWS
+++ b/NEWS
@@ -35,6 +35,8 @@ GNU coreutils NEWS -*- outline -*-
** Improvements
+ numfmt now parses numbers with a non-breaking space character before a unit.
+
wc -l now operates 10% faster on hosts that support AVX512 instructions.
diff --git a/src/numfmt.c b/src/numfmt.c
index 0cc12689e..752120a7d 100644
--- a/src/numfmt.c
+++ b/src/numfmt.c
@@ -25,6 +25,7 @@
#include "argmatch.h"
#include "c-ctype.h"
#include "mbswidth.h"
+#include "mcel.h"
#include "quote.h"
#include "skipchars.h"
#include "system.h"
@@ -645,15 +646,24 @@ simple_strtod_human (char const *input_str,
{
/* process suffix. */
- /* Skip any blanks between the number and suffix. */
- while (isblank (to_uchar (**endptr)))
- (*endptr)++;
+ /* Skip a single blank or NBSP between the number and suffix. */
+ mcel_t g = mcel_scanz (*endptr);
+ if (c32isblank (g.ch) || c32isnbspace (g.ch))
+ (*endptr) += g.len;
if (**endptr == '\0')
break; /* Treat as no suffix. */
if (!valid_suffix (**endptr))
- return SSE_INVALID_SUFFIX;
+ {
+ /* Trailing blanks are allowed. */
+ while (isblank (to_uchar (**endptr)))
+ (*endptr)++;
+ if (**endptr == '\0')
+ break;
+
+ return SSE_INVALID_SUFFIX;
+ }
if (allowed_scaling == scale_none)
return SSE_VALID_BUT_FORBIDDEN_SUFFIX;
diff --git a/tests/misc/numfmt.pl b/tests/misc/numfmt.pl
index 4dd9718c9..869dc27ae 100755
--- a/tests/misc/numfmt.pl
+++ b/tests/misc/numfmt.pl
@@ -164,6 +164,10 @@ my @Tests =
'--suffix=Foo' . 'x' x 122 . 'y 0',
{OUT => '0Foo' . 'x' x 122 . 'y'}],
['suf-21', "-d '' --from=si '4 '", {OUT => "4"}],
+ # Multiple spaces between number and suffix should be rejected
+ ['suf-22', "-d '' --from=auto '2 K'",
+ {ERR => "$prog: invalid suffix in input: '2 K'\n"},
+ {EXIT => 2}],
## GROUPING
@@ -1067,6 +1071,21 @@ my @Locale_Tests =
['lcl-fmt-7', '--format="%0\'\'6f" 1234',{OUT=>"01${lg}234"},
{ENV=>"LC_ALL=$locale"}],
+ # Single blank/NBSP acceptance between number and suffix
+ ['lcl-suf-1', "-d '' --from=auto '2 K'", {OUT => "2000"},
+ {ENV=>"LC_ALL=$locale"}],
+ ['lcl-suf-2', "-d '' --from=auto '2\tK'", {OUT => "2000"},
+ {ENV=>"LC_ALL=$locale"}],
+ # NBSP characters: U+00A0, U+2007, U+202F, U+2060
+ ['lcl-suf-3', "--from=auto '2\xc2\xa0K'", {OUT => "2000"},
+ {ENV=>"LC_ALL=$locale"}],
+ ['lcl-suf-4', "--from=auto '2\xe2\x80\x87Ki'", {OUT => "2048"},
+ {ENV=>"LC_ALL=$locale"}],
+ ['lcl-suf-5', "--from=auto '2\xe2\x80\xafK'", {OUT => "2000"},
+ {ENV=>"LC_ALL=$locale"}],
+ ['lcl-suf-6', "--from=auto '2\xe2\x81\xa0Ki'", {OUT => "2048"},
+ {ENV=>"LC_ALL=$locale"}],
+
);
if ($locale ne 'C')
{
--
2.51.0
From 5729f75bd777bf5f68ae8c10b07648a0f2308cf3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email protected]>
Date: Tue, 14 Oct 2025 21:06:03 +0100
Subject: [PATCH 2/3] numfmt: support reading numbers with grouping characters
This does not validate grouping character placement,
and currently just ignores grouping characters.
* src/numfmt.c (simple_strtod_int): Skip grouping chars
that are part of a number.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the improvement.
---
NEWS | 3 ++-
src/numfmt.c | 14 +++++++++++++-
tests/misc/numfmt.pl | 16 ++++++++++++++--
3 files changed, 29 insertions(+), 4 deletions(-)
diff --git a/NEWS b/NEWS
index 40d443942..a07fe298c 100644
--- a/NEWS
+++ b/NEWS
@@ -35,7 +35,8 @@ GNU coreutils NEWS -*- outline -*-
** Improvements
- numfmt now parses numbers with a non-breaking space character before a unit.
+ numfmt now parses numbers with a non-breaking space character before a unit,
+ and numbers containing grouping characters from the current locale.
wc -l now operates 10% faster on hosts that support AVX512 instructions.
diff --git a/src/numfmt.c b/src/numfmt.c
index 752120a7d..0339e663b 100644
--- a/src/numfmt.c
+++ b/src/numfmt.c
@@ -207,6 +207,8 @@ static bool debug;
/* will be set according to the current locale. */
static char const *decimal_point;
static int decimal_point_length;
+static char const *thousands_sep;
+static int thousands_sep_length;
/* debugging for developers. Enables devmsg(). */
static bool dev_debug = false;
@@ -515,6 +517,11 @@ simple_strtod_int (char const *input_str,
val += digit;
++(*endptr);
+
+ if (thousands_sep_length > 0
+ && STREQ_LEN (*endptr, thousands_sep, thousands_sep_length)
+ && c_isdigit ((*endptr)[thousands_sep_length]))
+ (*endptr) += thousands_sep_length;
}
if (! found_digit
&& ! STREQ_LEN (*endptr, decimal_point, decimal_point_length))
@@ -1473,6 +1480,11 @@ main (int argc, char **argv)
decimal_point = ".";
decimal_point_length = strlen (decimal_point);
+ thousands_sep = nl_langinfo (THOUSEP);
+ if (thousands_sep == nullptr)
+ thousands_sep = "";
+ thousands_sep_length = strlen (thousands_sep);
+
atexit (close_stdout);
while (true)
@@ -1601,7 +1613,7 @@ main (int argc, char **argv)
{
if (scale_to != scale_none)
error (EXIT_FAILURE, 0, _("grouping cannot be combined with --to"));
- if (debug && (strlen (nl_langinfo (THOUSEP)) == 0))
+ if (debug && thousands_sep_length == 0)
error (0, 0, _("grouping has no effect in this locale"));
}
diff --git a/tests/misc/numfmt.pl b/tests/misc/numfmt.pl
index 869dc27ae..276630483 100755
--- a/tests/misc/numfmt.pl
+++ b/tests/misc/numfmt.pl
@@ -1046,8 +1046,20 @@ my @Locale_Tests =
{OUT=>"7${lg}000${lg}000"},
{ENV=>"LC_ALL=$locale"}],
- # Input with locale'd decimal-point
- ['lcl-stdtod-1', '--from=si 12,2K', {OUT=>"12200"},
+ # Input with locale's grouping
+ ['lcl-strtod-1', '--from=si 1${lg}234K', {OUT=>"1234000"},
+ {ENV=>"LC_ALL=$locale"}],
+
+ # Input with locale's grouping. Note position not validated.
+ ['lcl-strtod-2', '--from=si 12${lg}34K', {OUT=>"1234000"},
+ {ENV=>"LC_ALL=$locale"}],
+
+ # Input with locale's decimal-point
+ ['lcl-strtod-3', '--from=si 12,2K', {OUT=>"12200"},
+ {ENV=>"LC_ALL=$locale"}],
+
+ # Input with locale's grouping and decimal-point
+ ['lcl-strtod-4', '--from=si 1${lg}23,4K', {OUT=>"123400"},
{ENV=>"LC_ALL=$locale"}],
['lcl-dbl-to-human-1', '--to=si 1100', {OUT=>"1,1k"},
--
2.51.0
From 6b7a40c1be507a20865fc9a231c525d760b4d411 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email protected]>
Date: Thu, 9 Oct 2025 14:24:12 +0100
Subject: [PATCH 3/3] numfmt: add --unit-separator
Output, accept, or disallow a string between the number and unit
as recommended in <https://physics.nist.gov/cuu/Units/checklist.html>
I.e. support outputting numbers of the form: "1234 M"
* src/numfmt.c (simple_strtod_human): Skip unit separator if present,
or disallow a unit separator if empty.
(double_to_human): Output unit separator if specified.
(main): Accept --unit-separator.
* tests/misc/numfmt.pl: Add test cases.
* doc/coreutils.texi: Describe the new option,
giving examples of interaction with --delimiter.
* NEWS: Mention the new feature.
* THANKS.in: Add Johannes Schauer Marin Rodrigues,
who provided a preliminary patch.
---
NEWS | 5 ++++
THANKS.in | 1 +
doc/coreutils.texi | 19 +++++++++++-
src/numfmt.c | 44 ++++++++++++++++++++++-----
tests/misc/numfmt.pl | 71 ++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 132 insertions(+), 8 deletions(-)
diff --git a/NEWS b/NEWS
index a07fe298c..b34513271 100644
--- a/NEWS
+++ b/NEWS
@@ -33,6 +33,11 @@ GNU coreutils NEWS -*- outline -*-
that use the GNU extension /NUM or +NUM formats.
[bug introduced in coreutils-8.28]
+** New Features
+
+ 'numfmt' now accepts the --unit-separator=SEP option, to output or accept
+ a separator between the number and unit. For e.g. "1234 M".
+
** Improvements
numfmt now parses numbers with a non-breaking space character before a unit,
diff --git a/THANKS.in b/THANKS.in
index 8c97a8138..8f6af1b61 100644
--- a/THANKS.in
+++ b/THANKS.in
@@ -315,6 +315,7 @@ Joey Hess [email protected]
Johan Boule [email protected]
Johan Danielsson [email protected]
Johannes Altmanninger [email protected]
+Johannes Schauer Marin Rodrigues [email protected]
John Bley [email protected]
John Gatewood Ham [email protected]
John Gotts [email protected]
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 26c9209a3..89534db72 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -19447,7 +19447,7 @@ Print (to standard error) warning messages about possible erroneous usage.
@itemx --delimiter=@var{d}
@opindex -d
@opindex --delimiter
-Use the character @var{d} as input field separator (default: whitespace).
+Use the character @var{d} as input field separator (default: newline or blank).
Using non-default delimiter turns off automatic padding.
@item --field=@var{fields}
@@ -19544,6 +19544,23 @@ the output numbers represent other units (e.g. to represent @samp{4,000,000}
bytes in blocks of 1kB, use @samp{--to=si --to-unit=1000}).
Suffixes are handled as with @samp{--from=auto}.
+@item --unit-separator=@var{sep}
+@opindex --unit-separator
+Support a separator @var{sep} between the number and unit,
+with @option{--from} or @option{--to} auto-scaled units.
+By default a blank or non-breaking space character is accepted on input,
+and no separator is printed on output.
+When parsing input, the specified unit separator has lower precedence
+than field delimiters. See the @option{--delimiter} option above.
+
+Examples:
+@example
+Add a space on output: @option{--unit-separator=' '}
+Disable blanks on input: @option{--unit-separator=''}
+Support blanks on input: @option{--delimiter=''}
+Ditto and output non-breaking space: @option{-d '' --unit-separator=$'\u00A0'}
+@end example
+
@optZeroTerminated
@newlineFieldSeparator
diff --git a/src/numfmt.c b/src/numfmt.c
index 0339e663b..c0c9363c1 100644
--- a/src/numfmt.c
+++ b/src/numfmt.c
@@ -60,7 +60,8 @@ enum
DEV_DEBUG_OPTION,
HEADER_OPTION,
FORMAT_OPTION,
- INVALID_OPTION
+ INVALID_OPTION,
+ UNIT_SEPARATOR_OPTION
};
enum scale_type
@@ -140,6 +141,7 @@ static struct option const longopts[] =
{"round", required_argument, nullptr, ROUND_OPTION},
{"padding", required_argument, nullptr, PADDING_OPTION},
{"suffix", required_argument, nullptr, SUFFIX_OPTION},
+ {"unit-separator", required_argument, nullptr, UNIT_SEPARATOR_OPTION},
{"grouping", no_argument, nullptr, GROUPING_OPTION},
{"delimiter", required_argument, nullptr, 'd'},
{"field", required_argument, nullptr, FIELD_OPTION},
@@ -172,6 +174,7 @@ static enum scale_type scale_to = scale_none;
static enum round_type round_style = round_from_zero;
static enum inval_type inval_style = inval_abort;
static char const *suffix = nullptr;
+static char const *unit_separator = nullptr;
static uintmax_t from_unit_size = 1;
static uintmax_t to_unit_size = 1;
static int grouping = 0;
@@ -653,10 +656,24 @@ simple_strtod_human (char const *input_str,
{
/* process suffix. */
- /* Skip a single blank or NBSP between the number and suffix. */
- mcel_t g = mcel_scanz (*endptr);
- if (c32isblank (g.ch) || c32isnbspace (g.ch))
- (*endptr) += g.len;
+ /* Skip a single blank, NBSP or specified unit separator.
+ Note an explicit empty --unit-sep should disable blank matching. */
+ bool matched_unit_sep = false;
+ if (unit_separator)
+ {
+ size_t sep_len = strlen (unit_separator);
+ if (STREQ_LEN (*endptr, unit_separator, sep_len))
+ {
+ matched_unit_sep = true;
+ (*endptr) += sep_len;
+ }
+ }
+ if (!matched_unit_sep)
+ {
+ mcel_t g = mcel_scanz (*endptr);
+ if (c32isblank (g.ch) || c32isnbspace (g.ch))
+ (*endptr) += g.len;
+ }
if (**endptr == '\0')
break; /* Treat as no suffix. */
@@ -761,7 +778,7 @@ double_to_human (long double val, int precision,
char *buf, idx_t buf_size,
enum scale_type scale, int group, enum round_type round)
{
- char fmt[sizeof "%'0.*Lfi%s%s%s" + INT_STRLEN_BOUND (zero_padding_width)];
+ char fmt[sizeof "%'0.*Lfi%s%s%s%s" + INT_STRLEN_BOUND (zero_padding_width)];
char *pfmt = fmt;
*pfmt++ = '%';
@@ -828,11 +845,12 @@ double_to_human (long double val, int precision,
devmsg (" after rounding, value=%Lf * %0.f ^ %d\n", val, scale_base, power);
- strcpy (pfmt, ".*Lf%s%s%s");
+ strcpy (pfmt, ".*Lf%s%s%s%s");
int prec = user_precision == -1 ? show_decimal_point : user_precision;
return snprintf (buf, buf_size, fmt, prec, val,
+ (power > 0 && unit_separator) ? unit_separator : "",
power == 1 && scale == scale_SI
? "k" : suffix_power_char (power),
&"i"[! (scale == scale_IEC_I && 0 < power)],
@@ -947,6 +965,10 @@ Reformat NUMBER(s), or the numbers from standard input if none are specified.\n\
fputs (_("\
--suffix=SUFFIX add SUFFIX to output numbers, and accept optional\n\
SUFFIX in input numbers\n\
+"), stdout);
+ fputs (_("\
+ --unit-separator=SEP insert SEP between number and unit on output,\n\
+ and accept optional SEP in input numbers\n\
"), stdout);
fputs (_("\
--to=UNIT auto-scale output numbers to UNITs; see UNIT below\n\
@@ -1555,6 +1577,10 @@ main (int argc, char **argv)
suffix = optarg;
break;
+ case UNIT_SEPARATOR_OPTION:
+ unit_separator = optarg;
+ break;
+
case DEBUG_OPTION:
debug = true;
break;
@@ -1606,6 +1632,10 @@ main (int argc, char **argv)
&& !grouping && (padding_width == 0) && (format_str == nullptr))
error (0, 0, _("no conversion option specified"));
+ if (debug && unit_separator && delimiter == DELIMITER_DEFAULT)
+ error (0, 0,
+ _("field delimiters have higher precedence than unit separators"));
+
if (format_str)
parse_format_string (format_str);
diff --git a/tests/misc/numfmt.pl b/tests/misc/numfmt.pl
index 276630483..367e52032 100755
--- a/tests/misc/numfmt.pl
+++ b/tests/misc/numfmt.pl
@@ -169,6 +169,77 @@ my @Tests =
{ERR => "$prog: invalid suffix in input: '2 K'\n"},
{EXIT => 2}],
+ ## Unit Separator
+ # Output with space separator
+ ['unit-sep-1', '--to=si --unit-separator=" " 1000', {OUT=>"1.0 k"}],
+ ['unit-sep-2', '--to=iec --unit-separator=" " 1024', {OUT=>"1.0 K"}],
+ ['unit-sep-3', '--to=iec-i --unit-separator=" " 2048', {OUT=>"2.0 Ki"}],
+
+ # Output with multi-character separator
+ ['unit-sep-4', '--to=si --unit-separator="__" 1000', {OUT=>"1.0__k"}],
+ ['unit-sep-5', '--to=iec --unit-separator="::" 2048', {OUT=>"2.0::K"}],
+
+ # Input with space separator
+ ['unit-sep-6', '-d "" --from=si --unit-sep=" " "1 K"', {OUT=>"1000"}],
+ ['unit-sep-7', '-d "" --from=iec --unit-sep=" " "2 M"', {OUT=>"2097152"}],
+
+ # Input with multi-character separator
+ ['unit-sep-8', '-d "" --from=si --unit-separator=" "',
+ {IN_PIPE=>"1 K\n2 M\n3 G\n"},
+ {OUT=>"1000\n2000000\n3000000000"}],
+ ['unit-sep-9', '--from=iec --unit-separator="'."\xC2\xA0".'"',
+ {IN_PIPE=>"4\xC2\xA0K\n"}, {OUT=>"4096"}],
+ ['unit-sep-10', '--from=iec --unit-separator="::"',
+ {IN_PIPE=>"4::K\n"}, {OUT=>"4096"}],
+
+ # input with empty separator
+ ['unit-sep-11', '-d "" --from=si --unit-separator=""',
+ {IN_PIPE=>"1K\n2M\n3G\n"},
+ {OUT=>"1000\n2000000\n3000000000"}],
+ ['unit-sep-12', '-d "" --from=si --unit-separator="" "1 K"',
+ {ERR=>"$prog: invalid suffix in input: '1 K'\n"},
+ {EXIT=>2}],
+
+ # Combined with suffix
+ ['unit-sep-13', '--to=si --unit-separator=" " --suffix=B 1000',
+ {OUT=>"1.0 kB"}],
+ ['unit-sep-14', '--to=si --unit-separator=" " --suffix=" B" 1000',
+ {OUT=>"1.0 k B"}],
+ ['unit-sep-15', '-d "" --from=si --unit-separator=" " --suffix=B',
+ {IN_PIPE=>"5 KB\n"}, {OUT=>"5000B"}],
+
+ # No separator when there's no unit (power=0)
+ ['unit-sep-16', '--to=si --unit-separator=" " 500', {OUT=>"500"}],
+
+ # Round-trip test
+ ['unit-sep-17', '--from=iec --to=iec --unit-separator="_"',
+ {IN_PIPE=>"1_K\n"}, {OUT=>"1.0_K"}],
+
+ # Currently field delimiters have higher precedence than unit separators.
+ # Even if this is changed in future, the following should hold.
+
+ # The space should act as a field delimiter here
+ ['unit-sep-18', '--from=si --unit-separator=" " "1 K_Field2"',
+ {OUT=>"1 K_Field2"}],
+ # Same as above but with 'i' suffix - should split at space with --from=si
+ ['unit-sep-19', '--from=si --unit-separator=" " "5 Ki_Field2"',
+ {OUT=>"5 Ki_Field2"}],
+ # With --from=auto, Ki followed by invalid char should also split
+ ['unit-sep-20', '--from=auto --unit-separator=" " "5 Ki_Field2"',
+ {OUT=>"5 Ki_Field2"}],
+ # With custom delimiter, space after K should not be treated as delimiter
+ ['unit-sep-21', '-d: --from=si --unit-separator=" " "5 K:Field2"',
+ {OUT=>"5000:Field2"}],
+ # Fail case: space after K with custom delimiter should error
+ ['unit-sep-22-fail', '-d: --from=si --unit-separator=" " "5 K Field2"',
+ {ERR=>"$prog: invalid suffix in input '5 K Field2': ' Field2'\n"},
+ {EXIT=>2}],
+
+ # If Unit separator consumed before delimiter char,
+ # this would change to outputting "5000 2"
+ ['unit-sep-23', '--from=si --field=1 --unit-separator=" " -d " " "5 K 2"',
+ {OUT=>"5 K 2"}],
+
## GROUPING
# "C" locale - no grouping (locale-specific tests, below)
--
2.51.0
