Am Do., 28. Mai 2026 um 20:13 Uhr schrieb Friedrich Romstedt <
[email protected]>:
> Any idea, comment or pointer would be welcome very much. I'd like to sort
> this problem thoroughly.
>
So I discussed the topic with ChatGPT, and we found a viable solution.
In essence, the problem boils down two the following situation:
1. Let there be a basis *b*, either 1000 for SI prefixes, 1024 for IEC
prefixes, or 10 for the remaining SI prefixes corresponding to 10^{-2,-1,
1, 2}.
2. Let there further be a *prefix exponent, p*, denoting the exponent of
*b* for a given prefix.
3. An finally, let there be a *unit exponent* *u*, indicating the
exponent the entire prefix has in the unit (e.g. -2 for a unit of the
form '1 / _m^2', where the underscore '_' indicates the position of the
prefix).
Then two numbers can be deduced:
1. The *numerical value* N = 1000 ** (p * u). This is the *numerical
value* handed over by the user, the value for which the best-fitting
prefix shall be determined. The name *numerical value* is a little
misleading, it refers to the *numerical value* combined with the *unit* to
the *quantity*.
2. The overall *prefix value*, P = (1000 ** p) ** u. This is the number
which is compared to N.
Mathematically, both are equal, but technically P and N can be a little
different, e.g. in the SI case for p = -2 and u = -2. These corner cases I
am interested in here.
To decide about the applicability of a prefix, P is compared to N as N >= P.
In the example p = -2 & u = -2, however N is slighly smaller than P.
ChatGPT proposed to iterate about all possible combinations of p and u. For
each pair, 1000.0 ** (p * u) can be compared to (1000.0 ** p) ** u. I am
comparing the difference of P and N, P - N, to the sum S of the two ULPs
deduced for P and N (like X - numpy.nextafter(X, 0)). More specifically, I
am calculating an "error coefficient" r = (P - N) / S. I did so both for SI
as well as for IEC prefixes.
The unit exponent u is always in [-3.0, -2.5, ..., 2.0. 2.5, 3.0].
For SI, p is in [-10.0, -9.0, ..., 9.0, 10.0]. Here, the maximum r is 1.0
(seldom), more often 0.5, most often 0; sometimes -0.5, rarely -1.0, one
time -1.5. S is the sum of the two ULPs, and apparently the ULPs are equal
for P and N, leading to the results for r being an integer multiple of 0.5.
For 10-based SI units (d, c, ...), the results are similar, but
bounded by [-0.5,
0.5].
With less regular exponents, e.g. with p = 3.0 and u = -2.1, r reaches
8.00. With even more irregular values, I am reaching r = 38.50 (p approx.
-8.95, u = 3.0).
For the IEC prefixes, I obtained no deviations between N and P at all
(p in [0.0,
1.0, ..., 7.0, 8.0]).
So my issue is sorted. I will install 2 * S as the safety margin when
comparing a *numerical value* to an *overall prefix value*, and this safety
margin will suffice in all regular situations tested.
Ideally, I would put this report on my web site and just link to it here,
but this web site is not yet there, so I am posting my results here, to
provide the solution of my problem.
Friedrich
_______________________________________________
NumPy-Discussion mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3//lists/numpy-discussion.python.org
Member address: [email protected]
