PR #23583 opened by Sebastian Ramacher (sebastinas) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23583 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23583.patch
Backport of 3ea6c2fe25 to release/8.0 for #23561. >From d96723953a41bae8e3f1022b5335476c1ac1fe05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martin=20Storsj=C3=B6?= <[email protected]> Date: Fri, 22 Aug 2025 23:26:11 +0300 Subject: [PATCH] aacencdsp: Improve consistency with assembly, for x87 math Currently, the aacencdsp checkasm tests fails for many seeds, if the C code has been built with x87 math. This happens because the excess precision of x87 math can make it end up rounding to a different integer, and the checkasm tests checks that the output integers match exactly between C and assembly. One such failing case is "tests/checkasm/checkasm --test=aacencdsp 41" when compiled with GCC. When compiled with Clang, the test seed 21 produces a failure. To avoid the issue, we need to limit the precision of intermediates to their nominal float range, matching the assembly implementations. This can be achieved when compiling with GCC, by just adding a single cast. To observe the effect of this cast, compile the following snippet, int cast(float a, float b) { return (int) #ifdef CAST (float) #endif (a + b); } with "gcc -m32 -std=c17 -O2", with/without -DCAST. For x86_64 cases (without the "-m32"), the cast doesn't make any difference on the generated code. This cast would seem to not have any effect, as a binary expression with float inputs also would have the type float. However, if compiling with GCC with -fexcess-precision=standard, the cast forces limiting the precision according to the language standard here - according to the GCC docs [1]: > When compiling C or C++, if -fexcess-precision=standard is > specified then excess precision follows the rules specified in > ISO C99 or C++; in particular, both casts and assignments cause > values to be rounded to their semantic types (whereas -ffloat-store > only affects assignments). This option is enabled by default for > C or C++ if a strict conformance option such as -std=c99 or > -std=c++17 is used. Ffmpeg's configure scripts enables -std=c17 by default. This only helps with GCC though - the cast doesn't make any difference for Clang. (Although, upstream Clang seems to default to SSE math, while Ubuntu provided Clang defaults to x87 math.) Limiting the precision with Clang would require casting to volatile float for both intermediates here - and that does have a code generation effect on all architectures. [1] https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html (cherry picked from commit 3ea6c2fe2569f3e8d6f1396982ce8bf8a3fc6e13) --- libavcodec/aacencdsp.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/aacencdsp.h b/libavcodec/aacencdsp.h index d0d86c3d70..684bbc254f 100644 --- a/libavcodec/aacencdsp.h +++ b/libavcodec/aacencdsp.h @@ -50,7 +50,7 @@ static inline void quantize_bands(int *out, const float *in, const float *scaled { for (int i = 0; i < size; i++) { float qc = scaled[i] * Q34; - int tmp = (int)FFMIN(qc + rounding, (float)maxval); + int tmp = (int)FFMIN((float)(qc + rounding), (float)maxval); if (is_signed && in[i] < 0.0f) { tmp = -tmp; } -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
