PR #23582 opened by Sebastian Ramacher (sebastinas) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23582 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23582.patch
Backport of 3ea6c2fe25 to release/7.1 for #23561. >From f12d3b1f5333c09c54deade5466d1b89adb0f566 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martin=20Storsj=C3=B6?= <[email protected]> Date: Fri, 22 Aug 2025 23:26:11 +0300 Subject: [PATCH] aacencdsp: Improve consistency with assembly, for x87 math Currently, the aacencdsp checkasm tests fails for many seeds, if the C code has been built with x87 math. This happens because the excess precision of x87 math can make it end up rounding to a different integer, and the checkasm tests checks that the output integers match exactly between C and assembly. One such failing case is "tests/checkasm/checkasm --test=aacencdsp 41" when compiled with GCC. When compiled with Clang, the test seed 21 produces a failure. To avoid the issue, we need to limit the precision of intermediates to their nominal float range, matching the assembly implementations. This can be achieved when compiling with GCC, by just adding a single cast. To observe the effect of this cast, compile the following snippet, int cast(float a, float b) { return (int) #ifdef CAST (float) #endif (a + b); } with "gcc -m32 -std=c17 -O2", with/without -DCAST. For x86_64 cases (without the "-m32"), the cast doesn't make any difference on the generated code. This cast would seem to not have any effect, as a binary expression with float inputs also would have the type float. However, if compiling with GCC with -fexcess-precision=standard, the cast forces limiting the precision according to the language standard here - according to the GCC docs [1]: > When compiling C or C++, if -fexcess-precision=standard is > specified then excess precision follows the rules specified in > ISO C99 or C++; in particular, both casts and assignments cause > values to be rounded to their semantic types (whereas -ffloat-store > only affects assignments). This option is enabled by default for > C or C++ if a strict conformance option such as -std=c99 or > -std=c++17 is used. Ffmpeg's configure scripts enables -std=c17 by default. This only helps with GCC though - the cast doesn't make any difference for Clang. (Although, upstream Clang seems to default to SSE math, while Ubuntu provided Clang defaults to x87 math.) Limiting the precision with Clang would require casting to volatile float for both intermediates here - and that does have a code generation effect on all architectures. [1] https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html --- libavcodec/aacencdsp.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/aacencdsp.h b/libavcodec/aacencdsp.h index 67836d8cf7..e901117df2 100644 --- a/libavcodec/aacencdsp.h +++ b/libavcodec/aacencdsp.h @@ -49,7 +49,7 @@ static inline void quantize_bands(int *out, const float *in, const float *scaled { for (int i = 0; i < size; i++) { float qc = scaled[i] * Q34; - int tmp = (int)FFMIN(qc + rounding, (float)maxval); + int tmp = (int)FFMIN((float)(qc + rounding), (float)maxval); if (is_signed && in[i] < 0.0f) { tmp = -tmp; } -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
