Steve Lhomme pushed to branch master at VideoLAN / VLC
Commits: 313d3f8e by Martin Storsjö at 2021-12-09T08:27:03+00:00 d3d_dynamic_shader: Fix uses of out of bounds values in MultMat43 Since 6ab9638604f3b2d36a6b2f928d3d1ee86e53c092, the matrices in the function are only 4x3, but the function kept using the fourth row of the right hand matrix. The out of bounds values were in practice multiplied with 0, which is why this wasn't initially noticed. However, 0 * <uninit> can also evaluate to NaN, if the uninitialized data happens to make the bit pattern of a NaN. In the upcoming Clang 14, the default options for float handling were changed slightly (in Clang commit https://github.com/llvm/llvm-project/commit/f04e387055e495e3e14570087d68e93593cf2918). As Clang can see that these multiplications use undefined values out of bounds, it assumes that those values are NaNs, which let Clang essentially optimize out most of D3D_SetupQuad, just setting all of quad->shaderConstants->Colorspace to NaNs. This fixes the colorspace matrix when built with Clang 14, by assuming the fourth, missing, row of the right hand matrix in MultMat43 to be [0, 0, 0, 1]. - - - - - 1 changed file: - modules/video_output/win32/d3d_shaders.c Changes: ===================================== modules/video_output/win32/d3d_shaders.c ===================================== @@ -289,27 +289,27 @@ static void MultMat43(FLOAT dst[4*3], const FLOAT left[4*3], const FLOAT right[4 FLOAT z = left[0*4 + 2]; FLOAT w = left[0*4 + 3]; // Perform the operation on the first row - dst[0*4 + 0] = (right[0*4 + 0] * x) + (right[1*4 + 0] * y) + (right[2*4 + 0] * z) + (right[3*4 + 0] * w); - dst[0*4 + 1] = (right[0*4 + 1] * x) + (right[1*4 + 1] * y) + (right[2*4 + 1] * z) + (right[3*4 + 1] * w); - dst[0*4 + 2] = (right[0*4 + 2] * x) + (right[1*4 + 2] * y) + (right[2*4 + 2] * z) + (right[3*4 + 2] * w); - dst[0*4 + 3] = (right[0*4 + 3] * x) + (right[1*4 + 3] * y) + (right[2*4 + 3] * z) + (right[3*4 + 3] * w); + dst[0*4 + 0] = (right[0*4 + 0] * x) + (right[1*4 + 0] * y) + (right[2*4 + 0] * z); + dst[0*4 + 1] = (right[0*4 + 1] * x) + (right[1*4 + 1] * y) + (right[2*4 + 1] * z); + dst[0*4 + 2] = (right[0*4 + 2] * x) + (right[1*4 + 2] * y) + (right[2*4 + 2] * z); + dst[0*4 + 3] = (right[0*4 + 3] * x) + (right[1*4 + 3] * y) + (right[2*4 + 3] * z) + w; // Repeat for all the other rows x = left[1*4 + 0]; y = left[1*4 + 1]; z = left[1*4 + 2]; w = left[1*4 + 3]; - dst[1*4 + 0] = (right[0*4 + 0] * x) + (right[1*4 + 0] * y) + (right[2*4 + 0] * z) + (right[3*4 + 0] * w); - dst[1*4 + 1] = (right[0*4 + 1] * x) + (right[1*4 + 1] * y) + (right[2*4 + 1] * z) + (right[3*4 + 1] * w); - dst[1*4 + 2] = (right[0*4 + 2] * x) + (right[1*4 + 2] * y) + (right[2*4 + 2] * z) + (right[3*4 + 2] * w); - dst[1*4 + 3] = (right[0*4 + 3] * x) + (right[1*4 + 3] * y) + (right[2*4 + 3] * z) + (right[3*4 + 3] * w); + dst[1*4 + 0] = (right[0*4 + 0] * x) + (right[1*4 + 0] * y) + (right[2*4 + 0] * z); + dst[1*4 + 1] = (right[0*4 + 1] * x) + (right[1*4 + 1] * y) + (right[2*4 + 1] * z); + dst[1*4 + 2] = (right[0*4 + 2] * x) + (right[1*4 + 2] * y) + (right[2*4 + 2] * z); + dst[1*4 + 3] = (right[0*4 + 3] * x) + (right[1*4 + 3] * y) + (right[2*4 + 3] * z) + w; x = left[2*4 + 0]; y = left[2*4 + 1]; z = left[2*4 + 2]; w = left[2*4 + 3]; - dst[2*4 + 0] = (right[0*4 + 0] * x) + (right[1*4 + 0] * y) + (right[2*4 + 0] * z) + (right[3*4 + 0] * w); - dst[2*4 + 1] = (right[0*4 + 1] * x) + (right[1*4 + 1] * y) + (right[2*4 + 1] * z) + (right[3*4 + 1] * w); - dst[2*4 + 2] = (right[0*4 + 2] * x) + (right[1*4 + 2] * y) + (right[2*4 + 2] * z) + (right[3*4 + 2] * w); - dst[2*4 + 3] = (right[0*4 + 3] * x) + (right[1*4 + 3] * y) + (right[2*4 + 3] * z) + (right[3*4 + 3] * w); + dst[2*4 + 0] = (right[0*4 + 0] * x) + (right[1*4 + 0] * y) + (right[2*4 + 0] * z); + dst[2*4 + 1] = (right[0*4 + 1] * x) + (right[1*4 + 1] * y) + (right[2*4 + 1] * z); + dst[2*4 + 2] = (right[0*4 + 2] * x) + (right[1*4 + 2] * y) + (right[2*4 + 2] * z); + dst[2*4 + 3] = (right[0*4 + 3] * x) + (right[1*4 + 3] * y) + (right[2*4 + 3] * z) + w; // x = left[3*4 + 0]; // y = left[3*4 + 1]; // z = left[3*4 + 2]; View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/313d3f8ee4691f253c301df40f023e6eb63fdf6f -- View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/313d3f8ee4691f253c301df40f023e6eb63fdf6f You're receiving this email because of your account on code.videolan.org.
_______________________________________________ vlc-commits mailing list [email protected] https://mailman.videolan.org/listinfo/vlc-commits
