Re: [PATCH] amdgpu/vcn: vcn dec_msg integer overflow fix

[Alex Deucher]

On Fri, Aug 7, 2026 at 4:20 PM David (Ming Qiang) Wu <david....@amd.com> wrote:
>
> if supplied msg[2] in the header is too large (around
> 0x40000000 and above), 4 times of this unsigned 32 bit value
> will overflow and the test could pass.
>
> Widen it with uint64_t instead. Also drop the redundant
> first check and make the check clear that 6 dwords of
> header plus 4 dwords of each msg cannot exceed the overall
> packet size in dwords.

Maybe cleaner to use check_mul_overflow() and just return an error in that case?

Alex

>
> Signed-off-by: David (Ming Qiang) Wu <david....@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 2 +-
>  drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c 
> b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
> index 81bba3ec2a93..06dcf736b15a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
> @@ -1965,7 +1965,7 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, 
> struct amdgpu_job *job,
>         num_buffers = msg[2];
>
>         /* Verify that all indices fit within the claimed length. Each index 
> is 4 DWORDs */
> -       if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
> +       if ((uint64_t)6 + (uint64_t)num_buffers * 4 > len_dw) {
>                 DRM_ERROR("VCN message has too many buffers!\n");
>                 r = -EINVAL;
>                 goto out;
> diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c 
> b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
> index 0cce78b205a8..7950a020f4e1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
> @@ -1881,7 +1881,7 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, 
> struct amdgpu_job *job,
>         num_buffers = msg[2];
>
>         /* Verify that all indices fit within the claimed length. Each index 
> is 4 DWORDs */
> -       if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
> +       if ((uint64_t)6 + (uint64_t)num_buffers * 4 > len_dw) {
>                 DRM_ERROR("VCN message has too many buffers!\n");
>                 r = -EINVAL;
>                 goto out;
> --
> 2.43.0
>