Re: [FFmpeg-devel] [PATCH 1/2] avcodec/hevc_ps: Fix integer overflow with num_tile_rows and num_tile_columns

2019-06-30 Thread Michael Niedermayer
On Wed, Jun 26, 2019 at 10:29:05AM -0300, James Almer wrote: > On 6/26/2019 9:41 AM, Michael Niedermayer wrote: > > On Tue, Jun 25, 2019 at 10:30:45AM -0300, James Almer wrote: > >> On 6/25/2019 5:55 AM, Michael Niedermayer wrote: > >>> Fixes: signed integer overflow: -2147483648 - 1 cannot be

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/hevc_ps: Fix integer overflow with num_tile_rows and num_tile_columns

2019-06-26 Thread James Almer
On 6/26/2019 9:41 AM, Michael Niedermayer wrote: > On Tue, Jun 25, 2019 at 10:30:45AM -0300, James Almer wrote: >> On 6/25/2019 5:55 AM, Michael Niedermayer wrote: >>> Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in >>> type 'int' >>> Fixes: >>>

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/hevc_ps: Fix integer overflow with num_tile_rows and num_tile_columns

2019-06-26 Thread Michael Niedermayer
On Tue, Jun 25, 2019 at 10:30:45AM -0300, James Almer wrote: > On 6/25/2019 5:55 AM, Michael Niedermayer wrote: > > Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in > > type 'int' > > Fixes: > >

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/hevc_ps: Fix integer overflow with num_tile_rows and num_tile_columns

2019-06-25 Thread James Almer
On 6/25/2019 10:30 AM, James Almer wrote: > On 6/25/2019 5:55 AM, Michael Niedermayer wrote: >> +num_tile_columns_minus1 >= sps->width - 1) { > > Should be sps->ctb_width > > From 7.4.3.3.1: > > "num_tile_columns_minus1 plus 1 specifies the number of tile columns > partitioning the

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/hevc_ps: Fix integer overflow with num_tile_rows and num_tile_columns

2019-06-25 Thread James Almer
On 6/25/2019 5:55 AM, Michael Niedermayer wrote: > Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in type > 'int' > Fixes: > 14880/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5130977304641536 > > Found-by: continuous fuzzing process >

[FFmpeg-devel] [PATCH 1/2] avcodec/hevc_ps: Fix integer overflow with num_tile_rows and num_tile_columns

2019-06-25 Thread Michael Niedermayer
Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int' Fixes: 14880/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5130977304641536 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: