Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Wed, Nov 22, 2017 at 10:56:07AM -0800, Dale Curtis wrote: > On Tue, Nov 21, 2017 at 7:05 PM, Michael Niedermayer > wrote: > > > > I dont think wrap_bits can/should be > 64 or do i miss something ? > > > > Good point, this seems true with the current code. > > > > >

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Tue, Nov 21, 2017 at 7:05 PM, Michael Niedermayer wrote: > > I dont think wrap_bits can/should be > 64 or do i miss something ? > Good point, this seems true with the current code. > > maybe a av_assert* for that would be better. > Done; used av_assert2().

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Tue, Nov 21, 2017 at 03:19:38PM -0800, Dale Curtis wrote: > Ah, realized this approach can work for wrap_bits == 64 too. Updated the > patch. > > On Mon, Nov 20, 2017 at 5:42 PM, Dale Curtis > wrote: > > > On Mon, Nov 20, 2017 at 2:24 PM, Michael Niedermayer < > >

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

Ah, realized this approach can work for wrap_bits == 64 too. Updated the patch. On Mon, Nov 20, 2017 at 5:42 PM, Dale Curtis wrote: > On Mon, Nov 20, 2017 at 2:24 PM, Michael Niedermayer < > mich...@niedermayer.cc> wrote: > >> >> I think that could end with the correct

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Mon, Nov 20, 2017 at 2:24 PM, Michael Niedermayer wrote: > > I think that could end with the correct result > > Thanks for the review. Done. - dale From fc7fb3511aa40810e64d9dacbd33d1e9336d0c52 Mon Sep 17 00:00:00 2001 From: Dale Curtis

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Mon, Nov 20, 2017 at 01:15:24PM -0800, Dale Curtis wrote: > On Mon, Nov 20, 2017 at 12:40 PM, Michael Niedermayer < > mich...@niedermayer.cc> wrote: > > > On Mon, Nov 20, 2017 at 12:05:05PM -0800, Dale Curtis wrote: > > > On Sat, Nov 18, 2017 at 2:44 AM, Michael Niedermayer > >

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Mon, Nov 20, 2017 at 12:40 PM, Michael Niedermayer < mich...@niedermayer.cc> wrote: > On Mon, Nov 20, 2017 at 12:05:05PM -0800, Dale Curtis wrote: > > On Sat, Nov 18, 2017 at 2:44 AM, Michael Niedermayer > > > wrote: > > > > > > this would skip the code for wrap_bits

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Mon, Nov 20, 2017 at 12:05:05PM -0800, Dale Curtis wrote: > On Sat, Nov 18, 2017 at 2:44 AM, Michael Niedermayer > wrote: > > > > this would skip the code for wrap_bits >= 63, this does not look > > correct > > > > Why do you think that's incorrect? The max int64_t

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Sat, Nov 18, 2017 at 2:44 AM, Michael Niedermayer wrote: > > this would skip the code for wrap_bits >= 63, this does not look > correct > Why do you think that's incorrect? The max int64_t value is 1 << 63 and 2 << 63 == 1 << 64 ? - dale

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

On Fri, Nov 17, 2017 at 01:46:39PM -0800, Dale Curtis wrote: > Derp, actually, 2 << 63 doesn't fit in int64_t either, this check should be > < 63. Fixed. > > > > On Fri, Nov 17, 2017 at 1:38 PM, Dale Curtis > wrote: > > > 2 << (wrap_bits=64 - 1) does not fit in

Re: [FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

Derp, actually, 2 << 63 doesn't fit in int64_t either, this check should be < 63. Fixed. On Fri, Nov 17, 2017 at 1:38 PM, Dale Curtis wrote: > 2 << (wrap_bits=64 - 1) does not fit in int64_t; apply the check > used in other places that handle wrap bits to ensure the