Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0

2018-07-20 Thread Thomas Guillem
On Thu, Jul 19, 2018, at 18:27, Aman Gupta wrote: > On Fri, Jul 6, 2018 at 10:55 PM Thomas Guillem wrote: > > > > > > > On Thu, Jul 5, 2018, at 20:49, Aman Gupta wrote: > > > On Wed, Jul 4, 2018 at 3:05 AM Thomas Guillem wrote: > > > > > > > On macOS, a zero rc_max_rate cause an error from > >

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0

2018-07-19 Thread Aman Gupta
On Fri, Jul 6, 2018 at 10:55 PM Thomas Guillem wrote: > > > On Thu, Jul 5, 2018, at 20:49, Aman Gupta wrote: > > On Wed, Jul 4, 2018 at 3:05 AM Thomas Guillem wrote: > > > > > On macOS, a zero rc_max_rate cause an error from > > > VTSessionSetProperty(kVTCompressionPropertyKey_DataRateLimits). >

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0

2018-07-06 Thread Thomas Guillem
On Fri, Jul 6, 2018, at 09:21, Kári Helgason wrote: > Sorry for jumping in with a slightly OT question, > > Am I understanding correctly that you have noticed that when using kVTCompre > ssionPropertyKey_DataRateLimits you will sometimes get random garbage from > the encoder in the output callbac

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0

2018-07-06 Thread Thomas Guillem
On Thu, Jul 5, 2018, at 20:49, Aman Gupta wrote: > On Wed, Jul 4, 2018 at 3:05 AM Thomas Guillem wrote: > > > On macOS, a zero rc_max_rate cause an error from > > VTSessionSetProperty(kVTCompressionPropertyKey_DataRateLimits). > > > > on iOS (depending on device/version), a zero rc_max_rate cau

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0

2018-07-06 Thread Kári Helgason
Sorry for jumping in with a slightly OT question, Am I understanding correctly that you have noticed that when using kVTCompre ssionPropertyKey_DataRateLimits you will sometimes get random garbage from the encoder in the output callback? How frequently does this happen, and can you reproduce it r

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0

2018-07-05 Thread Aman Gupta
On Wed, Jul 4, 2018 at 3:05 AM Thomas Guillem wrote: > On macOS, a zero rc_max_rate cause an error from > VTSessionSetProperty(kVTCompressionPropertyKey_DataRateLimits). > > on iOS (depending on device/version), a zero rc_max_rate cause invalid > arguments from the vtenc_output_callback after few

Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0

2018-07-04 Thread Thomas Guillem
On Wed, Jul 4, 2018, at 09:05, Thomas Guillem wrote: > On macOS, a zero rc_max_rate cause an error from > VTSessionSetProperty(kVTCompressionPropertyKey_DataRateLimits). > > on iOS (depending on device/version), a zero rc_max_rate cause invalid > arguments from the vtenc_output_callback after few

[FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0

2018-07-04 Thread Thomas Guillem
On macOS, a zero rc_max_rate cause an error from VTSessionSetProperty(kVTCompressionPropertyKey_DataRateLimits). on iOS (depending on device/version), a zero rc_max_rate cause invalid arguments from the vtenc_output_callback after few frames and then a crash within the VideoToolbox library. --- l