Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On Sun, Dec 6, 2015 at 6:36 PM, Andreas Cadhalpun wrote: > The other is a regression since 01ecb71, so I hope you know how to fix that. > In search_for_pns in libavcodec/aaccoder.c: > for (w = 0; w < sce->ics.num_windows; w += sce->ics.group_len[w]) { > [...]

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On Wed, Dec 9, 2015 at 4:42 PM, Andreas Cadhalpun wrote: >>> [...] >>> for (w2 = 0; w2 < sce->ics.group_len[w]; w2++) { >>> [...] >>> } > > Now we are after the w2-loop and thus: > w2 = sce->ics.group_len[w] = 2 > Ah, I see, it's outside

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On 09.12.2015 21:38, Claudio Freire wrote: > On Wed, Dec 9, 2015 at 5:29 PM, Claudio Freire wrote: >> On Wed, Dec 9, 2015 at 4:42 PM, Andreas Cadhalpun >> wrote: > [...] > for (w2 = 0; w2 < sce->ics.group_len[w]; w2++)

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On Wed, Dec 9, 2015 at 5:29 PM, Claudio Freire wrote: > On Wed, Dec 9, 2015 at 4:42 PM, Andreas Cadhalpun > wrote: [...] for (w2 = 0; w2 < sce->ics.group_len[w]; w2++) { [...] } >> >> Now we are

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On 09.12.2015 19:08, Claudio Freire wrote: > On Sun, Dec 6, 2015 at 6:36 PM, Andreas Cadhalpun > wrote: >> The other is a regression since 01ecb71, so I hope you know how to fix that. Let me add example numbers to the code. That should make the problem obvious.

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On 05.12.2015 02:58, Claudio Freire wrote: > Alright, I see what's going on. minscaler-maxscaler and/or q0-q1 are > empty ranges, so that results in no possible solutions. > > I pushed a fix for that, leaving your "safety net" intact. Thanks. > You may wish to re-fuzz ;) I did that and found

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On Fri, Dec 4, 2015 at 2:23 PM, Andreas Cadhalpun wrote: > If minq is negative, the range of sf_idx can be larger than > SCALE_MAX_DIFF allows, causing assertion failures later in > encode_scale_factors. > > Signed-off-by: Andreas Cadhalpun

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On 04.12.2015 23:49, Claudio Freire wrote: > On Fri, Dec 4, 2015 at 2:23 PM, Andreas Cadhalpun > wrote: >> If minq is negative, the range of sf_idx can be larger than >> SCALE_MAX_DIFF allows, causing assertion failures later in >> encode_scale_factors. >> >>

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On Fri, Dec 4, 2015 at 9:21 PM, Andreas Cadhalpun wrote: > On 04.12.2015 23:49, Claudio Freire wrote: >> On Fri, Dec 4, 2015 at 2:23 PM, Andreas Cadhalpun >> wrote: >>> If minq is negative, the range of sf_idx can be larger than

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On 05.12.2015 01:32, Claudio Freire wrote: > On Fri, Dec 4, 2015 at 9:21 PM, Andreas Cadhalpun > wrote: >> On 04.12.2015 23:49, Claudio Freire wrote: >>> On Fri, Dec 4, 2015 at 2:23 PM, Andreas Cadhalpun >>> wrote: If minq

Re: [FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

On Fri, Dec 4, 2015 at 9:52 PM, Andreas Cadhalpun wrote: > Pushed. > >> Do you have the problematic input at hand? If so, send it privately. > > Sure, I'll send you a sample. > >> If I find a better solution I may try to push that instead, or at >> least add the

[FFmpeg-devel] [PATCH] aaccoder: prevent crash of anmr coder

If minq is negative, the range of sf_idx can be larger than SCALE_MAX_DIFF allows, causing assertion failures later in encode_scale_factors. Signed-off-by: Andreas Cadhalpun --- libavcodec/aaccoder.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff