On Sat, Mar 16, 2013 at 1:23 PM, Anton Khirnov <[email protected]> wrote: > > On Sat, 16 Mar 2013 12:34:54 +0100, Hendrik Leppkes <[email protected]> > wrote: >> If no buffer could be allocated, the unref_picture function doesn't clear >> the hwaccel_picture_private, and alloc_picture triggers an assert when its >> re-using the previously failed picture, because hwaccel_picture_private is >> still set. >> >> This patch ensures that hwaccel_picture_private is always cleared on >> buffer allocation failure. >> --- >> libavcodec/h264.c | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/libavcodec/h264.c b/libavcodec/h264.c >> index 9b196c7..9cfa574 100644 >> --- a/libavcodec/h264.c >> +++ b/libavcodec/h264.c >> @@ -364,6 +364,8 @@ static int alloc_picture(H264Context *h, Picture *pic) >> return 0; >> fail: >> unref_picture(h, pic); >> + av_buffer_unref(&pic->hwaccel_priv_buf); >> + pic->hwaccel_picture_private = NULL; >> return (ret < 0) ? ret : AVERROR(ENOMEM); >> } >> >> -- >> 1.8.1.msysgit.1 >> > > Ok, will push soonish. >
Actually, i've been thinking about this patch, and i think it would be better to move the hwaccel privdata allocation below the buffer allocation, like all other related frame data, which would automatically solve the problem. As a bonus, it would solve another issue i've encountered when i flush the decoder from within my get_buffer2 callback (which may or may not be supported, but it worked before) What do you think? -- now also sent to the list, i need to pay more attention. _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
