On 06/02/2011 04:59 PM, Ronald S. Bultje wrote:
Hi John,
On Thu, Jun 2, 2011 at 4:50 PM, John Stebbins<[email protected]> wrote:
On 06/02/2011 04:42 PM, Ronald S. Bultje wrote:
Hi John,
On Thu, Jun 2, 2011 at 4:39 PM, John Stebbins<[email protected]>
wrote:
On 06/02/2011 03:31 PM, Ronald S. Bultje wrote:
On Thu, Jun 2, 2011 at 3:29 PM, John Stebbins<[email protected]>
wrote:
Fair enough. There appear to be some cases remaining where av_malloc
is
called with a size of 0. I'll set a break point and find out where the
request is coming from.
Thanks, if you can just show me a backtrace (I just run commands and
set a if (size == 0) abort() in av_malloc), I'll gladly accept the
fix.
The first occurrence of the problem that I find is in vc1_decode_init.
avctx->mb_stride gets set (from avctx->width) when
ff_msmpeg4_decode_init
is called. But at the point ff_msmpeg4_decode_init is called
avctx->width
hasn't been set yet (i.e. it is 0). width gets set in
vc1_decode_sequence_header which is called after ff_msmpeg4_decode_init.
Since width is 0, mb_stride is 0 and av_malloc gets called with 0.
I don't know exactly how to untangle this because I don't know the
dependencies between these initialization functions. Can anyone make any
suggestions?
Feel free to send me a file that triggers the problem and I'll happily
fix it for you.
I think this would take more than just a sample file to reproduce. I am
reproducing this with HandBrake which has it's own TS demux. So we are not
using libavformat in this case and are allocating and initializing
AVCodecContext, initializing extradata, and calling avcodec_open ourselves.
We've always noticed a little weirdness with vc1 in this bit of code. We
have to call avcodec_open twice to get it to initialize properly due to this
order of operations problem in vc1_decode_init.
Which allocation specifically?
There is a collection of av_malloc's in vc1_decode_init
/* Allocate mb bitplanes */
v->mv_type_mb_plane = av_malloc(s->mb_stride * s->mb_height);
v->direct_mb_plane = av_malloc(s->mb_stride * s->mb_height);
v->acpred_plane = av_malloc(s->mb_stride * s->mb_height);
v->over_flags_plane = av_malloc(s->mb_stride * s->mb_height);
v->n_allocated_blks = s->mb_width + 2;
v->block = av_malloc(sizeof(*v->block) * v->n_allocated_blks);
v->cbp_base = av_malloc(sizeof(v->cbp_base[0]) * 2 * s->mb_stride);
v->cbp = v->cbp_base + s->mb_stride;
v->ttblk_base = av_malloc(sizeof(v->ttblk_base[0]) * 2 * s->mb_stride);
v->ttblk = v->ttblk_base + s->mb_stride;
v->is_intra_base = av_malloc(sizeof(v->is_intra_base[0]) * 2 *
s->mb_stride);
v->is_intra = v->is_intra_base + s->mb_stride;
v->luma_mv_base = av_malloc(sizeof(v->luma_mv_base[0]) * 2 * s->mb_stride);
v->luma_mv = v->luma_mv_base + s->mb_stride;
/* allocate block type info in that way so it could be used with
s->block_index[] */
v->mb_type_base = av_malloc(s->b8_stride * (s->mb_height * 2 + 1) +
s->mb_stride * (s->mb_height + 1) * 2);
I fixed MPV_common_init(), which is
doing most allocs here. The correct thing to do is to re-run
MPV_common_init() after the sequence header has been read, can you
experiment with that and see if that fixes it? You may have to
MPV_common_end() first to prevent memory leaks.
The actual call chain to get to MPV_common_init is
vc1_decode_init->ff_msmpeg4_decode_init->ff_h263_decode_init->MPV_common_init. It is necessary to call
ff_msmpeg4_decode init again instead of MPV_common_init directly because ff_msmpeg4_decode init uses mb_height after
MPV_common_init sets it. Here's a patch that shows what I did. With this patch applied, decoding works, I don't have
to call avcodec_open twice, and I no longer get av_malloc(0).
I've test this with mpeg2, h.264, vc-1 video and ac3, dts, truehd, dts-hd audio codecs in TS files (basically a bunch of
BD m2ts files). So it's looking pretty solid.
diff --git a/libavcodec/vc1dec.c b/libavcodec/vc1dec.c
index 5c931b1..9391001 100644
--- a/libavcodec/vc1dec.c
+++ b/libavcodec/vc1dec.c
@@ -3491,6 +3491,14 @@ static av_cold int vc1_decode_init(AVCodecContext *avctx)
}
v->res_sprite = (avctx->codec_tag == MKTAG('W','V','P','2'));
}
+ // MPV_common_init must be called again to initialize mb_stride
+ // since width isn't set till headers are parsed.
+ avctx->coded_width = avctx->width;
+ avctx->coded_height = avctx->height;
+ MPV_common_end(s);
+ if(ff_msmpeg4_decode_init(avctx) < 0)
+ return -1;
+
avctx->profile = v->profile;
if (v->profile == PROFILE_ADVANCED)
avctx->level = v->level;
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
