On 11/15/05, Bryan Mayland wrote: > Geoffrey Hausheer wrote: > With debug enabled it faults in the avcodec_encode_video() function > (specifically in the dct_quantize_SSE2 method). Here's a backtrace from > the thread: > (gdb) bt ... > #6 0xb71d022a in avcodec_encode_video (avctx=0x84555f0, > buf=0xffffff80 <Address 0xffffff80 out of bounds>, buf_size=614400, > pict=0x8140f60) at utils.c:868 > #7 0x0804c838 in MPEG2fixup::build_frame (this=0xbfa71084, pkt=0xbfa70c2c, > fname=0x0) at mpeg2fix.cpp:929 > #8 0x0804ce67 in MPEG2fixup::convert_to_i (this=0xbfa71084, frameNum=0, > numFrames=2, headPos=0) at mpeg2fix.cpp:1194 > #9 0x08050699 in MPEG2fixup::start (this=0xbfa71084) at mpeg2fix.cpp:1458 > #10 0x08051a89 in main (argc=7, argv=0xbfa72404) at mpeg2fix.cpp:1775 > I don't understand why buf == -128 that isn't an allowed output from malloc. Have you tried replacing the malloc with a memalign(16, outbuf_size)? there may be some stack corruption that is causing buf to be invalid after the call to avccodec_encode_video The same may go for intra_matrix. It could be aligned (I think) via: uint16_t intra_matrix[64] ATTR_ALIGNED(16);
> Opening /mnt/store/1015_20051108183000.mpg > [mpeg2video @ 0xb7454f04]libavcodec: CPU flags: mmx mmxext sse sse2 > Input #0, mpeg, from '/mnt/store/1015_20051108183000.mpg': > Duration: N/A, bitrate: N/A > Stream #0.0[0x1e0], 29.97 fps: Video: mpeg2video, yuv420p, 640x480, > 6000 kb/s > Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 384 kb/s > #0 PTS:36039 Delta: 0.000000ms queue: 11 > #1 PTS:34685 Delta: 15.044444ms queue: 2 > Mux rate: 6.49 Mbit/s > [mpeg2video @ 0xb7454f04]libavcodec: CPU flags: > Warning, QMAT_SHIFT is larger then 21, overflows possible > Converting frame 0 to an I-frame ((null)) This looks normal (though I still have no idea what those QMAT_SHIFT messages mean) .Geoff _______________________________________________ mythtv-dev mailing list [email protected] http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev
