On Fri, May 23, 2025 at 02:45:59AM +0200, Michael Niedermayer wrote: > Hi Ronald > > On Thu, May 22, 2025 at 07:59:06AM -0400, Ronald S. Bultje wrote: > > Hi, > > > > On Wed, May 21, 2025 at 9:34 AM Timothée < > > timothee.informati...@regaud-chapuy.fr> wrote: > > > > > Hello, > > > > > > I am interested in expanding ffmpeg's capabilities to extract > > > low-level data from video codecs. Specifically, I'd like to implement > > > functionality that would allow exporting frame data, macroblock > > > information, quantization tables, and similar codec-specific elements > > > to binary files for further analysis. > > > > > > After searching through the documentation and existing features, I > > > haven't found similar functionality, though I may have missed > > > something. Has this been implemented before, or are there related > > > features I should examine? > > > > > > Some older codecs implement minor variants for this, e.g. grep > > for AV_FRAME_DATA_MOTION_VECTORS, which attaches a frame's motion vectors > > to the picture data. I believe there's an example app and possibly a filter > > to overlay MVs on top of the video frame based on this concept. You could > > extend this to cover other (macro)block info. There used to be a variant of > > this for quant-tables also but I can't find it, maybe it was removed. > > For motion vectors: > ./ffplay -flags2 +export_mvs -i matrixbench_mpeg2.mpg -vf > codecview=mv=pf+bf+bb > > For macroblock segmentation and type vissualization + also motion vectors: > ffplay-3.4.13 -debug vis_mb_type matrixbench_mpeg2.mpg -vf > codecview=mv=pf+bf+bb > > For QP vissualization + also motion vectors: > ffplay-3.4.13 -debug vis_qp matrixbench_mpeg2.mpg -vf codecview=mv=pf+bf+bb > > For qp values dumped on the console > ./ffplay -debug qp -i matrixbench_mpeg2.mpg
And this can easily be extended to other codecs, ATM it should work with all 16x16 MB based codecs like msmpeg4*/wmv*/mpeg1/2/4/h263/h264 mbtype and qp vissualization need codecview to be extended or versions around 3.4 which implemented it differently Implementing vissualization as done currently with sidedata and codecview is simple and efficient. It also would allow exporting the data to json by writing a codec2json filter in place of codecview Also all decoders already have all this data parsed and available so its simpler than trying to do it in a decoder independant way I would thus suggest implementations of this for modern codecs to follow the same path as the existing code. thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB "You are 36 times more likely to die in a bathtub than at the hands of a terrorist. Also, you are 2.5 times more likely to become a president and 2 times more likely to become an astronaut, than to die in a terrorist attack." -- Thoughty2
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
