From: Aman Gupta <>

This patchset is not intended to be merged, rather to start a discussion about
some limitations of the current filter graph design in ffmpeg.c

My goal here is to create preview images for a video stream with ffmpeg, with
black bars automatically cropped out. Currently, to acheive this you need to run
ffmpeg twice: once with vf_cropdetect to get the crop parameters, then again 
vf_crop to apply the cropping.

After this patchset, the whole thing can be done in a single command like:

  ./ffmpeg -hide_banner -ss 2 -i 4x3-letterbox.mpg -vf cropdetect -vframes 1 -y 
-c:v jpeg2000 preview.jpg

The main problem is that video filters cannot change the height/width of a 
frame. The
resolution is calculated up-front when the filtergraph and encoder are created,
and any attempts by a vf to change the outlink height/width are essentially 
This is because the encoder AVCodecContext has already been initialized, and 
none of the
codecs in lavc can handle resolution changes on the fly.

With this patchset, I taught the jpeg2000 encoder to reinitialize itself when
it detects a resolution change. Then, instead of using vf_crop (which could 
easily be
taught to read cropdetect parameters from the AVFrame metadata dict), I changed
vf_cropdetect to export the crop parameters directly into the AVFrame crop 
However to make this have an effect, I also needed to add an 
call in between the filter graph and encoder. At the moment, 
is done right after decoding only, meaning filters cannot influence cropping 

Obviously this is a very blunt change that fixes one very specific use-case.
But I'm curious what (if any) changes could be made to allow this to work more
generally across other existing filters and encoders.

Aman Gupta (3):
  avcodec/j2kenc: reinitialize state if frame size changes
  avfilter/vf_cropdetect: don't ignore frames, export crop parameters
  ffmpeg: apply frame cropping between filter graph and encoder

 fftools/ffmpeg.c            |  4 ++++
 libavcodec/j2kenc.c         | 14 +++++++++++---
 libavfilter/vf_cropdetect.c |  9 +++++++--
 3 files changed, 22 insertions(+), 5 deletions(-)


ffmpeg-devel mailing list

Reply via email to