Hi,
Richard Spindler schrieb:
[...]
Linux Video Editing:
Open Problems
Problem #1: Detect Interlacing in Videofiles/streams, through Image Analysis
Subproblems:
* Detect 3x2 Pulldown, vs. Video.
* Topfield first vs. Bottomfield first.
Should be detected by the source (e.g. the codec). libquicktime usually is
quite reliable when detecting the interlacing mode. For the very few cases,
where a source is misflagged, a simple filter (which does nothing but change
the interlacing flags) would be enough.
Relevant Links:
* http://silicontrip.net/~mark/lavtools/
This one is also pretty good:
http://mjpeg.cvs.sourceforge.net/mjpeg/mjpeg_play/yuvdeinterlace/
It lets you choose if you want to double the framerate or not. Not realtime
capable though.
Possible Problems:
* Later Editing could introduce sudden changes in pulldown sequence
Possible Approches:
* Compare fields, to detect duplicates, which is common in
pulleddown material.
IMO pulldown should be detected and removed at the beginning of the pipeline.
It will also speed up things by descreasing the number of images.
Filters might change the image data such that later comparison of fields will
fail.
Problem #2: Develop Algorithms and Processing code, for not-upsampled
YUV 4:2:0 or 4:1:1 Source Footage. Common Filters and Operations.
Chroma-Keyers, etc.
Either the algorithms work out of the box with subsampled material
(e.g. Hue, Brightness, Saturation, Contrast, Scaling) or they need to
interpolate (== upsample) the missing chroma values and downsample
them again after processing.
Now if you have a pipeline of multiple filters, which do
upsampling -> processing -> downsampling it's better to usample at the
beginning of
the pipeline and downsample at the very end (both speed and quality wise).
Problem #4: General Purpose Image Processing Library, accellerated
using OpenGL Graphics Hardware.
Ideas: Processing Graph, Automating Parameters, Different Colorspaces,
Packed vs. Unpacked, Software Fallbacks for unavailable Hardware
features, Scaling, Aspect-Ratios, Interlacing, take model of OpenGL
Shaders into account.
IMO, a pure software solution (in C) should be the *reference* not just
a *fallback*. OpenGL is an optional acceleration (pretty much like MMX
and SSE).
If you make OpenGL the "default", you will loose the possibility to
use Algorithms (e.g. better scaling methods), which are not available
under GL.
Problem #6: Temporally compressed Video processing, MPEG, GOPs
Subproblem 1#: „Give me Frame 10"! No matter whether B, P or I
Frame. No consideration of Performance.
This is handled by libquicktime and (AFAIK) by quicktime4linux as well.
Needless to say, that editing temporally compressed sources is suboptimal
to begin with...
* Index Building (File index, endianess neutral)
Different editors using the same index format would be cool...
To make some self-advertisement:
gavl (http://gmerlin.sourceforge.net/gavl.html)
already handles 26 different pixelformats (RGB(A), YUV(A), most common
subsampling
modes, different precisions ranging from 15 bpp RGB to floating point.
Included are
- Converters for all pixelformats (better than the one in libquicktime both
speed and quality wise)
- A generic scaler with subpixel precision, which can also correct chroma
placement,
resample chroma planes and do simple deinterlacing. Also supports vertical
scaling
of interlaced images :)
New (in CVS for now) is
- A method to (mis)use the scaler as a generic FIR filter, which can do
arbitrary
(x-y seperable) convolutions, like blurring, sharpening etc.
- Routines for comparing frames (using sum of absolute differences)
- Routines for interpolating frames.
All routines work for all pixelformats directly. Many conversion and scaling
routines
are available in MMX optimized versions or in (slower) high quality versions.
You can
pass a quality value (1..5) which will select the proper routines.
Of course I'm biased here, but I'm pretty sure, that gavl is quite unique with
respect
to it's features. Even if it's suboptimal (in which case I like to know why), it
saves you a *lot* of work.
Burkhard
_______________________________________________
Cinelerra mailing list
[email protected]
https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra
