Hi all, In Kdenlive, we often have projects with multiple tracks, and track compositing is one of the bottleneck we have with MLT.
Shotcut uses the frei0r.cairoblend transition for compositing. cairoblend first requests an rgba image for the top (b_frame), and checks if it really has transparency with an "is_opaque" function that check the alpha value of every pixel in the image. The downside is that if we have an opaque video on the top track, this causes a noticeable slowdown when playing. In the qtblend transition, I tried to work around it be first requesting the top (b frame) image in the consumer requested format (usually yuv). Then we check if the frame has an alpha channel using the "mlt_frame_get_alpha" function. If there is no alpha, we directly return the frame, otherwise we do another request for an rgba image to process. The advantage over cairoblend is that there is almost no overhead when the top track contains an opaque video. The downside is that there is a noticeable overhead when there is an alpha channel as we first request an yuv frame and then an rgba. So to improve this an try to reduce yuv <> rgb conversions, I was thinking to add a property to all producers "producer_get_frame" method (that is defined before we attempt to fetch an image), to inform the framework about the default mlt_image_format produced by the producer. For example, the image producer would set the "format" property to mlt_image_rgba if it is an image with alpha channel, and mlt_image_rgb if there is no alpha channel. This information could then be retrieved by the framework to optimize the frame conversion processes. In the case of my qtblend transition, if we know that the producer creates rgba images by default, we can directly retrieve the rgba frame, leading to much better performance. Of course, effects can change the format of a frame, adding or removing an alpha channel, but in many cases this would allow us to optimize the performance. What do you think of this ? If the idea seems ok, I volunteer to produce a patch implementing the feature (I already tested it for qimage and kdenlivetitle producers). If you don't like the idea, do you have any better idea to implement a good performance transition that would work efficiently for both cases (top frames with and without alpha channel) ? Thanks in advance and best regards, Jean-Baptiste _______________________________________________ Mlt-devel mailing list Mlt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mlt-devel
