Let's take a step back here. Reflect for a short moment about what loader producer does and what framebuffer producer does and how it works. As an app developer, I would like to be able to change the speed and direction of a clip by not having to put "framebuffer:" in front of it. As a users, I would like to be able to change the speed and direction of a clip by simply adding query string speed parameter to the URL. However, producers like avformat also process query string parameters. Therefore, we will namespace prefix ours with mlt_ and call it mlt_speed. I can picture all of that quite easily by simply merging framebuffer into loader.
How about a way to generalize that? What if mlt_filter=yadif2x attaches the yadif2x filter to the child producer? How about mlt_filter=telecide&mlt_guide=1 to attach telecide and set its guide property to 1? Use "mlt_" to filter query string params. "mlt_filter" is keyword to attach filter; anything else sets a property on most recent filter, and order is important. Can mlt_speed be handled as a filter or is that special because there needs to be framerate normalization in the loader2 producer? Later, what can be done about audio? Forget about what to do on speed changes, but for things like reverse telecine and bob deinterlacing, it sucks that audio cannot be retained. On Thu, Dec 19, 2013 at 9:57 AM, Dan Dennedy <d...@dennedy.org> wrote: > On Thu, Dec 19, 2013 at 7:20 AM, Brian Matherly <pez4br...@yahoo.com> wrote: >>>> I'd like to implement a deinterlace mode similar to the VLC Yadifx2 filter >>>> that produces a unique frame from each field (bob) with interpolation. But >>> Does this output images at half the video's vertical resolution? >> >> For each field on the input, it outputs a full resolution frame on the >> output by interpolating missing lines. My understanding is that the >> interpolation can be quite sophisticated - using information from fields >> before and after to perform the interpolation. >> >>>> I'm not sure if this is possible in MLT because the filter would have to >>>> change the frame rate of the producer. >>>> >>>> One idea i have is to intercept the deinterlace mode in producer_avformat >>>> and attach the libavfilter yadif filter with mode=1 >>> >>> libavfilter needs more investigation into how it can fit into MLT. >>> Last time I looked, it looked like it had some internal buffering that >>> could make it tricky, but I could be wrong. Also the mapping of >>> parameters into mlt_properties needs investigation. >> >> I'll do some thinking about a generic libafilter wrapper. Last time I >> looked, there weren't many compelling filters in libavfilter that aren't >> covered by frei0r, etc. >> >> >>>> http://www.ffmpeg.org/ffmpeg-filters.html#yadif-1 >>>> But this seems like a hack. >>>> >>>> Any other ideas of how I could go about this? >>> >>> Not sure right now. Frame rate handling needs more work in general. >>> Take a look at the framebuffer producer. That serves as an example of >>> not only what could be done but also points at something that needs >>> solving in general. For example, the dgraft/telecide filter for >>> inverse telecine has frame rate reduction needs as well. Maybe >>> mlt_producer needs to be enhanced. >> >> Ok. Those were helpful examples. I've identified some constraints of the >> filter system (which I'm sure you are already well aware of): >> >> 1) A filter can not change the frame rate of a producer. >> This might be used for a deinterlace or a telecine/reverse-telecine filter. >> It could also be used to normalize the native frame rate to the profile >> frame rate (I'm not actually sure how that is done today, but I assume there >> is some magic in the producer). >> >> 2) A filter can not change the duration of a producer. >> This might be used by a slow motion filter (which would result in making the >> producer longer) or a fast forward filter (which would make the producer >> slower). >> >> 3) A filter can not get random access to arbitrary frames of a producer. >> This might be used by a reverse filter (when frame 0 is requested, the >> filter would return the last frame from the producer, etc.). > > Yes, and these limitations could also be considered features that > prevent filters from negatively affecting performance by allowing them > to be more concurrent and not impose costly seeking. When YADIF was > added I expanded the producer to include references to the previous > and next frames to make it a bit more flexible for filters that have > some temporal needs but also need to support random seeking. That > works in conjunction with the small image caching available to > producers such as avformat and decklink to eliminate the impact of the > additional frame requests around current frame. > >> I think that the framebuffer producer provides an interesting idea. It acts >> as a proxy between MLT and the real producer. I wonder if this idea could be >> extended to be fully supported in the framework. >> >> My current proposal would be: >> 1) Generalize the "cut producer" concept to allow pluggable proxy producers. >> 2) Create a new service type called "proxy_producer" (better name to be >> determined later). > > Eeks, no new service type as that will require touching upon melt and > xml and who knows what else. I now require a set of unit tests > liberally covering affected areas of on any major changes in the > framework. (I created one for properties when I did the property > animation stuff.) > >> 3) A proxy inherits the producer interface, but it takes an existing >> producer as a parameter. >> 4) Proxy producers can be chained together to accomplish various conversions >> on the producer. >> 5) Modify the framework to automatically chain proxies as necessary to >> normalize the framerate of producers (as is done now with other normalizing >> filters) >> 6) Start adding new proxies >> * Deinterlace (with field doubling) >> * Frame rate conversion >> * Reverse >> * Slow motion / fast forward >> * telecine / reverse telecine >> >> Example: 29.97i input, 24p output: >> Consider this chain: >> 1) Input producer: output = 29.97i >> 2) Deinterlace proxy: output = 59.94p (using field doubling for maximum >> temporal resolution) >> 3) Frame rate proxy: output = 24p (using frame dropping) >> The two proxies would be stacked on top of the input producer, and the rest >> of MLT would just think it is dealing with a producer that has 24p output. >> >> Creating a whole new service type may seem heavy. I can consider other >> options, like just extending the producer service with specific capabilities >> rather than making something pluggable. >> >> I'm willing to put in the wrench time to design and implement - as long as >> you are willing to check and approve my work. But I know you are busy with >> other things and may not want to take the time to babysit me. I would >> understand that. >> >> So, let me know if this is worth perusing. If not, I understand. But I do >> think a solution to these constraints would be valuable. > > I would keep it out of the framework and simpler to start. When I had > done some thinking on this in the past, I was thinking about merging > some of the logic of the framebuffer producer into the loader > producer, trying to generalize it, and adding support for audio. Then, > it could can use hints placed on the frame by something like telecide > filter where it indicates a repeated frame. (It will only work when > using filters attached to the encapsulated producer and not when > planted in the multitrack field.) You might need a way to tell this > producer to attach additional filters to the encapsulated producer, > maybe through the query string on the argument/resource URL. Or, > instead of attached filters, it uses your producer chaining idea. > Currently, a producer can be specified through a scheme prefixed to > the URL. Maybe for each additional scheme at the front it creates an > encapsulated producer recursively. > > In case you do not know, almost all producers are produced via the > loader producer, which is actually like a factory and does not > actually return its own object. It is used by the framework because > mlt_environment(MLT_PRODUCER) defaults to "loader". Make a copy of the > loader producer, say loader2, and try to do your work in there. You > can tell MLT to use by setting env var MLT_PRODUCER=loader2 until we > find it to be a suitable replacement. > >> ~Brian > -- > +-DRD-+ -- +-DRD-+ ------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk _______________________________________________ Mlt-devel mailing list Mlt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mlt-devel
