Hi Dan,
Thanks for your reply!
I guess it's not trivial in my case. My project mlt file is generated by
shotcut, where each clip is a producer, and there's only 1 (or 2, where the
other is just a wrapper) playlist. So the "autoclose" trick might not work.
But maybe I can refactor the mlt file for this trick.
Regarding cache, I don't think it'll help either, as webvfx/qtwebkit
objects are big individually and not aware of the cache. Maybe it'll help
if all the webpage objects share the (web/http) resource pool, but I'm not
sure, also I heard of memory leak issues with qtwebkit.
regards,
- Lu
On Tue, Jul 31, 2018 at 7:34 PM, Dan Dennedy <d...@dennedy.org> wrote:
> On Tue, Jul 31, 2018 at 10:14 AM Lu Wang <coolwan...@gmail.com> wrote:
>
>> Hi All,
>>
>> Yesterday I noticed the huge memory usage by the webvfx plugin, as melt
>> crashed due to "out of memory". My mlt project contains only one track,
>> which contains a serious of clips, say C1, C2, ... There might be standard
>> simple transitions between clips.
>>
>> A webvfx filter is attached for each cilp. The filter will load a
>> complicated HTML page which involves 3d map etc. The page may use up to 1GB
>> memory in practice.
>>
>> I noticed that whenever a new cilp starts being rendered, melt will use
>> 1GB memory more memory. At first I thought it's some memory leak in
>> QtWebKit, but after playing around with the webvfx files
>> <https://github.com/mltframework/webvfx/tree/master/mlt>, I noticed that
>> services (also filters, effects) are being created but never destroyed!
>> From the code it seems that the objects will be destroyed when the whole
>> rendering sequence is finished, but I couldn't verify that as melt always
>> crashed before finishing.
>>
>> This means that, say when C5 is being rendered, the webvfx filters for
>> C1 - C4 still exist in memory, which will actually never be used again.
>>
>>
> The MLT execution engine does not know whether the project will be
> accessed sequentially or randomly (seeking). By default, all MLT objects
> are held in memory until the project is closed (as well as all mlt_frames).
> The mlt_playlist takes a property called "autoclose" which if you set to 1
> will close each producer in its list after it is done. You can set this in
> XML on the <playlist> element: autoclose=1. However, I suspect there are
> scenarios where this will not work.
>
>
>>
>> I can think of only 2 possible solutions:
>>
>> 1. To reuse existing webvfx plugin. I might try to hack to code, to use
>> some singleton service/filter/effect internally. But this is too hacky and
>> may not work in general for other projects.
>>
>>
> Some services should not hold everything it manages in memory or file
> handles per process and memory will be exhausted. For that, they use
> mlt_cache to store objects they are using (think buffers or library structs
> - not MLT object) and keep a limited number of them open or in-memory. It
> is then responsible for restoring the object when it gets a cache hit. You
> can see this at work in avformat, pixbuf, and qimage producers.
>
>
>> 2. To unload a plugin which is no longer needed, but I'm not sure how to
>> do so. It seems that the frames are rendered individually and the filters
>> are loaded on demand. Are there events/signals when a clip is fully
>> rendered?
>>
>> What do you think?
>>
>>
>
>>
>> regards,
>> - Lu
>>
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______
>> _________________________________________
>> Mlt-devel mailing list
>> Mlt-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/mlt-devel
>>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Mlt-devel mailing list
Mlt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlt-devel
