On Tue, Jul 31, 2018 at 10:14 AM Lu Wang <coolwan...@gmail.com> wrote:
> Hi All,
>
> Yesterday I noticed the huge memory usage by the webvfx plugin, as melt
> crashed due to "out of memory". My mlt project contains only one track,
> which contains a serious of clips, say C1, C2, ... There might be standard
> simple transitions between clips.
>
> A webvfx filter is attached for each cilp. The filter will load a
> complicated HTML page which involves 3d map etc. The page may use up to 1GB
> memory in practice.
>
> I noticed that whenever a new cilp starts being rendered, melt will use
> 1GB memory more memory. At first I thought it's some memory leak in
> QtWebKit, but after playing around with the webvfx files
> <https://github.com/mltframework/webvfx/tree/master/mlt>, I noticed that
> services (also filters, effects) are being created but never destroyed!
> From the code it seems that the objects will be destroyed when the whole
> rendering sequence is finished, but I couldn't verify that as melt always
> crashed before finishing.
>
> This means that, say when C5 is being rendered, the webvfx filters for
> C1 - C4 still exist in memory, which will actually never be used again.
>
>
The MLT execution engine does not know whether the project will be accessed
sequentially or randomly (seeking). By default, all MLT objects are held in
memory until the project is closed (as well as all mlt_frames). The
mlt_playlist takes a property called "autoclose" which if you set to 1 will
close each producer in its list after it is done. You can set this in XML
on the <playlist> element: autoclose=1. However, I suspect there are
scenarios where this will not work.
>
> I can think of only 2 possible solutions:
>
> 1. To reuse existing webvfx plugin. I might try to hack to code, to use
> some singleton service/filter/effect internally. But this is too hacky and
> may not work in general for other projects.
>
>
Some services should not hold everything it manages in memory or file
handles per process and memory will be exhausted. For that, they use
mlt_cache to store objects they are using (think buffers or library structs
- not MLT object) and keep a limited number of them open or in-memory. It
is then responsible for restoring the object when it gets a cache hit. You
can see this at work in avformat, pixbuf, and qimage producers.
> 2. To unload a plugin which is no longer needed, but I'm not sure how to
> do so. It seems that the frames are rendered individually and the filters
> are loaded on demand. Are there events/signals when a clip is fully
> rendered?
>
> What do you think?
>
>
>
> regards,
> - Lu
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Mlt-devel mailing list
> Mlt-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mlt-devel
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Mlt-devel mailing list
Mlt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlt-devel
