On 12/3/07, Gustavo Sverzut Barbieri <[EMAIL PROTECTED]> wrote:
> On Dec 3, 2007 1:41 PM, Nathan Ingersoll <[EMAIL PROTECTED]> wrote:
>
> I'd like to have the minimum overhead with the safest approach, for
> that I'd like to have just one "worker" process, if it crashes (or
> take too long to reply) it would be restarted... but fork+exec
> everytime is a no go... general usecase (not just canola) is to
> thumbnail jpeg/png, these can be considered pretty safe/stable and
> wouldn't cause much trouble... however using emotion to provide movie
> thumbnails or an svg/edje reader may cause problems due the
> flexibility to do bad things :-P
Just to be clear, I'm not advocating fork+exec for each thumbnail for
every case. My thinking is more along the lines that we keep thumbnail
generation for core formats (jpeg, png) inside the primary worker
thread. The difference is rather than requiring a new plugin for each
format we want to support, allow for fork+exec style generation for
specified file formats in order to re-use existing infrastructure.
This could also be improved if some external handlers could take a
batch of arguments. We can certainly provide plugins for formats that
we support, which could improve efficiency, but we can't expect to
support all file types.
Thumbnailing is also a task that fits well in a parallel model, since
the task has a fair amount of latency reading from disk. By using
multiple workers you can take advantage of this latency and have CPU
bound processes while others are IO bound. Also, since the tasks are
independent they can be scaled out nicely across multiple cores. Both
of these aspects may be irrelevant for Canola, since it is running off
a flash based drive on single core CPU.
> I'm not really found of DBus, actually I was planning to have a
> hand-written system to do the communication instead of ecore_{ipc,con}
> because the messages are really simple and can be done to fit a fixed
> size, much easier and fast to handle, also processing on peers could
> avoid alloc/free and thus not trash the memory. If these are well
> encapsulated like epsilon is today, I see no problem. Also, by having
> such hand-written message system I'd not require ecore at all for the
> server process, just the worker, it's good to have things as light as
> possible.
While I agree that keeping things light is important, I would like to
know your reasoning behind why dbus is not a reasonable candidate in
this case? I don't have any data to back it up, but I find it hard to
believe that the communication overhead of dbus would be anywhere near
the relative slowness of the actual thumbnail generation. As long as
you can queue requests, the latency should be completely dominated by
waiting for the resulting thumbnail. The IPC speed should really only
be a problem if you are making round-trip requests and waiting on the
result before the next request.
> I doubt it worth the pain, because if we go FDO we must remove any
> attempt to use .edj as thumbnail format, and it's a good alternative
> if you want to provide movie thumbnail. Going "too generic" may hurt
> more than help if nobody else use it, and believe me these guys are
> hard to get using done by others, you know the NIH syndrome...
Except what you're encouraging is exactly NIH (not that we have any
lack of that around here). What is the advantage of using edje for
storing generated thumbnails vs using the filesystem? The main
advantage of edje is efficient bundling of data into a single file,
but with thumbnails, they are more tied to their original source than
with other surrounding files. Also, edje writes out the entire eet
each time a save is done, this means that you will now be writing out
every enclosed thumbnail instead of the single one that is updated.
It's difficult to say if no one else would use it if we're not talking
to anyone else about it. I think it is worth opening a discussion with
FDO members with a base spec as a point of reference.
> As for INdT today, yes, it won't help, but it's good to discuss the
> long term so we can help. Probably I'll hack some nasty epsilon2 to
> get it done in time, but I'll not even commit to CVS, it'll stay in
> staff.get-e.org GIT... after we have something good for the long term,
> then I can help with it and then convert canola to use it.
Oops, I got the capitalization scheme of INdT wrong again. This sounds
like a reasonable plan for your short-term issue.
Good luck with your release.
-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell. From the desktop to the data center, Linux is going
mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
enlightenment-devel mailing list
enlightenment-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
