On Sun, 2009-01-11 at 00:08 +0000, Andrew Church wrote:
> >Since we're talking about radical changes, and since there is a
> >repository change in sight, what I'm proposing (and what I *really* like
> >to do) is to do a fresh start.
>
> I'm absolutely in favor of that, and agree that it's probably needed
> anyway. I try not to think about how many headaches transcode.c gave
> me. (:
Sounds good.
So, I volunteer for maintining the legacy code (the current src/*
stuff).
To unblock the code -broken right now-, when the new repo is set and
1.1.0 is finally out, I'll go ahead following the (conservative)
strategies I described in the first mail.
> This sounds like a good way to do it in principle. In fact, I don't think
> we even have to think about "stages", since we can just construct a tree
> of modules and pass frames down the tree (though it may be useful on the
> user-interface side to talk about the demux stage or the encode stage or
> whatever).
Exactly. I used the term "stage" only for (hopefully!) helping the
comprehension of the new model, by mapping it into existing concepts.
> Looking at this:
>
> >typedef struct tcprocessoritem_ TCProcessorItem;
> >struct tcprocessoritem_ {
> > TCProcessorItem *upstrem;
> > TCModule *module;
> > TCFrameBuffer *frame;
> >
> > int (*get_frame)(TCProcessorItem *P, TCFrameBuffer *frame);
> >};
> [...]
> >static int int generic_get_frame(TCProcessorItem *P, TCFrameBuffer *frame)
> >{
> > int err = TC_OK;
> > if(!P->frame) {
> > P->frame = tc_framebuffer_alloc(); /* lifetime equals to the
> > processor */
> > }
> > err = P->upstream->get_frame(P->upstream, P->frame);
> > if (!err) {
> > err = tc_module_process(P->module, P->frame, frame);
> > }
> > return err;
> >}
>
> Do you intend this generic_get_frame() to be part of the core, serving as
> a wrapper for each module's processing function?
Yes, exactly. BTW, the name "generic" derives from the fact that all
intermediate nodes, except leaves and the root, should be "traversed"[1]
by appling this function. :)
+++
One of the constraint that I've set is to require minimal or, much
better, no changes into modules -at all-.
I thought this project initially as side-project, hosted on a separate
tree (tcforge), so the total compatibility at module level was
absolutely required.
> I.e. you'd have one
> thread for each module, looping through this function until the module or
> its upstream link returns end-of-stream or error?
Yes, the core idea is this. As said, some things have still to be sorted
out exactly, but the very basic idea that the root pulls frames from
subtrees until there is data, and each node in turn pulls data from the
upstream.
Quite franly, I still not found a convincing (read: efficient) way of
handling nodes with threads, every design that I thought so far has
drawbacks.
The point I'm blocked on is how to blend together real parallelism and
efficient frame passing. If we apply the strategy I exposed verbatim in
a multi-{cpu,core} environment, I'm afraid just one node will
effectively run at any given time, while all the others are blocked in
pulling. Of course this is no good.
> My main concern is how to handle cases of multiple links in or out. The
> obvious case is for muxers or demuxers, though I could also imagine, for
> example, an audio module that took a 5.1 PCM stream as input and encoded
> it to both 5.1 AAC and plain stereo AAC. For an initial implementation
> it would probably suffice to use a pair of frame parameters (video and
> audio) for demux and mux modules, and treat everything else as a filter;
> but in the long run we'll probably need a way for each module to declare
> (dynamically, since e.g. DVDs can have any number of streams) what inputs
> and outputs it uses.
I agree. I not discussed this issue since it requires first a
(convincing) new module interface. Anyway, this has to be sorted out
too.
If we come out with a clean solution about handling multithreading and
(so-called) pull model dealing with multi-I/O shouldn't be a big deal:
The biggest problem I see is to avoid being forced to effectivly wait
for _all_ frames to come sequentially (so expending _the sum_ of the
waiting times sleeping) instead of doing that in parallel (so being
sleepy only for _the longest_ waiting time).
> >(and don't be afraid to say "you still don't get it!" if it's the
> >case ;))
> No, I think you got it this time. (: Your method is similar, anyway, and
> in fact I think I like it better than the one I proposed.
Nice :)
Bests,
--
Francesco Romani // Ikitt
http://fromani.exit1.org ::: transcode homepage
http://tcforge.berlios.de ::: transcode experimental forge
