Thanks for confirming the examples. I figure that fleshing this out now
should speed development at a future time.
Tim Dudgeon wrote:
Ken Tanaka wrote:
Tim Dudgeon wrote:
See comments below.
Tim
...
2. each connection is explicity defined and could have extra
attributes added in future (e.g. a disable attribute to disable
execution of that part of the pipeline.
3. The concept of input can probably be generalised to include the
"feed", allowing multiple feeds to be used (as discussed earlier in
this thread). e.g. stage1 would also have an input that would be the
feed.
Do you envision a stage with two inputs (aPort and bPort) waiting
until there are inputs on both before its stageDriver invokes the
process method? If stage5 needs two inputs, and stage2 provides 3
values and stage3 provides 2 values, there are just 2 complete pairs
of values. The third value from stage2 could wait indefinitely for a
matching input from stage3. Currently stages run until their queue is
empty, but with multiple inputs that could be imbalanced, it might be
better to set the quit condition to any one queue is empty and all
upstream stages claim to be complete. Any non-empty queues on exit
can trigger a warning.
Yes, there are multiple scenarios here. e.g.
1. pairwise processing.
a1 + b1
a2 + b2
a3 + b3
...
2. combinatorial processing
a1 + b1
a1 + b2
a1 + b3
a2 + b1
a2 + b2
....
(maybe others too)
Clearly 1's inputs has to be matched, and if they are not some exit
rules would be required.
Also, as you have mentioned before there are potential memory issues
with combinatorial processing of large sets of data, but I think
solutions can be found for this.
In general I prefer to stick to scenario 1. Any deviation can be handled
by making a stage behave differently rather than putting this capability
into the framework. A stage can be written to do nothing but accept
pairwise arguments and emit combinatorial output, it can then be
inserted before any stage to which you want to add combinatorial input
processing.
Ken
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
