On Tuesday 29 September 2009 18:48:49 erisian wrote:
> I've modified my axon port so that it schedules according to a rather
> well-known message passing kernel policy:
>
> 1. The sender transmits to the receiver via a FIFO buffer.
> 2. The sender only blocks if it tries to send on a full FIFO
> 3. The receiver blocks if it reads from an empty FIFO
> 4. Transmitter and receiver can both check the state of the FIFO
> before attempting a send/receive.
> 5. The receiver can override the full buffer behavior by:
> a) asserting the "ready" boolean property before yielding, which
> will allow the sender to continue sending even on full FIFO
> b) by overriding the "ready" boolean with a "ready" boolean
> function which can take mitigating action to recover from the full
> state. Some examples might be dynamically increasing it's maximum
> FIFO size or correct from a data overrun condition by clearing out the
> FIFO and resetting the pipeline state etc
> c) by overriding the "ready" with a function and setting the size
> to 0 or 1 so as to simulate an actual callback or interrupt function.
> Though I haven't tried this yet, it was interesting that it just seems
> to naturally fall out of the code structure. (One of the reasons I
> ported axon was to avoid having to deal with nasty callback
> structures.)
>
> I might be over-architecting in feature 5, but it looked like a
> natural interrupt-like capability for out-of-band processing.
This sounds like the same decision making process we went through with Axon.
With ours we decided to set a maximum "pipe-width" for the link. That is
whilst we implemented our FIFO using a plain old python list, we recognised
that it was possible that such lists could grow without bounds in some
scenarios, cause a memory leak and go bang.
Since storage is logically owned by the recipient (inboxes are real storage,
outboxes are proxies), this means that a reciever gets to be able to change
that maximum size.
Now, in practice, it was unclear what to set that limit to. Default to 1?
Default to 10? Unlimited? Setting it too low would cause un-necessary
processing. Setting it too high could cause resource explosion.
In practice, we decided to try leaving it to unlimited and see what problems
occurred. We've found that in practice that this has been less of an issue
than expected.
Like yourself though a "Box" does have a callback. This is primarily useful in
allowing the storage of a piece of code to wake a component when something
happens. This isn't a callback that a user usually interacts with, and is
used primarily by the system to optimise CPU usage.
Probably the only other difference though is items 2 & 3. In those scenarios,
we don't block, we simply fail gracefully. As a result our code structure
tends to be more line:
for data in self.Inbox("someinbox")
doSomething(data)
for data in self.Inbox("someotherinbox")
doSomething(data)
if not self.anyReady():
self.pause()
yield 1
This is largely because you don't have the equivalent of a stackless or
greenlet .switch statement in plain cpython. The ability to be woken only
when there's data to process though means this isn't as bad as it may appear
at first glance. (If you're sleeping waiting for data on an inbox and you're
only woken when data is put in it, that's kinda what you're after :-)
> I'm writing the unit tests right now and might do one refactoring to
> clean up single responsibility and encapsulation issues.
Sounds good.
> As soon as I get something cleaned up a bit with unit tests I'll post.
If you need somewhere to post to, the offer of space in /trunk/Sketches in
kamaelia's SVN tree still stands :-)
Best Regards,
Michael.
--
http://yeoldeclue.com/blog
http://twitter.com/kamaelian
http://www.kamaelia.org/Home
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~----------~----~----~----~------~----~------~--~---
