On Oct 4, 11:48 am, Michael Sparks <[email protected]> wrote:
> On Tuesday 29 September 2009 18:48:49 erisian wrote:
>
>
>
> > I've modified my axon port so that it schedules according to a rather
> > well-known message passing kernel policy:
>
> > 1. The sender transmits to the receiver via a FIFO buffer.
> > 2. The sender only blocks if it tries to send on a full FIFO
> > 3. The receiver blocks if it reads from an empty FIFO
> > 4. Transmitter and receiver can both check the state of the FIFO
> > before attempting a send/receive.
> > 5. The receiver can override the full buffer behavior by:
> > a) asserting the "ready" boolean property before yielding, which
> > will allow the sender to continue sending even on full FIFO
> > b) by overriding the "ready" boolean with a "ready" boolean
> > function which can take mitigating action to recover from the full
> > state. Some examples might be dynamically increasing it's maximum
> > FIFO size or correct from a data overrun condition by clearing out the
> > FIFO and resetting the pipeline state etc
> > c) by overriding the "ready" with a function and setting the size
> > to 0 or 1 so as to simulate an actual callback or interrupt function.
> > Though I haven't tried this yet, it was interesting that it just seems
> > to naturally fall out of the code structure. (One of the reasons I
> > ported axon was to avoid having to deal with nasty callback
> > structures.)
>
> > I might be over-architecting in feature 5, but it looked like a
> > natural interrupt-like capability for out-of-band processing.
>
> This sounds like the same decision making process we went through with Axon.
> With ours we decided to set a maximum "pipe-width" for the link.
Yes.
> That is
> whilst we implemented our FIFO using a plain old python list, we recognised
> that it was possible that such lists could grow without bounds in some
> scenarios, cause a memory leak and go bang.
>
Yes. My inboxes are simply templated (type-specific) linked lists,
but I set a nominal max size for them.
> Since storage is logically owned by the recipient (inboxes are real storage,
> outboxes are proxies), this means that a reciever gets to be able to change
> that maximum size.
>
> Now, in practice, it was unclear what to set that limit to. Default to 1?
> Default to 10? Unlimited? Setting it too low would cause un-necessary
> processing. Setting it too high could cause resource explosion.
>
I set it to DEFAULT_INBOX_SIZE. :-)
(which defaults to 32)
> In practice, we decided to try leaving it to unlimited and see what problems
> occurred. We've found that in practice that this has been less of an issue
> than expected.
>
Interesting. I thought I'd set it to a "nice" power of two, which
hopefully won't be re-sized too often. My thinking was that poorly
sized-inboxes shouldn't happen very often and would require some sort
of receiver fixup if it ever happened. Mostly I was concerned with
situations where some microprocesses might be created and destroyed
often. I wanted to avoid possible situations where the garbage
collector had a big mess to clean up.
Given, this is a "seat of the pants" experience thing, but that's why
I allow the receiver to take action on its own if its "really
interested". If 32 seems to small or to large for a particular
implementation they can always recompile the lib or make it a config
value.
>
> Probably the only other difference though is items 2 & 3. In those scenarios,
> we don't block, we simply fail gracefully. As a result our code structure
> tends to be more line:
> for data in self.Inbox("someinbox")
> doSomething(data)
> for data in self.Inbox("someotherinbox")
> doSomething(data)
> if not self.anyReady():
> self.pause()
> yield 1
>
Yes. While I could also wrap the producers in a generator facade
(probably a future feature), I wanted to put as much of the
self.pausing mechanisms in the kernel as possible. (Idea being that
having to do an explicit yield is enough of a wart in a cooperative
system.)
> This is largely because you don't have the equivalent of a stackless or
> greenlet .switch statement in plain cpython.
True enough. I like python, but sometimes you want access to the
metal.
> The ability to be woken only
> when there's data to process though means this isn't as bad as it may appear
> at first glance. (If you're sleeping waiting for data on an inbox and you're
> only woken when data is put in it, that's kinda what you're after :-)
>
Correct, though you still have the *option* of detecting the state of
the inbox and taking requisite action.
> If you need somewhere to post to, the offer of space in /trunk/Sketches in
> kamaelia's SVN tree still stands :-)
>
I certainly will.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~----------~----~----~----~------~----~------~--~---
