Thanks for the response, Michael. First off, I think it would be helpful to explain where I'm coming from. One of my major interests is image processing, and I have a modest library of routines to perform various tasks (enhancement, segmention, labelling, etc.). (It's publicly available at https://launchpad.net/pimp if anyone's interested.) What I would like to have is a way to "wire" these routines together.
In most cases, it is convenient (at least logically) to think of the component's input as mapping to a function that gets called when the inputs change. The return value gets mapped to the output. I currently have a system based on generators that can be used to chain processors together into a pipeline. This works surprisingly well, but has the significant shortcoming that it can not represent arbitrary graphs -- it's pretty much restricted to pipelines. Most of the time, a DAG would probably be sufficient. Some cases may call for feedback loops, however. It's also worth noting that a single output may be connected to more than one input. I bring this up mainly because doing this in Kamaelia seems to require the use of Backplanes. On Fri, Jul 24, 2009 at 4:59 AM, Michael Sparks <[email protected]> wrote: > On Wednesday 22 July 2009 20:55:20 Nathan Davis wrote: > [snip] > > > In particular, I find coding and wiring components to be cumbersome. > > There's a reason behind this. Again, it's not shocking reasons, but there > are reasons. The key two however are these: "Explicit is better than > implicit" > & "In the face of ambiguity, refuse the temptation to guess." > I'm not trying to suggest components should automagically wire themselves up. I'm just trying to make it easier to code. [snip] > Beyond that, I also tend to be pragmatic. I don't believe that anyone can > see the future, beyond the obvious approach of trying to build it, and then > seeing what works. For example, in Kamaelia things that follow that model > is the co-ordinating assistant tracker is nice, but the STM code is nicer, > and should replace the fomer. Implementations that have been played with > are the most compelling arguments that can be made. > IMHO, some of the most complicated (and subsequently worst) designs I've made were when I tried to anticipate the future too much. I've personally come to the conclusion that it's best to code for today, but keep things flexible so you can adapt. [snip] > Pragmatically, you have to pick a language, but beyond that we didn't want > to > be tied forever to that language, and it needed to support multiple modes > of > concurrency. (This is similar to not wanting to be tied to any particular > operating system just because a given concurrency model on that OS is good > or > cheap. Threads have very different scaling properties on different OS's and > different OS versions after all). > > This means that when a decision has been taken on a particular kind of > syntactic sugar, careful thought has been taken "can this model be > implemented in another language". With the hardest case generally being > "done we do this is C or C++" (C++ is easier of course). Occasionally I'll > pick up another language as well, and see if we can implement a piece of > syntax in that language. > I'm not sure what you mean here. Anything that can be implemented in one turing-complete language can be implemented in any other. The syntax may be wildly different. It may be vastly easier/more difficult to implement. But the end result is the same. So I'm not sure what your concern is -- it is inevitable that implementations in different languages will be different even if they are functionally equivalent. So you may as well use the features of each language to your advantage. > > The key here is to allow an optimisation route in the future if needed. So > far > this hasn't been needed. > > As a result that's also part of the point behind the mini-axon tutorial. It > provides a core base for most core parts of kamaelia. The C++ > implementation > is pretty hideous/naive from a C++ perspective, but the core is "it works". > The same can be said of the Ruby & Java implementations. > > So in discussing your proposals, I don't want to constrain the discussion > to > that, but the implementation will need to consider these forces: > * Should have a logical path to implementation in other languages Well, .NET properties are the closest that come to mind. But you can do getters/setters in virtually any language. > > * Should be accessible to the average developer I'm not sure what you mean by 'accessible'. But I think appropriate use of descriptors/properties is a well-established practice within the python community (think SQLAlchemy, Elixir, various parts of Django ... even python methods). Using established practices is one way to reduce the learning curve and help developers keep the framework 'in their brain.' > > * Should not preclude optimisations I don't think this will be an issue with what I'm currently envisioning. However, I agree that performance is important and should take priority (though features are important as well -- it's all about the balance). > > * Preferably not cause a rewrite of existing code (!) > Actually, I've already written a partial implementation of my ideas that builds on top of Kamaelia in its current form. The initial results are very encouraging, and I haven't seen anything so far that would indicate we would need to make changes that would break existing code. I'll post the code for the group shortly, after I've cleaned it up a little. > > You'll note that none of this uses the word "pythonic". I don't actually > like > that word, because I find it to be content free. Each person using it knows > what *they* mean by it, but generally means something different. After all, > twisted is very different from (say) pygame. Both are heavily used > libraries > and could claim the other is non-pythonic, and this would be bogus. > > Generally, I find that pythonic is used as a short hand for "I like that in > my > python code" and "un-pythonic" is used as a short hand for "i don't like > this > in my python code". > Point taken, though you'll probably have to remind a few times ;-) [snip] > > There's too much mucking around with strings and dictionaries when it > would > > be much more natural to use attributes -- it's just not 'pythonic'. > > What I propose is a new interface -- one that uses the deeper, more > > 'magical', aspects of python to make things more pythonic. This would > not > > necessarily replace the existing interface -- there may be cases > where that > > old interface would handle better. Instead, it could build on the > existing > > interface as appropriate. > > My reading of this therefore is as follows. > * You don't like the existing mechanisms like this: > self.send(thing, "named outbox") > self.recv("named inbox") > etc > > And would prefer something *like* : > outbox.send( value)" > inbox.recv() > That's part of it. My original intention was that assignment to an outbox would basically do a send() (or wire it up, depending on the context) and getting an inbox would get its latest value. However, I think your example is a better match to the Kamaelia community as a whole. > > AND other syntactic sugar around managing boxes, and managing the main > control > thread inside a component, etc. > Yes, managing boxes is the main issue -- particularly wiring. Pipeline helps when you only need to chain components together. It's clear and concise -- the way I like it! Things get a lot messier, though, when you need Graphlines. Some of this is inevitable -- the thing you are describing is more complex. The easier it is, however, the better. I think one way to do this is to let the user wire components through attribute assignment. > > If this is correct, I would actually greatly appreciate a short description > of > what specific parts of kamaelia (or axon really) you find find ugly or > non-idiomatic, along with specific examples of what you find clearer. > There are a few things: a) wiring -- as discussed above b) inboxes/outboxes -- minor, but I prefer to deal with python attributes and objects as much as possible (i.e., I don't like passing strings to send() and rec()). > > [ Whilst this can be taken in some cultures combatively, I really > appreciate > that sort of thing, since I know for every one person willing to say these > things there's lots more who don't feel able to. I may get a bruised ego, > but > that's irrelevent vs improving the system :-) ] > > > I could spend a lot of time describing things, but I think a simple > example > > would be more efficient. > > It would, but for clarity I'd prefer to see also what the traditional > kamaelia > code for this example looks like. Where there's elements where it doesn't > match the existing system, that would be useful, and where it replaces or > removes things (logically speaking) that would be useful too. > Ok, I have some code that does roughly the same thing in Kamaelia. I'll post it shortly. In the meantime, it's basically just a counter that starts at 0 and increments each step until it is reset. There's also a Printer component that does a print of its input, and a FireOn component that is used to fire a "signal" when its input reaches a user-specified condition (in this case, it's used to reset the counter when the counter reaches 10). Note that I originally introduced the concept of a "signal" because I was wanting inputs to retain the value it was last set to. A "signal", on the other hand, would be used to indicate some event occured, and could be connected to a function that would be called when the signal "fired". However, I've since realize that this sort of specialization is better suited to a seperate, optional layer that users can use if they want. > > > So here is a fairly simple example that illustrates how I envision > > things "should" work: > > [snip example] > > > Well, there it is. I think it's pretty well self-explanatory, > > That's extremely useful. For what it's worth, I can't read how it maps > across > to what we have at present, though I don't doubt for a second that it's > clear > to you. I think/hope you find that amusing rather than frustrating! :-) > By "can't read how it maps across", do you mean you are having difficulty understanding the intent of the code, or that you just don't see how it would be implemented with Kamaelia, or possibly both? I was hoping that at least understanding the intent would be fairly easy. I understand, though, that it's not easy to pick up someone else's code and comprehend it. > > To me it's incredibly useful, and I'll follow up to this what I *think* > you're doing here, but wanted to give you some feedback as to how > I'm going to think about the ideas first, and my initial impressions > of it. > > But as I said earlier, implementations and testing (eg through the > use/creation of a specialised mini-axon - one of the quickest easiest ways > to > do this) how it feels/works in practice, is naturally a more important > issue. > > Also, I tend to prefer A *and/or* B rather than A *xor* B. It may be that > your ideas can find a place in kamaelia as a new component type that sits > happily alongside existing components. > As I said earlier, I already have a partial implementation started. It's currently based on AdaptiveCommsComponent (i.e., my Component class subclasses AdaptiveCommsComponent). The reason I used AdaptiveCommsComponent is simply because it was easier to use addInbox/addOutbox than bother setting up Inboxes, etc. before calling the constructor. Anyway, I'll post the code in a bit, after I've cleaned it up a little. The biggest "problem" so far is handling multiple consumers for an output. I solved this by creating a backplane for each output, but that doesn't seem efficient! It also increases message latency significantly. Can you suggest a better way to handle this, or would it be possible to change the core so that plugging an output into multiple inputs would work in this manner? > > After all, what you're describing sounds a bit like the Circuits library, > and > being able to combine the two sets of ideas in a single system would be > quite > neat. (It could also provide a path to making the two sorts of approaches > easier to use together - and to me *that's* pythonic ;-) > Do you mean this circuits: http://trac.softcircuit.com.au/circuits/? I've only taken a quick look, but the "wiring" mechanism there looks quite a bit different (but maybe it's just superficial). > > > but let me know if you have any questions. > > I've got a load, but I'll try and answer them for myself first, and just > wanted to thank you for starting this discussion. I've wanted for a while > to > revisit things like main loops, syntactic sugar etc. > Good, I look forward to hearing them! > > Now that we're considering dropping support for python 2.2.late, and > possibly > through 2.3 & 2.4, it opens up a number of options, in terms of the python > implementation. > Mmm yes, that could be a bit of an issue. I *think* this should all (mostly) work with Python 2.2, but I'd have to check to be certain. [snip] > > I may also get in contact with Richard Taylor incidentally who has been > looking into a (very old) system called MASCOT for a while, where channels > and pools - which have remarkable similarities to inboxes & the > tracker/STM, > for a summary of those. > > The reason behind that is because a) MASCOT worked b) it's been largely > forgotten, but is hugely relevant c) if we start making boxes attributes or > objects or properties or ... then we're upgrading these things in Kamaelia > to > first class objects, and since there's a body of work on that, I'd be > interested to know what didn't work out and why last time someone tried > this. > Is there a website for this? Is it open source? > > I think for example, the idea of decoupled boxes or decoupled boxes > relating to items in a store could be potentially very interesting - > allowing > spreadsheet like capabilities. > > (Spreadsheets & unix shell pipelines being the two most commonly (and > widely) > used forms of concurrency - both of which are declarative, and both of > which > people use because they're useful tools, not because they think "I have a > concurrent problem" :-) > > Once again, many thanks! > > Regards, > > > Michael. > -- > http://yeoldeclue.com/blog > http://twitter.com/kamaelian > http://www.kamaelia.org/Home > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "kamaelia" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/kamaelia?hl=en -~----------~----~----~----~------~----~------~--~---
