On Wednesday 22 July 2009 20:55:20 Nathan Davis wrote:
> I just started using Kamaelia a little. First off, let me just say that I
> like the idea of wiring components together very much. That model works
> very well for certain types of applications, and the closer your code gets
> to that model the easier it is to use.
That's very nice to hear :-)
> So far, Kamaelia is the closest to my ideal for this domain.
Cool :-) Let's see if we can get it closer.
> That being said, I think there is a lot of room for improvement.
There wouldn't be any checkins ever again if it was perfect ;-)
FWIW, I also think that there's a long way to go, but based on experience
gained with the current system now, I feel it's a solid base to build on.
(which kinda surprises me :-)
> I'm sure that's a big shocker to everyone, given Kamaelia's relatively young
> age ;-).
:-)
> In particular, I find coding and wiring components to be cumbersome.
There's a reason behind this. Again, it's not shocking reasons, but there
are reasons. The key two however are these: "Explicit is better than implicit"
& "In the face of ambiguity, refuse the temptation to guess."
In order to explore the most sensible route for making coding and wiring
components less cumbersome, it's useful to cover some background on
Kamaelia and those reasons/design forces.
Let me be clear on one thing first though - I very much welcome this email
as an opening point for a discussion. I think it's a nice starting point, and
I view this sort of discussion as pretty vital for the future of a project.
After all unless people like yourself find the project *usable* and *nice*
to work with, it will be an evolutionary dead end. (same goes for other types
of people too of course :)
Beyond that, I also tend to be pragmatic. I don't believe that anyone can
see the future, beyond the obvious approach of trying to build it, and then
seeing what works. For example, in Kamaelia things that follow that model
is the co-ordinating assistant tracker is nice, but the STM code is nicer,
and should replace the fomer. Implementations that have been played with
are the most compelling arguments that can be made.
Projects need to evolve or they become irrelevant.
Given that, it's useful to outline the reasons/design forces.
* Originally it was unclear if the model would be a good idea or not.
* It was unclear if there was a need for inboxes & outboxes, or if "boxes"
alone would suffice. It turns out that from an implementation
perspective "boxes" is sufficient, but that the abstractions of inboxes
& outboxes helps, a lot. (directly helps with late wiring, protecting
youself from potentially shooting yourself in the foot, unless you
really want to (passthrough linkages) etc)
* It was unclear whether python would be a sensible choice in the
short/medium term.
* It is unclear if python (espeically cpython) is a sensible choice in the
long term. (10 year time frame)
Now those are all forces on the project, but not reasons for existence of the
project. Also, since you've started using, I'm assuming you've noticed the 3
way split of the project:
1 Axon, core tools
2 Kamaelia, big old bucket of components for building new apps
3 Kamaelia based apps, a growing collection of slowly becoming more
formalised applications.
Now, if the focus of the project had always been 1,2,3 and the focus had
been, "I want to build 1,2,3 that is the perfect 1,2,3 for python", then the
project would look very different.
However, the focus has always been 3,2,1, with the original application being
developed (or question asked) was "How can we build a system that allows us
to organically create and play with highly scalable network systems, in the
least constraining manner, which results in easily maintainable systems".
The other less obvious consequence of this is that this means the system
needs to be accessible to the vast majority of developers, crucially including
those starting their careers. The obvious model to build on is the sort of
model legions of sys admins round the world are happy with, hence
inbox/outbox (stdin/out), component (vs process), the tracker (unix
environment variables), and to try and anthropomorphise the model since any
model you can explain with little robots or act out with people is easier
(IMO) to pick up.
Pragmatically, you have to pick a language, but beyond that we didn't want to
be tied forever to that language, and it needed to support multiple modes of
concurrency. (This is similar to not wanting to be tied to any particular
operating system just because a given concurrency model on that OS is good or
cheap. Threads have very different scaling properties on different OS's and
different OS versions after all).
This means that when a decision has been taken on a particular kind of
syntactic sugar, careful thought has been taken "can this model be
implemented in another language". With the hardest case generally being
"done we do this is C or C++" (C++ is easier of course). Occasionally I'll
pick up another language as well, and see if we can implement a piece of
syntax in that language.
The key here is to allow an optimisation route in the future if needed. So far
this hasn't been needed.
As a result that's also part of the point behind the mini-axon tutorial. It
provides a core base for most core parts of kamaelia. The C++ implementation
is pretty hideous/naive from a C++ perspective, but the core is "it works".
The same can be said of the Ruby & Java implementations.
So in discussing your proposals, I don't want to constrain the discussion to
that, but the implementation will need to consider these forces:
* Should have a logical path to implementation in other languages
* Should be accessible to the average developer
* Should not preclude optimisations
* Preferably not cause a rewrite of existing code (!)
You'll note that none of this uses the word "pythonic". I don't actually like
that word, because I find it to be content free. Each person using it knows
what *they* mean by it, but generally means something different. After all,
twisted is very different from (say) pygame. Both are heavily used libraries
and could claim the other is non-pythonic, and this would be bogus.
Generally, I find that pythonic is used as a short hand for "I like that in my
python code" and "un-pythonic" is used as a short hand for "i don't like this
in my python code".
Beyond this, I personally find the argument "I'm designing this as a pythonic
library" a particularly painful one, since it's very much the tail wagging
the dog. It says "I'm picking this tool (lang), to build this other tool
(lib)", rather than "I'm solving this problem so I'm building this solution
(lib), using this tool (lang)".
For example, I can claim Kamaelia is pythonic because it matches (or is
intended to match) these lines from "import this" :
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Readability counts.
Although practicality beats purity.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
Some of these are clearly aesthetic and up for debate.
However, that means you can come right back and say it's not pythonic because
it doesn't necessarily match (depending on your point of view):
Beautiful is better than ugly.
Sparse is better than dense.
Readability counts.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
At which point, it becomes obvious that pythonic really is in the eye of the
beholder, and if you force someone to explain what they mean, we come right
back to brass tacks of "I like this" vs "I don't like this".
For me that's much more preferable, since those become concrete things which
can be worked on.
> There's too much mucking around with strings and dictionaries when it would
> be much more natural to use attributes -- it's just not 'pythonic'.
> What I propose is a new interface -- one that uses the deeper, more
> 'magical', aspects of python to make things more pythonic. This would not
> necessarily replace the existing interface -- there may be cases where that
> old interface would handle better. Instead, it could build on the existing
> interface as appropriate.
My reading of this therefore is as follows.
* You don't like the existing mechanisms like this:
self.send(thing, "named outbox")
self.recv("named inbox")
etc
And would prefer something *like* :
outbox.send( value)"
inbox.recv()
AND other syntactic sugar around managing boxes, and managing the main control
thread inside a component, etc.
If this is correct, I would actually greatly appreciate a short description of
what specific parts of kamaelia (or axon really) you find find ugly or
non-idiomatic, along with specific examples of what you find clearer.
[ Whilst this can be taken in some cultures combatively, I really appreciate
that sort of thing, since I know for every one person willing to say these
things there's lots more who don't feel able to. I may get a bruised ego, but
that's irrelevent vs improving the system :-) ]
> I could spend a lot of time describing things, but I think a simple example
> would be more efficient.
It would, but for clarity I'd prefer to see also what the traditional kamaelia
code for this example looks like. Where there's elements where it doesn't
match the existing system, that would be useful, and where it replaces or
removes things (logically speaking) that would be useful too.
> So here is a fairly simple example that illustrates how I envision
> things "should" work:
[snip example]
> Well, there it is. I think it's pretty well self-explanatory,
That's extremely useful. For what it's worth, I can't read how it maps across
to what we have at present, though I don't doubt for a second that it's clear
to you. I think/hope you find that amusing rather than frustrating! :-)
To me it's incredibly useful, and I'll follow up to this what I *think*
you're doing here, but wanted to give you some feedback as to how
I'm going to think about the ideas first, and my initial impressions
of it.
But as I said earlier, implementations and testing (eg through the
use/creation of a specialised mini-axon - one of the quickest easiest ways to
do this) how it feels/works in practice, is naturally a more important issue.
Also, I tend to prefer A *and/or* B rather than A *xor* B. It may be that
your ideas can find a place in kamaelia as a new component type that sits
happily alongside existing components.
After all, what you're describing sounds a bit like the Circuits library, and
being able to combine the two sets of ideas in a single system would be quite
neat. (It could also provide a path to making the two sorts of approaches
easier to use together - and to me *that's* pythonic ;-)
> but let me know if you have any questions.
I've got a load, but I'll try and answer them for myself first, and just
wanted to thank you for starting this discussion. I've wanted for a while to
revisit things like main loops, syntactic sugar etc.
Now that we're considering dropping support for python 2.2.late, and possibly
through 2.3 & 2.4, it opens up a number of options, in terms of the python
implementation.
> It's not perfect, but hopefully its a
> start. Let me know what you think, and if you have any suggestions.
I will do, I'll come back to this later in the week, though it may be the
weekend before I get headspace to give a decent response!
I may also get in contact with Richard Taylor incidentally who has been
looking into a (very old) system called MASCOT for a while, where channels
and pools - which have remarkable similarities to inboxes & the tracker/STM,
for a summary of those.
The reason behind that is because a) MASCOT worked b) it's been largely
forgotten, but is hugely relevant c) if we start making boxes attributes or
objects or properties or ... then we're upgrading these things in Kamaelia to
first class objects, and since there's a body of work on that, I'd be
interested to know what didn't work out and why last time someone tried
this.
I think for example, the idea of decoupled boxes or decoupled boxes
relating to items in a store could be potentially very interesting - allowing
spreadsheet like capabilities.
(Spreadsheets & unix shell pipelines being the two most commonly (and widely)
used forms of concurrency - both of which are declarative, and both of which
people use because they're useful tools, not because they think "I have a
concurrent problem" :-)
Once again, many thanks!
Regards,
Michael.
--
http://yeoldeclue.com/blog
http://twitter.com/kamaelian
http://www.kamaelia.org/Home
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~----------~----~----~----~------~----~------~--~---
