On Oct 3, 2006, at 7:09 PM, Daniel Stenning wrote:
While waiting for my mac to run a particularly long task, I did some
browsing and came across an Icelandic company CCP that wrote and
runs a
massively multiplayer online game called "Eve". It says in its
technical
blurb that it uses a special version of Python to achieve high
concurrency
and parallelism - called "Stackless" python. And states this
allowed them to
gain much better performance and scaling with regard to parallelism ,
concurrency etc.
Did a a little brief reading , but does anyone out there know
about this?
Is this something that we could benefit from in RB?. It seems to be a
"Python" thing at the moment only. But I doubt if theres anything
fundamental to stop it from being used in other languages..
A little programming language history: back in the day (60s-70s)
there were all kinds of wacky languages with all kinds of language
structures. Then along came C (and some other languages, particularly
early BASICs) and they established various conventions. New, popular
languages such as Python and Ruby are rediscovering some of these
alternative language structures from 30 years ago.
One particular stricture that C introduced that everyone now thinks
is perfectly natural is that transferring control from method to
method should be via a stack. The advantage of this is that it is
fast and easy to manage the memory on the stack. So this made sense
back then. But the disadvantages now thoroughly outweight the
advantages.
The effect you see in a language like REALbasic is that as you call
from method to method to method, if you don't ever return whence you
came, eventually you run out of stack space and you get an error.
The feature that Stackless Python implements is the coroutine. This
is like the methods you know and love, except that you aren't
required to return from them. It gives you a different type of call
from one method to another, that just transfers control without
mucking about with the stack, suspending the one making the call and
resuming the one being called, right where it left off. Under the
hood, each method keeps its local variables on the heap, which is
where all your object properties and the like live, and where things
can be created and destroyed in any order, limited only by available
memory. So you can just transfer control from one to another for as
long as you like, and you're not building up anything anywhere.
You could get almost the same thing, and I've proposed this to the
REAL Software engineers, if you had a call that, in one step,
suspended the current thread, and activated another, presumably
suspended, thread. Then, you could transfer control from thread to
thread in a similar way.
An example of where this is useful: you are doing communications
between programs. You have some logic where you want to fetch
something, do some processing, fetch another thing, and so on. You
get your results back in an event, not in the code where you want to
think about this processing and fetching.
As things stand, you have to break this simple fetch-act-fetch logic
so it's scattered in multiple places. You'll have a socket object
which always receives the information coming in, through an Event
Handler, and it has to dispatch it to some other objects for
processing. Those objects, rather than having simple fetch-act-
fetch-... code, have to store their state in variables so they know
which step they were up to so they can resume it when the socket
sends them the next part. The simple logic of what you want to do has
to get bent entirely out of shape, because of the stack based language.
If, otoh, we had coroutines, the logic could be (coroutine call-and-
suspend)-(resume and process)-(call-and-suspend)... The socket could
just wake up the thread when the information it needs is available,
and it just carries on where it was, then it passes on its fetch
request and goes to sleep again. Nothing is bent out of shape.
Many algorithms, and particularly the logic of communicating
applications, can be an order of magnitude simpler when you have
coroutines available. I don't know the innards of Ruby on Rails, but
I've thought for ten years or more that sooner or later, a language
would come along that offered coroutines, and it would make writing
web serving applications much easier. And, ta-da! Ruby has coroutines
(in fact, it offers something even more general, called a
continuation, but we won't go into the differences here).
Finally, I'll note that Stackless Python was a fork of mainstream
Python, and the latest release of Python has added coroutines. They
may be a little more limited than those of Stackless; I need to look
into that.
Regular method call-and-return may seem the most natural thing in the
world to you right now. But if you're writing applications that do a
lot of communicating, consider signing up for:
<http://www.realsoftware.com/feedback/viewreport.php?reportid=qipjlsck>
Guyren G Howe
Relevant Logic LLC
guyren-at-relevantlogic.com ~ http://relevantlogic.com
REALbasic, PHP, Python programming
PostgreSQL, MySQL database design and consulting
Technical writing and training
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>
Search the archives of this list here:
<http://support.realsoftware.com/listarchives/lists.html>
