Hi Kristjan:
Message: 1
Date: Mon, 28 Mar 2011 19:53:19 +0800
From: Kristj?n Valur J?nsson <[email protected]>
To: The Stackless Python Mailing List <[email protected]>
Subject: Re: [Stackless] Thoughts on I/O Driven Scheduler
Message-ID:
<[email protected]>
Content-Type: text/plain; charset="iso-8859-1"
>I actually think the discussion about the performance of soft switching >vs
>hard switching to be irrelevant. Tasklet switching in any meaningful
>>application happens so rarely that it doesn't show up on the radar. If >your
>application spends any measurable time just switching contexts, then >you
>ought to think about changing your algorithm.
Depends on the architecture and the type of application one is building. For
the most part, I think meaningful applications sit around and wait. Again, I
speak from the highly theoretical world of BPEL like applications - something I
have only written small examples to test features rather than performance.
However if one has a lot of small agents communicating over channels, then
context switching ought to matter. When the numbers are big enough, non-value
producing activity like context switches will add up. Maybe an extreme example
is the sieve of Eratosthenes.
>Now, stackless however was supposed to be able to use its soft switching
>>algorithm to simplify execution of regular python code (by being >stackless).
> I have no idea whether this turned out to be useful in >practice.
Well it seems to me what you are really questioning is whether synchronous
message passing is useful. I say yes but I am biased. In my PyCon 2011 talk, I
use select() and channels to write the Santa Claus problem. If I threw in join
patterns, the solution would be shorter yet. In contrast, look at 'solutions'
that use mutexes and semaphores. And we are dealing with only twenty-one
threads. The rub is that they are interacting in a non-trivial fashion.
I read Go-Lang Nuts. A lot of people don't get this paradigm. You still see
demands for low level stuff like mutexes and what not.
I believe the applications that will most benefit from Stackless are still in
their infancy....
>As for real world data, I added instrumentation to our Stackless python
>>branch a while back. I was trying to optimize this time by modifying the
>>stack 'cache'.
>Below is a snapshot from one of our game proxies. We have some 10 times >more
>hard switches than soft switches. Despite this, hard switching >accounts for
>0.3% of wallclock time.
>The process is running at some 13%cpu, so that means that hard switching >for
>us is consuming some 0.03% of application time.
My silly mailer scrubs the png. But I have no doubt about the validity of your
data. I suspect most of the time is spent waiting. Also it would be interesting
to know how many tasklets were running.
Still, if anything, I wish I was far more competent at instrumenting Stackless
code and interpreting results.
Cheers,
Andrew
_______________________________________________
Stackless mailing list
[email protected]
http://www.stackless.com/mailman/listinfo/stackless
