Hi Folks:
Message: 3
Date: Wed, 15 Jul 2009 09:42:58 +0000
From: Kristj?n Valur J?nsson <[email protected]>
To: Kristj?n Valur J?nsson <[email protected]>, Larry Dickson
<[email protected]>
Cc: Henning Diedrich <[email protected]>,
"[email protected]" <[email protected]>
Subject: Re: [Stackless] irc threads
Message-ID:
<[email protected]>
Content-Type: text/plain; charset="iso-8859-1"
>select() as such is recognized as scaling badly, even on unix() which is >why
>(I've been told) there are now alternatives for asynchronous IO >commonly
>available on linux.
My understanding is that select() is slow when it comes to large number of
descriptors due to the copying of file descriptors and excessive scanning of
descriptors. I believe on Linux, epoll() is a preferred alternative. To me,
signals just looks messy....
>Also, I'd like to add that we don't generally want to block, even for
>>select(). In our framework, the application may have numerous tasks to
>do when it is not servicing IO. This necessitates using select to poll
>all the sockets, with the associated setup overhead.
This was a part of the problem I encountered two years ago - the reactor
tasklet (which just happened to be Twisted) would block the application until
it received input. Consequently other tasklets that could run, did not run.
Luckily this problem was relatively easy to solve....
That said Kristjan, I think it is a balancing act to effectively do network I/O
and use a single OS thread. From the tests I have so far conducted, I think one
of the main tradeoffs is between response time and transactions per some time
unit.
One of these days, I will conduct more structured tests to see how sensitive
performance is, all other things being equal, to the following factors:
1) The I/O system call (i.e., select(), epoll())
2) The number of context switches (controlled through channel preferences)
3) The number of times a particular I/O call is made (how frequently do we call
select() - this addresses setup overhead).
4) The use of an optimizer such as Psyco on the network (read reactor tasklet).
Cheers,
Andrew
_______________________________________________
Stackless mailing list
[email protected]
http://www.stackless.com/mailman/listinfo/stackless
