Hello, Good news for stacklesssocket :)
I would like to know more about the inconsistencies when dealing with Threads in Stackless. I think I have bumped into one or more of them. Cheers, Alejandro 2010/8/22 Richard Tew <[email protected]>: > Hi, > > For anyone who is using stacklesssocket.py, I've just upgraded it so > that it passes the non-UDP Python standard library socket unit tests > (tested on 2.6 and 2.7). Note that it actually passes all of the UDP > unit tests except for two, so that's a good sign that the changes I > have made make it inherently more correct. I would also add that the > tests do not pass on 2.5, but this looks to be due to minor > exception/behaviour related changes and it should work as well as it > ever did with 2.5. > > http://code.google.com/p/stacklessexamples/source/browse/trunk/examples/networking/stacklesssocket.py > > I still have to do stress testing work on these changes. > > Some relevant things you might want to know about the changes: > - The way it worked with the socket module and asyncore and garbage > collection was to have a dictionary ('asyncore.socket_map') which weak > references to tasklets were stored in. This was bad, as it meant that > dead weak references would sit in the dictionary until events > propagated and they were removed. This in turn would cause calls to > 'asyncore.poll(seconds)' to have reference errors raised out of it > under load as it choked on those dead weak references. Now it > replaces 'asyncore.socket_map' with a WeakValueDictionary and when > weak references die, they are removed immediately and there no more > reference errors happen in the described way. > - If you set 'stacklesssocket._sleep_func' to your sleep function, > then the socket timeout mechanism will become enabled and take > advantage of it (e.g. 'my_socket.settimeout(num_seconds'). You can > also implement your own timeout mechanism with a > 'stacklesssocket._timeout_func', but that is a little more complex and > suits in the situation where you doing interthread communication with > channels and wish to avoid the concerns that come with tasklets > belonging to different threads and channel operations. > - You can now call 'my_socket.setblocking(0)' and it will work as expected. > - A wider swatch of the socket object API is now supported, with > regard to weird functions like recv_into and the like. > - No internal buffering of data, which means it should be more > optimal. You get the data the socket provides directly (well, as > direct as an asynchronous operation can be made to look synchronous > using channels). > > Most of the work in this change was in contriving a script > ('stacklesssocket_unittest.py') that would set up an environment where > the Python unit tests could be run with the stacklesssocket module > monkey patched into place. This involved writing a module > ('stacklessthread.py') that turns a range of thread blocking > operations into tasklet blocking operations, when they happen on the > main thread (where the expectation is that the scheduler is only being > run on that thread). While over the years we have fleshed out > Christian's threading support for special Stackless functionality, > there are still a few inconsistencies and when you are dealing with > Python threads and Stackless they tend to pop up. This is the reason > why 'stacklesssocket_unittest.py' supports avoiding creating tasklets > on Python threads other than the main one, and why > 'stacklesssocket.py' needs a '_timeout_func' to access that. > > http://code.google.com/p/stacklessexamples/source/browse/trunk/examples/networking/stacklesssocket_unittest.py > http://code.google.com/p/stacklessexamples/source/browse/trunk/examples/threading/stacklessthread.py > > Note that if you are using the 3.x version of the module, this has not > been updated. But if I recall correctly, the 3.x version is simply > the 2.x version with required structural changes to match how things > have changed. It should be an easy project to take the 3.x version > and apply the 2.x changes. And to adapt the new unit test script that > will run the 3.x socket unit tests against it. > > http://code.google.com/p/stacklessexamples/source/browse/trunk/examples/networking/stacklesssocket30.py > > Cheers, > Richard. > > -- > > Revision: 143 > Author: [email protected] > Date: Sat Aug 21 23:42:50 2010 > Log: Minor changes that allow the unit tests to run against Python > 2.5, which does not have 'some_thread.ident'. > > The unit tests pass in Stackless Python 2.7 exactly the same as > Stackless Python 2.6. > http://code.google.com/p/stacklessexamples/source/detail?r=143 > > Modified: > /trunk/examples/networking/stacklesssocket_unittest.py > /trunk/examples/threading/stacklessthread.py > > Revision: 142 > Author: [email protected] > Date: Sat Aug 21 23:26:45 2010 > Log: Fleshed out stacklesssocket.py: > - Calls to asyncore.poll() and similar functions no longer error > occasionally under heavy use with reference errors when a garbage > collected socket is still in its socket map. This is the difference > between storing weak references in a dictionary manually, and using > WeakValueDictionary instead. > - If the user of the module sets a value for '_sleep_func' or > '_timeout_func', then socket timeout support is enabled via the > standard function 'mysocket.settimeout(seconds)'. > - Stackless sockets are "blocking", but block the calling tasklet > rather than the current thread. Now they can be toggled to be > non-blocking via the standard function 'mysocket.setblocking(0)', > which will mean that if they will behave like normal non-blocking > sockets. > - All non-UDP standard Python socket unit tests pass when it is run on > Python 2.6, and only two of the UDP unit tests fail or error. The > unit tests are run against stacklesssocket.py with > stacklesssocket_unittest.py. The two failing UDP-related tests are > BasicUDPTest.testSendtoAndRecv and UDPTimeoutTest.testUDPTimeout. > - The internal stacklesssocket.py have been tested to pass against > Python 2.5 and 2.6. > > Fleshed out stacklessthread.py: > - This covers monkey patching a variety of thread blocking operations, > so that if they are happening in the main thread, they are turned into > tasklet blocking operations (and farmed out to a new custom created > thread). > > http://code.google.com/p/stacklessexamples/source/detail?r=142 > > Modified: > /trunk/examples/networking/stacklesssocket.py > /trunk/examples/networking/stacklesssocket_unittest.py > /trunk/examples/threading/stacklessthread.py > > _______________________________________________ > Stackless mailing list > [email protected] > http://www.stackless.com/mailman/listinfo/stackless > _______________________________________________ Stackless mailing list [email protected] http://www.stackless.com/mailman/listinfo/stackless
