Hi, For anyone who is using stacklesssocket.py, I've just upgraded it so that it passes the non-UDP Python standard library socket unit tests (tested on 2.6 and 2.7). Note that it actually passes all of the UDP unit tests except for two, so that's a good sign that the changes I have made make it inherently more correct. I would also add that the tests do not pass on 2.5, but this looks to be due to minor exception/behaviour related changes and it should work as well as it ever did with 2.5.
http://code.google.com/p/stacklessexamples/source/browse/trunk/examples/networking/stacklesssocket.py I still have to do stress testing work on these changes. Some relevant things you might want to know about the changes: - The way it worked with the socket module and asyncore and garbage collection was to have a dictionary ('asyncore.socket_map') which weak references to tasklets were stored in. This was bad, as it meant that dead weak references would sit in the dictionary until events propagated and they were removed. This in turn would cause calls to 'asyncore.poll(seconds)' to have reference errors raised out of it under load as it choked on those dead weak references. Now it replaces 'asyncore.socket_map' with a WeakValueDictionary and when weak references die, they are removed immediately and there no more reference errors happen in the described way. - If you set 'stacklesssocket._sleep_func' to your sleep function, then the socket timeout mechanism will become enabled and take advantage of it (e.g. 'my_socket.settimeout(num_seconds'). You can also implement your own timeout mechanism with a 'stacklesssocket._timeout_func', but that is a little more complex and suits in the situation where you doing interthread communication with channels and wish to avoid the concerns that come with tasklets belonging to different threads and channel operations. - You can now call 'my_socket.setblocking(0)' and it will work as expected. - A wider swatch of the socket object API is now supported, with regard to weird functions like recv_into and the like. - No internal buffering of data, which means it should be more optimal. You get the data the socket provides directly (well, as direct as an asynchronous operation can be made to look synchronous using channels). Most of the work in this change was in contriving a script ('stacklesssocket_unittest.py') that would set up an environment where the Python unit tests could be run with the stacklesssocket module monkey patched into place. This involved writing a module ('stacklessthread.py') that turns a range of thread blocking operations into tasklet blocking operations, when they happen on the main thread (where the expectation is that the scheduler is only being run on that thread). While over the years we have fleshed out Christian's threading support for special Stackless functionality, there are still a few inconsistencies and when you are dealing with Python threads and Stackless they tend to pop up. This is the reason why 'stacklesssocket_unittest.py' supports avoiding creating tasklets on Python threads other than the main one, and why 'stacklesssocket.py' needs a '_timeout_func' to access that. http://code.google.com/p/stacklessexamples/source/browse/trunk/examples/networking/stacklesssocket_unittest.py http://code.google.com/p/stacklessexamples/source/browse/trunk/examples/threading/stacklessthread.py Note that if you are using the 3.x version of the module, this has not been updated. But if I recall correctly, the 3.x version is simply the 2.x version with required structural changes to match how things have changed. It should be an easy project to take the 3.x version and apply the 2.x changes. And to adapt the new unit test script that will run the 3.x socket unit tests against it. http://code.google.com/p/stacklessexamples/source/browse/trunk/examples/networking/stacklesssocket30.py Cheers, Richard. -- Revision: 143 Author: [email protected] Date: Sat Aug 21 23:42:50 2010 Log: Minor changes that allow the unit tests to run against Python 2.5, which does not have 'some_thread.ident'. The unit tests pass in Stackless Python 2.7 exactly the same as Stackless Python 2.6. http://code.google.com/p/stacklessexamples/source/detail?r=143 Modified: /trunk/examples/networking/stacklesssocket_unittest.py /trunk/examples/threading/stacklessthread.py Revision: 142 Author: [email protected] Date: Sat Aug 21 23:26:45 2010 Log: Fleshed out stacklesssocket.py: - Calls to asyncore.poll() and similar functions no longer error occasionally under heavy use with reference errors when a garbage collected socket is still in its socket map. This is the difference between storing weak references in a dictionary manually, and using WeakValueDictionary instead. - If the user of the module sets a value for '_sleep_func' or '_timeout_func', then socket timeout support is enabled via the standard function 'mysocket.settimeout(seconds)'. - Stackless sockets are "blocking", but block the calling tasklet rather than the current thread. Now they can be toggled to be non-blocking via the standard function 'mysocket.setblocking(0)', which will mean that if they will behave like normal non-blocking sockets. - All non-UDP standard Python socket unit tests pass when it is run on Python 2.6, and only two of the UDP unit tests fail or error. The unit tests are run against stacklesssocket.py with stacklesssocket_unittest.py. The two failing UDP-related tests are BasicUDPTest.testSendtoAndRecv and UDPTimeoutTest.testUDPTimeout. - The internal stacklesssocket.py have been tested to pass against Python 2.5 and 2.6. Fleshed out stacklessthread.py: - This covers monkey patching a variety of thread blocking operations, so that if they are happening in the main thread, they are turned into tasklet blocking operations (and farmed out to a new custom created thread). http://code.google.com/p/stacklessexamples/source/detail?r=142 Modified: /trunk/examples/networking/stacklesssocket.py /trunk/examples/networking/stacklesssocket_unittest.py /trunk/examples/threading/stacklessthread.py _______________________________________________ Stackless mailing list [email protected] http://www.stackless.com/mailman/listinfo/stackless
