Standard Threads vs Weightless Threads
Recently I read Charming Python: Implementing Weightless Threads (http://www-128.ibm.com/developerworks/linux/library/l-pythrd.html) by David D. I'm not an authority on threading architectures so I'd like to ask the following: 1)What is the difference (in terms of performance, scalability,[insert relevant metric here]) between microthreads and system threads? 2)If microthreads really are superior then why aren't they the standard Python implementation (or at least within the standard library)? (where my assumption is that they are not the standard implementation and are not contained within the standard library). ps. I fear that my questions and assumptions about threading may be rather naive. If they are, it's because I haven't yet done any significant research. -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Threads vs Weightless Threads
yoda wrote: 1)What is the difference (in terms of performance, scalability,[insert relevant metric here]) between microthreads and system threads? System-level threads are relatively heavyweight. They come with a full call stack, and they take up some level of kernel resources [generally less than a process]. In exchange, they're scheduled by the OS, with the primary benefit (on uniprocessor systems) that if one thread executes a blocking task (like IO writes) another thread will receive CPU attention. The primary disadvantage is that they're scheduled by the CPU. This leads to the concurrency nightmare, where the developer needs to keep track of what blocks of code (and data) need locks to prevent deadlock and race conditions. 2)If microthreads really are superior then why aren't they the standard Python implementation (or at least within the standard library)? (where my assumption is that they are not the standard implementation and are not contained within the standard library). Microthreads are very different; they're entirely internal to the Python process, and they're not seen at all by the operating system. Scheduling is done explicitly by the microthread implementation -- multitasking is not preemptive, as with system threads. They're not in the standard library because implementing microthreads has thus far required a very large rewrite of the CPython architecture -- see Stackless Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Threads vs Weightless Threads
Christopher Subich [EMAIL PROTECTED] writes: the primary benefit (on uniprocessor systems) that if one thread executes a blocking task (like IO writes) another thread will receive CPU attention. On multiprocessor systems, of course, the benefit is that threads can run on separate CPUs. This could possibly be addressed by transparently running one OS thread for each CPU, each containing a set of light weight threads, and I hear people are thinking about that for Erlang (light weight processes language extravaganza). In the meanwhile, you can of course solve this by running two separate instances of your program that communicates with each other somehow. Another benefit is that you can have more threads than the limit the OS imposes on you, and they can also be made quite a bit less memory consuming. Some language implementations manages to deal with hundreds of thousands, and in some cases even millions, of threads this way. They're not in the standard library because implementing microthreads has thus far required a very large rewrite of the CPython architecture -- see Stackless Python. Ruby's built-in threading also work something like this (if I understand Stackless Python correctly). (http://www.rubycentral.com/book/tut_threads.html) Personally, though, I think treating treating light weight threads as separate processes (which can only pass messages in between each other, but do not share scope in any way), like Erlang, seems to make dealing with massive concurrency easier. -- Björn Lindström [EMAIL PROTECTED] Student of computational linguistics, Uppsala University, Sweden -- http://mail.python.org/mailman/listinfo/python-list
