At 06:10 PM 9/20/2005 -0400, Bob Ippolito wrote: >My use case for this isn't so much about threads, but plug-ins. >Writing multiple Python-based plug-ins for an application is always a >mess, because they share too much (sys.modules, sys.path, etc.). >PyObjC would benefit greatly from this feature, because you can write >Python-based plug-ins for any Cocoa app that supports plug-ins, even >if they're otherwise unaware of Python's existence. There are >workarounds, of course, with import hooks and similar hacks. I think >that mod_python would also benefit from this, and probably other such >systems.
To that list of shared things, one can easily add sys.argv, sys.stdin, sys.stdout, sys.stderr, os.environ, and on and on and on. But the ones you mention present special issues for testing, especially testing of import hooks, path management tools and so on. Decimal contexts are slightly better in that they're at least thread-local. But even if all of these things were thread-local, you'd still have problems with task switches between pseudothreads, or just maintaining logically separate contexts between e.g. different plugins. I've actually got an idea of how to solve these problems, at least for Python-level code, although for it to work right with Python's import internals it'd be necessary to use PyObject_* APIs against sys.modules instead of PyDict_* ones. Apart from that, it's implementable in pure Python, so non-CPython implementations should be able to use it too. I've been considering writing a PEP for it, since it also adds a generally-useful way of dealing with "context variables" (like the Decimal context and the sys.* variables), while being able to support switching *all* context variables simultaneously and instantaneously when changing execution contexts (like switching between coroutines). For example, it would let 1000 pseudothreads each have a unique current value for sys.stdout or the current decimal context, addressing a gap in the fit between PEP 343 and PEP 342. There's currently no reasonable way to task switch between co-routines in the body of a 'with:' statement, so things like "with redirecting_stdout()" to two places in different coroutines breaks horribly. But context managers implemented via context variables would handle such situations automatically, and with isolation of effects to the current thread. It would also allow you to have logically distinct plugins by swapping their context variables in when you make callbacks to them, so the notion of the "task" to which a set of context variables applies is not limited to coroutines. My prototype implementation of the basic idea is <200 lines of Python and very fast: ~2usec on my budget PC to switch between two arbitrarily-sized sets of context variables. The data structure is O(1) for switching sets or changing values; it's only O(N) the next time you change a value after taking a read-only snapshot of the whole context. (Taking a snapshot is O(1) due to copy-on-write; you can take as many snapshots of the same unchanged state without creating any new objects.) It also takes about 2uS to read a context variable, alas, but a C version could drop the overhead down to as little as a single dictionary lookup if the "context pointer" were kept in the thread state structure. (A fair bit of the Python implementation's overhead is just getting to the context mapping for the current thread.) The idea isn't fully PEP-ready at this point, though. I still need to flesh out my ideas regarding how context variables would be initialized in new threads, for example, or when you spawn new coroutines. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
