[Python-Dev] Undocumented PEP 302 protocol change by need-for-speed sprint
While investigating the need to apply http://python.org/sf/1525766 I found that there was a modification to pkgutil during the need-for-speed sprint that affects the PEP 302 protocol in a backwards incompatible way. Specifically, PEP 302 documents that path_importer_cache always contains either importer objects or None. Any code written to obtain importer objects is therefore now broken, because import.c is slapping False in for non-existent filesystem paths. The pkgutil module was then hacked to work around this problem, thereby hiding the breakage from at least the standard library, but not any external libraries that follow the PEP 302 protocol to find importers. There are several options as to how to proceed: 1. Revert the change 2. Document the breakage, update PEP 302, and make everybody update their code 3. Make it not break existing code, by using a NonexistentPathImporter or NullImporter type in place of False in sys.path_importer_cache. Any thoughts? Personally, the only code I know of that implements the PEP 302 protocol besides the pkgutil module that would be affected is pkg_resources in setuptools, so it's not like I can't fix it for 2.5. However, I don't know if anybody else is using the protocol, and if so, how bad the breakage would be. This should really only affect code that is walking sys.path, because paths with False in sys.path_importer_cache by definition cannot have any importable modules associated with them. So, although I don't like option 2 on general principles, it may be an acceptable solution. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Undocumented PEP 302 protocol change by need-for-speed sprint
At 12:28 PM 7/20/2006 -0700, Brett Cannon wrote: On 7/20/06, Phillip J. Eby mailto:[EMAIL PROTECTED][EMAIL PROTECTED] wrote: While investigating the need to apply http://python.org/sf/1525766http://python.org/sf/1525766 I found that there was a modification to pkgutil during the need-for-speed sprint that affects the PEP 302 protocol in a backwards incompatible way. Specifically, PEP 302 documents that path_importer_cache always contains either importer objects or None. Any code written to obtain importer objects is therefore now broken, because import.c is slapping False in for non-existent filesystem paths. The pkgutil module was then hacked to work around this problem, thereby hiding the breakage from at least the standard library, but not any external libraries that follow the PEP 302 protocol to find importers. There are several options as to how to proceed: 1. Revert the change 2. Document the breakage, update PEP 302, and make everybody update their code 3. Make it not break existing code, by using a NonexistentPathImporter or NullImporter type in place of False in sys.path_importer_cache. Any thoughts? Revert it. Is it really that much of a bonus to use False over None? Both evaluate to false and both are already singleton so you can use 'is' for testing. The changed code still uses None. PEP 302 defines None as meaning that a sys.path entry does not have an importer. It's just that the need-for-speed patch *adds* the use of True and False. None still means no importer, but True now means no importer, path exists and False now means no importer, path does not exist. The idea is that import.c can then skip checking the existence of the path when it sees True or False, but it then means that code that gets data from path_importer_cache needs to know about these new special values, or else it will get an attribute error when it tries to call True.find_module(). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Undocumented PEP 302 protocol change by need-for-speed sprint
On 7/20/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 12:28 PM 7/20/2006 -0700, Brett Cannon wrote:On 7/20/06, Phillip J. Ebymailto:[EMAIL PROTECTED][EMAIL PROTECTED] wrote:While investigating the need to applyhttp://python.org/sf/1525766http://python.org/sf/1525766 I foundthat there was a modification to pkgutil during the need-for-speed sprintthat affects the PEP 302 protocol in a backwards incompatible way.Specifically, PEP 302 documents that path_importer_cache always contains either importer objects or None.Any code written to obtain importerobjects is therefore now broken, because import.c is slapping False in fornon-existent filesystem paths. The pkgutil module was then hacked to work around this problem, therebyhiding the breakage from at least the standard library, but not anyexternal libraries that follow the PEP 302 protocol to find importers. There are several options as to how to proceed:1. Revert the change2. Document the breakage, update PEP 302, and make everybody update theircode3. Make it not break existing code, by using a NonexistentPathImporter or NullImporter type in place of False in sys.path_importer_cache.Any thoughts?Revert it.Is it really that much of a bonus to use False overNone?Both evaluate to false and both are already singleton so you can use 'is' for testing.The changed code still uses None.PEP 302 defines None as meaning that asys.path entry does not have an importer.It's just that theneed-for-speed patch *adds* the use of True and False.None still means no importer, but True now means no importer, path exists and False nowmeans no importer, path does not exist.Ah. Sounds like None is not really even needed with the change (although I am not suggesting the removal of None). The idea is that import.c can then skip checking the existence of the path when it sees True or False, but it then means that code that gets data frompath_importer_cache needs to know about these new special values, or elseit will get an attribute error when it tries to call True.find_module ().Well, I have not played with the PEP 302 stuff so I don't know how helpful they are to have around. But it is definitely a semantic change that either needs to be reverted or documented. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Undocumented PEP 302 protocol change by need-for-speed sprint
On Thu, 20 Jul 2006 14:57:07 -0400, Phillip J. Eby [EMAIL PROTECTED] wrote: While investigating the need to apply http://python.org/sf/1525766 I found that there was a modification to pkgutil during the need-for-speed sprint that affects the PEP 302 protocol in a backwards incompatible way. It just so happens that the bug that is reported was probably reported because I'm working on some controversial new functionality in Twisted - controversial because it replicates the functionality that bug is about in pkgutil. This functionality does make some use of PEP 302 functionality :). See http://twistedmatrix.com/trac/ticket/1940 Specifically, PEP 302 documents that path_importer_cache always contains either importer objects or None. Any code written to obtain importer objects is therefore now broken, because import.c is slapping False in for non-existent filesystem paths. Oddly, for once I'm going to say I don't care about this change. The code I've written so far doesn't depend on this, and I was pretty careful to be conservative about depending too much on the stuff described in PEP 302. It documents several features which don't exist (get_data, and methods in the imp module which don't exist in python2.3 or python2.4, where it was nominally accepted). There are several options as to how to proceed: 2. Document the breakage, update PEP 302, and make everybody update their code Personally I'd prefer it if PEP 302 were updated for a variety of reasons. It's very hard to use as a reference for writing actual code because so many features are optional or open issues, and there's no description in the PEP of what their status is. Better yet, this breakage (and other things) should be documented in the Python reference, and the PEP should link to the documentation for different versions, which can each describe the PEP's implementation status. The importing modules section of the library reference seems like a natural place to put it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
