Re: [Python-Dev] Pickling objects that return string from reduce
Bruce Christensen wrote: If obj has no __module__ attribute (or if it is None), pickle (didn't check cPickle) also does for n, module in sys.module.items(): if module-ignored: continue if getattr(module, result, None) is obj: break # use n as module name What is module-ignored above? It's obviously not a literal string... It's skipped if module is None (skip dummy package entries) or n=='__main__'. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
That's great. I just read your draft but I have little comments to do but before let me say that I liked the idea to borrow concepts from E. I've crossed the E's path in the beginning of this year and I found it a pot of really nice ideas (for promises and capabilities). Here are my comments about the draft: - it's not really clear to me what the powerbox is. I think I got the concept of super process but maybe it's to be clarified, isn't it? It become clear in the threat model paragraph - I hope no Rubystas will read the Problem of No Private Namespace section because they have private/protected keywords to enforce this stuff :-) Writing proxies in C will slow down the dev process (altough will speed up the performance maybe) but in a far future someone will come up with an alternative closer to the Python level - Can you write down a simple example of what you mean with changing something of the built-in objects? (in Problem of mutable shared state) - What about the performance issues of the capabilities model overall? - I know what you meant to say but the paragraph about pythonicness and the security model seems a little fuzzy to me. Which are the boundaries of the allowed changes for the security stuff? - You don't say anything about networking and networked resources in the list of the standard sandboxed interpreter - Suppose we have a .py module. Based on your security model we can import it, right? When imported it generates a .pyc file. The second time we import it what happens? .pyc is ignored? import is not allowed at all? We can't rely on the name of the file.pyc because an attacker who knows the file.py is secure and the second import is done against file.pyc can replace the secure file.pyc with an implementation not secure and can do some kind of harm to the sandbox - About Filesystem information. Does the sandboxed interpreter need to know all that information about file paths, files and so on? Can't we reset those attributes to something arbitrary? - About sys module: I think the best way is to have a purged fake sys module with only the stuff you need. pypy has the concept of faked modules too (altough for a different reason) - About networking: what do you think about the E's model of really safe networking, protected remotable objects and safe RPC? Is that model applicable to Python's in some way? We can't use the E's model as a whole (ask people to generate a safe key and send it by email is unfeasible) - is the protected memory model a some kind of memory monitor system? I think that's all for the draft. I wrote these comments during the reading of the document. Hope some of these help -- Lawrence http://www.oluyede.org/blog ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] os.utime and os.chmod failures (etc) Python 2.5b2
Hello all, There may be a reasonable cause for this (i.e. it is likely to be my fault) - but it is consistent across two different machines I have tried it on. With Python 2.5b2 (from the msi at Python.org), running on Windows XP Pro SP2, ``os.utime`` and ``os.chmod`` fail with WindowsError. The same code runs fine on Python 2.3 and Python 2.4. [err] shutil.copytree(thisentry, targetdir) [err] File C:\Python25\lib\shutil.py, line 130, in copytree [err] copystat(src, dst) [err] File C:\Python25\lib\shutil.py, line 67, in copystat [err] os.utime(dst, (st.st_atime, st.st_mtime)) [err] WindowsError: [Error 13] Access is denied: 'lib\\Pmw' [err] The script uses ``shutil.copytree`` to copy a directory (using relative paths). The source code snippet is : if os.path.isdir(targetdir): shutil.rmtree(targetdir) shutil.copytree(thisentry, targetdir) The code in shutil.py is : def copystat(src, dst): Copy all stat info (mode bits, atime and mtime) from src to dst st = os.stat(src) mode = stat.S_IMODE(st.st_mode) if hasattr(os, 'utime'): os.utime(dst, (st.st_atime, st.st_mtime)) if hasattr(os, 'chmod'): os.chmod(dst, mode) Additionally, after running a couple of times I get core dumps when trying to run the interactive interpreter. Oddly IDLE and other programs still run. However this is after installing py2exe, wxPython and pywin32 (all of which have Python 2.5 builds), so there is possibly some weird interaction. My install directory has the following three dlls in it : MSVCIRT.dll MSVCP60.dll MSVCRT.dll It has none of the msvc7 dlls that I would expect. I'm now at work and so don't have time to experiment, but tonight I will try uninstalling all my Python 2.5 stuff and re-installing just 2.5b2. I will see what dlls are present, run the interpreter a few times, and test a simple script with shutil.copytree. All the best, Michael Foord http://www.voidspace.org.uk/python/index.shtml ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Strategy for converting the decimal module to C
Greg Ewing [EMAIL PROTECTED] wrote: But couldn't you just put in an interrupt handler that counts the interrupts, for the purpose of measurement? No, but the reasons are very arcane. The general reason is that taking an interrupt handler and returning is not transparent, and is often not possible on modern systems. If that problem is at the hardware level (as on the Alpha and 3086/7), you are stuffed. But, more often, it is due the the fact that the architecture means that such handling can only be done at maximally privileged level. Now, interrupting into that level has to be transparent, in order to support TLB misses, clock interrupts, device interrupts, machine-check interrupts and so on. But the kernels rarely support transparent callbacks from that state into user code (though they used to); it is actually VERY hard to do, and even the mainframes had problems. This very commonly means that such counting breaks other facilities, unless it is done IN the privileged code. Of course, a GOOD hardware architecture wouldn't leave the process state when it gets a floating-point interrupt, but would just invoke an asynchronous routine call. That used to be done. As I said, none of this is directly relevant to emulated implementations, such as the current Python ones, but it IS to the design of an arithmetic specification.It could become relevant if Python wants to start to use a hardware implementation, because your proposal would mean that it would have to try to ensure that such callbacks are transparent. As one of the few people still working who has extensive experience with doing that, I can assure you that it is an order of magnitude fouler than you can imagine. A decimal order of magnitude :-( But note that I wasn't saying that such things should be put into the API, merely that there is a very good reason to do so for hardware implementations and ones used to tune code for such implementations. Personally, I wouldn't bother. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
Giovanni Bajo [EMAIL PROTECTED] wrote: This recipe for safe_eval: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496746 which is otherwise very cute, does not handle this case as well: it tries to catch and interrupt long-running operations through a secondary thread, but fails on a single long operation because the GIL is not released and the alarm thread does not get its chance to run. Grin :-) You have put your finger on the Great Myth of such virtualisations, which applies to the system-level ones and even to the hardware-level ones. In practice, there is always some request that a sandbox can make to the hypervisor that can lock out or otherwise affect other sandboxes. The key is, of course, to admit that and to specify what is and is not properly virtualised, so that the consequences can at least be analysed. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
Nick Maclaren wrote: This recipe for safe_eval: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496746 which is otherwise very cute, does not handle this case as well: it tries to catch and interrupt long-running operations through a secondary thread, but fails on a single long operation because the GIL is not released and the alarm thread does not get its chance to run. Grin :-) You have put your finger on the Great Myth of such virtualisations, which applies to the system-level ones and even to the hardware-level ones. In practice, there is always some request that a sandbox can make to the hypervisor that can lock out or otherwise affect other sandboxes. The key is, of course, to admit that and to specify what is and is not properly virtualised, so that the consequences can at least be analysed. I agree, and in fact Brett's work on a proper security model is greatly welcome. It's just that us mere mortals need to use eval() *now*, and that recipe is good enough for many practice uses. If you can't win, you can at least lose with dignity :) -- Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
For code objects, their construction is already commonly written as compile(source). For type objects, the constructor doesn't let you do anything you can't already do with a class statement. It doesn't need securing. For rewriting import.c in Python, the PEP 302 compliant import system API in pkgutil would be a good starting point. Your doc also asks about the imp.get_suffixes() list, and wonder where to set it from Python. As far as I am aware, you can't. get_suffixes() is built from _PyImport_FileTab, which is a C array. A switch statement is used to get from the file table entries to the appropriate handler functions. Quoting from the suggestions I put to the Py3k list: Use smarter data structures --- Currently, the individual handlers to load a fully identified module are exposed to Python code in a way that reflects the C-style data structures used in the current implementation. Simply switching to more powerful data structures for the file type handlers (i.e. use a PyTuple for filedescr values, a PyList for _PyImport_FileTab, and a PyDict instead of a switch statement to go from filedescr values to module loading/initialisation functions) and manipulating them all as normal Python objects could make the code in import.c much easier to follow. Extensible file type handling - If the file type handlers are stored in normal Python data structures as described above, it becomes feasible to make the import system extensible to different file types as well as to different file locations. This could be handled on a per-package basis, e.g. via a __file_types__ special attribute in packages. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.utime and os.chmod failures (etc) Python 2.5b2
Michael Foord schrieb: Hello all, There may be a reasonable cause for this (i.e. it is likely to be my fault) - but it is consistent across two different machines I have tried it on. With Python 2.5b2 (from the msi at Python.org), running on Windows XP Pro SP2, ``os.utime`` and ``os.chmod`` fail with WindowsError. The same code runs fine on Python 2.3 and Python 2.4. [err] shutil.copytree(thisentry, targetdir) [err] File C:\Python25\lib\shutil.py, line 130, in copytree [err] copystat(src, dst) [err] File C:\Python25\lib\shutil.py, line 67, in copystat [err] os.utime(dst, (st.st_atime, st.st_mtime)) [err] WindowsError: [Error 13] Access is denied: 'lib\\Pmw' [err] The script uses ``shutil.copytree`` to copy a directory (using relative paths). IMO this is a bug in Python 2.5, on Windows. The problem is that the call to 'copystat(src, dst)' was added to the shutil.copytree function, in svn r38363 probably. It will fail always on Windows, since os.utime does not work on directories (as the docs correctly explain). I guess that a patch similar to this one should fix it: Index: shutil.py === --- shutil.py (Revision 50710) +++ shutil.py (Arbeitskopie) @@ -127,7 +127,12 @@ # continue with other files except Error, err: errors.extend(err.args[0]) -copystat(src, dst) +try: +copystat(src, dst) +except WindowsError: +pass +except OSError, err: +errors.extend(err.args[0]) if errors: raise Error, errors But you should report this to the bug tracker. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.utime and os.chmod failures (etc) Python 2.5b2
Thomas Heller wrote: Michael Foord schrieb: Hello all, There may be a reasonable cause for this (i.e. it is likely to be my fault) - but it is consistent across two different machines I have tried it on. With Python 2.5b2 (from the msi at Python.org), running on Windows XP Pro SP2, ``os.utime`` and ``os.chmod`` fail with WindowsError. The same code runs fine on Python 2.3 and Python 2.4. [err] shutil.copytree(thisentry, targetdir) [err] File C:\Python25\lib\shutil.py, line 130, in copytree [err] copystat(src, dst) [err] File C:\Python25\lib\shutil.py, line 67, in copystat [err] os.utime(dst, (st.st_atime, st.st_mtime)) [err] WindowsError: [Error 13] Access is denied: 'lib\\Pmw' [err] The script uses ``shutil.copytree`` to copy a directory (using relative paths). IMO this is a bug in Python 2.5, on Windows. The problem is that the call to 'copystat(src, dst)' was added to the shutil.copytree function, in svn r38363 probably. It will fail always on Windows, since os.utime does not work on directories (as the docs correctly explain). I guess that a patch similar to this one should fix it: Index: shutil.py === --- shutil.py (Revision 50710) +++ shutil.py (Arbeitskopie) @@ -127,7 +127,12 @@ # continue with other files except Error, err: errors.extend(err.args[0]) -copystat(src, dst) +try: +copystat(src, dst) +except WindowsError: +pass +except OSError, err: +errors.extend(err.args[0]) if errors: raise Error, errors But you should report this to the bug tracker. Ok, thanks. Michael http://www.voidspace.org.uk/python/index.shtml Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] logging module broken because of locale
On Thu, Jul 20, 2006 at 05:09:38AM -0500, [EMAIL PROTECTED] wrote: Misa Good point. Does the attached patch look reasonable? ... Misa -self.when = string.upper(when) Misa +self.when = unicode(when).upper() ... The use of the string module instead of string methods suggests to me that the logging package attempts to work with older versions of Python. Looking at PEP 291 it looks like 1.5.2 compatibility is desired (no string methods, no unicode). I think a conscious decision by someone (probably Vinay Sajip) to give up that compatibility would be required. Agreed. There is a note that the module should work with python = 1.5.2 at the top of the module. It's up to Vinay to decide if we want to drop support for 1.5.2 in the module included in newer pythons, or the attached patch would make it work for 1.5.2 as well (as in it's not more broken than before). I would like to redo the patch once more to get rid of the try-except and use __builtins__ instead (but for some reason it kept jumping from being a module to being a dictionary and I just wanted the proof of concept). Misa --- Python-2.4.3/Lib/logging/handlers.py.nolocale 2006-07-19 12:15:46.0 -0400 +++ Python-2.4.3/Lib/logging/handlers.py2006-07-20 09:45:57.0 -0400 @@ -162,7 +162,7 @@ def __init__(self, filename, when='h', interval=1, backupCount=0, encoding=None): BaseRotatingHandler.__init__(self, filename, 'a', encoding) -self.when = string.upper(when) +self.when = _upper(when) self.backupCount = backupCount # Calculate the real rollover interval, which is just the number of # seconds between rollovers. Also set the filename suffix used when @@ -645,7 +645,7 @@ msg = self.log_format_string % ( self.encodePriority(self.facility, -string.lower(record.levelname)), +_lower(record.levelname)), msg) try: if self.unixsocket: @@ -854,7 +854,7 @@ (GET or POST) logging.Handler.__init__(self) -method = string.upper(method) +method = _upper(method) if method not in [GET, POST]: raise ValueError, method must be GET or POST self.host = host @@ -1007,3 +1007,25 @@ self.flush() self.target = None BufferingHandler.close(self) + +def _upper(s): +A version of upper() that tries to be locale-independent by converting +the string to unicode (which is not subject to case conversion being +locale specific) + +try: +ret = str(unicode(s).upper()) +except NameError: +ret = string.upper(s) +return ret + +def _lower(s): +A version of lower() that tries to be locale-independent by converting +the string to unicode (which is not subject to case conversion being +locale specific) + +try: +ret = str(unicode(s).lower()) +except NameError: +ret = string.lower(s) +return ret ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] logging module broken because of locale
Mihai, It does make sense to document this limit for people writing subclasses, or using a Turkic codeset. I'm not sure that logging is the right place to document it, and I don't think changing the base classes is a good idea. TimedRotatingFileHandler and HTTPHandler restrict their input to a tiny subset of ASCII; anything that would be handled differently after these changes will already raise a ValueError unless you subclass. Even SysLogHandler.emit doesn't actually print the string; it is only used as a lookup key for a dictionary whose keys are all lower-case ASCII. In theory, you could monkey-patch that dictionary to add additional values, but then you might as well subclass to do the right thing with your new keys. (It *might* make sense to change the base function and dictionary to accept unicode, including undotted-i synonyms.) -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] logging module broken because of locale
On Thu, Jul 20, 2006 at 06:08:05PM +0200, Martin v. Löwis wrote: Mihai Ibanescu wrote: It's up to Vinay to decide if we want to drop support for 1.5.2 in the module included in newer pythons, or the attached patch would make it work for 1.5.2 as well (as in it's not more broken than before). That still wouldn't work with Python 1.5.2, as that version did not support Unicode at all. Yes, as I said, it won't be more broken than before applying the patch (my first patch was breaking 1.5.2 completely). If people choose to compile python 2.4.3 without unicode support, it won't work either. But if you choose to disable unicode you probably live in a very constrained environment and you may not be affected by the locale bug at all. I agree the patch is not perfect :-) but I think it solves the problem for most people. Misa ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] logging module broken because of locale
On Thu, Jul 20, 2006 at 11:39:27AM -0400, Jim Jewett wrote: Mihai, It does make sense to document this limit for people writing subclasses, or using a Turkic codeset. I'm not sure that logging is the right place to document it, and I don't think changing the base classes is a good idea. TimedRotatingFileHandler and HTTPHandler restrict their input to a tiny subset of ASCII; anything that would be handled differently after these changes will already raise a ValueError unless you subclass. Even SysLogHandler.emit doesn't actually print the string; it is only used as a lookup key for a dictionary whose keys are all lower-case ASCII. In theory, you could monkey-patch that dictionary to add additional values, but then you might as well subclass to do the right thing with your new keys. (It *might* make sense to change the base function and dictionary to accept unicode, including undotted-i synonyms.) Hi Jim, I am afraid you lost me. My initial approach was to compute the values of the dictionary based on the current locale - and that doesn't work. The issue is not that the dictionary doesn't accept unicode, it's that there's no reliable way to do lookups in it. If you could sketch an example of what you had in mind with the monkey-patching, please do so, since I don't think I grasped the idea. Misa ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] logging module broken because of locale
Mihai Ibanescu wrote: Yes, as I said, it won't be more broken than before applying the patch (my first patch was breaking 1.5.2 completely). Ah, I didn't notice that it deals with unicode() not being a builtin. That's fine then. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
On 7/20/06, Giovanni Bajo [EMAIL PROTECTED] wrote: Brett Cannon wrote: The new doc is named securing_python.txt and can be found through the svn web interface at http://svn.python.org/view/python/branches/bcannon-sandboxing/securing_python.txt?rev=50717view=log.How do you plan to handle CPU-hogs? Stuff like execution of a giganticinteger multiplication. I don't. =) Protecting the CPU is damn hard to do in any form of portable fashion. And even getting it to work on an OS you do know the details of leads to probably an interrupt implementation and that doesn't sound fun. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] logging module broken because of locale
On 7/20/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Thu, Jul 20, 2006 at 11:39:27AM -0400, Jim Jewett wrote: Even SysLogHandler.emit doesn't actually print the string; it is only used as a lookup key for a dictionary whose keys are all lower-case ASCII. In theory, you could monkey-patch that dictionary to add additional values, but then you might as well subclass to do the right thing with your new keys. (It *might* make sense to change the base function and dictionary to accept unicode, including undotted-i synonyms.) My initial approach was to compute the values of the dictionary based on the current locale - and that doesn't work. Correct, it would need to be done by hand. As part of the current class construction, SysLogHandler.priority_names[info] = SysLogHandler.LOG_INFO You could add other entries, so that SysLogHandler.priority_names[my_info] = SysLogHandler.LOG_INFO If you also changed SysLogHandler.encode_priority to accept unicode keys, you could even add uinfo and the equivalent with an undotted-i. I'm not sure this is worth doing, though, since they supposedly represent symbolic constants. -jJ The issue is not that the dictionary doesn't accept unicode, it's that there's no reliable way to do lookups in it. If you could sketch an example of what you had in mind with the monkey-patching, please do so, since I don't think I grasped the idea. Misa ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
On 7/20/06, Nick Coghlan [EMAIL PROTECTED] wrote: For code objects, their construction is already commonly written ascompile(source).Right, but some people like to construct directly from bytecode. For type objects, the constructor doesn't let you do anything you can'talready do with a class statement. It doesn't need securing.I figured as much, but when I was making the list I was not sure and didn't want to stop my writing momentum to check. For rewriting import.c in Python, the PEP 302 compliant import system API in pkgutil would be a good starting point.Yep. Plan on looking at all of the various modules in the stdlib that assist with importing, package PEP (I think there is one), and PEP 302. Your doc also asks about the imp.get_suffixes() list, and wonder where to setit from Python.As far as I am aware, you can't. get_suffixes() is built from_PyImport_FileTab, which is a C array. A switch statement is used to get from the file table entries to the appropriate handler functions.Ah, OK. Quoting from the suggestions I put to the Py3k list:Use smarter data structures---Currently, the individual handlers to load a fully identified module areexposed to Python code in a way that reflects the C-style data structures used in the current implementation.Simply switching to more powerful data structures for the file type handlers(i.e. use a PyTuple for filedescr values, a PyList for _PyImport_FileTab, anda PyDict instead of a switch statement to go from filedescr values to module loading/initialisation functions) and manipulating them all as normal Pythonobjects could make the code in import.c much easier to follow.Yep. I just kind of glanced at the rest of your suggestions, Nick, since I assumed a lot of it would change (or could be changed) if import was redone in as much Python as possible. Extensible file type handling-If the file type handlers are stored in normal Python data structures as described above, it becomes feasible to make the import system extensible todifferent file types as well as to different file locations.Yep. Although I am more interested in restricting than broadening the file types. This could be handled on a per-package basis, e.g. via a __file_types__special attribute in packages. Maybe. I don't want to get into introducing new abilities to start, though.-Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
On 7/20/06, Lawrence Oluyede [EMAIL PROTECTED] wrote: That's great. I just read your draft but I have little comments to dobut before let me say that I liked the idea to borrow concepts from E.I've crossed the E's path in the beginning of this year and I found it a pot of really nice ideas (for promises and capabilities). Here aremy comments about the draft:- it's not really clear to me what the powerbox is. I think I gotthe concept of super process but maybe it's to be clarified, isn't it? It become clear in the threat model paragraphThe powerbox is the thing that gives your security domains their initial abilities. The OS gives the process its abilities, but it does not directly work with the interpreter. Since the process does, though, it is considered the powerbox and farms out abilities that it has been given by the OS. I have tried to clarify the definition at the start of the doc.- I hope no Rubystas will read the Problem of No Private Namespace section because they have private/protected keywords to enforce thisstuff :-) Writing proxies in C will slow down the dev process (altoughwill speed up the performance maybe) but in a far future someone will come up with an alternative closer to the Python levelMaybe. As I said in the doc, any changes must be Pythonic and adding private namespaces right now wouldn't be without much more thought and work. And if Ruby ends up with this security model but more thoroughly, more power to them. Their language is different in the right ways to support it.As for coding in C, thems the breaks. I plan in adding stuff to the stdlib for the common case. I might eventually think of a good, generic proxy object that could be used, but as of right now I am not worrying about that since it would be icing on the cake. - Can you write down a simple example of what you mean with changing something of the built-in objects? (in Problem of mutable sharedstate)Done. - What about the performance issues of the capabilities model overall?Should be faster than an IBAC model since certain calls will not need to check the identity of the caller every time.But I am not worrying about performance, I am worrying about correctness, so I did not try to make any performance claims. - I know what you meant to say but the paragraph about pythonicnessand the security model seems a little fuzzy to me. Which are the boundaries of the allowed changes for the security stuff?Being pythonic is a fuzzy term in itself and Guido is the only person who can make definitive claims over what is and is not Pythonic. As I have said, this doc was mostly written with python-dev in mind since they are the ones I have to convince to let this into the core and they all know the term. But I have tacked in a sentence on what the term means.- You don't say anything about networking and networked resources in the list of the standard sandboxed interpreterNope. Have not started worrying about that yet. Just trying to get the basic model laid out. - Suppose we have a .py module. Based on your security model we canimport it, right? When imported it generates a .pyc file. The secondtime we import it what happens? .pyc is ignored? import is not allowedat all? We can't rely on the name of the file.pyc because an attackerwho knows the file.py is secure and the second import is done againstfile.pyc can replace the secure file.pyc with an implementation notsecure and can do some kind of harm to the sandbox It will be ignored. But I am hoping that through rewriting the import machinery more control over generating .pyc files can be had (see Skip Montanaro's PEP on this; forget the number). This is why exact details were left out of the implementation details. I just wanted people understand the approach to everything, not the concrete details of how it will be coded up. - About Filesystem information. Does the sandboxed interpreter need to know all that information about file paths, files and so on? Can'twe reset those attributes to something arbitrary?That is the point. It is not that the sandbox needs to know it, its that it needs to be hidden from the sandbox. - About sys module: I think the best way is to have a purged fake sysmodule with only the stuff you need. pypy has the concept of faked modules too (altough for a different reason)OK. - About networking: what do you think about the E's model of really safe networking, protected remotable objects and safe RPC? Is thatmodel applicable to Python's in some way? We can't use the E's modelas a whole (ask people to generate a safe key and send it by email isunfeasible) I have not looked at it. I am also not trying to build an RPC system *and* a security model for Python. That is just too much work right now. - is the protected memory model a some kind of memory monitor system?Basically. It just keeps a size_t on the memory cap and another on memory usage, and when memory is requested it makes sure that it won't go over the cap. And when memory is freed the usage goes
Re: [Python-Dev] new security doc using object-capabilities
Brett Cannon wrote: http://svn.python.org/view/python/branches/bcannon-sandboxing/securing_python.txt?rev=50717view=log . How do you plan to handle CPU-hogs? Stuff like execution of a gigantic integer multiplication. I don't. =) Protecting the CPU is damn hard to do in any form of portable fashion. And even getting it to work on an OS you do know the details of leads to probably an interrupt implementation and that doesn't sound fun. I think the trick used by the safe_eval recipe (a separate thread which interrupts the script through thread.interrupt_main()) shows that, in most cases, it's possible to make sure that an embedded script does not take too long to execute. Do you agree that this usage case (allow me to timeout an embedded script) is something which would be a very good start in the right direction? Now, I wonder, in a restricted execution environment such as that depicted in your document, how many different ways are there to make the Python interpreter enter a long calcolation loop which does not release the GIL? I can think of bignum*bignum, bignum**bignum or similar mathematical operations, but there are really a few. If we could make those release the GIL (or poll some kind of watchdog used to abort them, pretty much like they normally poll CTRL+C), then the same trick used by the recipe could be used. -- Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
On 7/20/06, Giovanni Bajo [EMAIL PROTECTED] wrote: Brett Cannon wrote:http://svn.python.org/view/python/branches/bcannon-sandboxing/securing_python.txt?rev=50717view=log . How do you plan to handle CPU-hogs? Stuff like execution of a gigantic integer multiplication. I don't.=)Protecting the CPU is damn hard to do in any form of portable fashion.And even getting it to work on an OS you do know the details of leads to probably an interruptimplementation and that doesn't sound fun.I think the trick used by the safe_eval recipe (a separate thread which interrupts the script through thread.interrupt_main()) shows that, in mostcases, it's possible to make sure that an embedded script does not take toolong to execute. Do you agree that this usage case (allow me to timeout an embedded script) is something which would be a very good start in the rightdirection?Probably. I just don't feel like worrying about it right now. =) Now, I wonder, in a restricted execution environment such as that depictedin your document, how many different ways are there to make the Pythoninterpreter enter a long calcolation loop which does not release the GIL? I can think of bignum*bignum, bignum**bignum or similar mathematicaloperations, but there are really a few. If we could make those release theGIL (or poll some kind of watchdog used to abort them, pretty much like they normally poll CTRL+C), then the same trick used by the recipe could be used.Well, any work that does most of its calculation within C code and that does not touch base with the interpreter on a semi-regular basis would need to relesae the GIL. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Undocumented PEP 302 protocol change by need-for-speed sprint
While investigating the need to apply http://python.org/sf/1525766 I found that there was a modification to pkgutil during the need-for-speed sprint that affects the PEP 302 protocol in a backwards incompatible way. Specifically, PEP 302 documents that path_importer_cache always contains either importer objects or None. Any code written to obtain importer objects is therefore now broken, because import.c is slapping False in for non-existent filesystem paths. The pkgutil module was then hacked to work around this problem, thereby hiding the breakage from at least the standard library, but not any external libraries that follow the PEP 302 protocol to find importers. There are several options as to how to proceed: 1. Revert the change 2. Document the breakage, update PEP 302, and make everybody update their code 3. Make it not break existing code, by using a NonexistentPathImporter or NullImporter type in place of False in sys.path_importer_cache. Any thoughts? Personally, the only code I know of that implements the PEP 302 protocol besides the pkgutil module that would be affected is pkg_resources in setuptools, so it's not like I can't fix it for 2.5. However, I don't know if anybody else is using the protocol, and if so, how bad the breakage would be. This should really only affect code that is walking sys.path, because paths with False in sys.path_importer_cache by definition cannot have any importable modules associated with them. So, although I don't like option 2 on general principles, it may be an acceptable solution. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new security doc using object-capabilities
Should be faster than an IBAC model since certain calls will not need to check the identity of the caller every time. But I am not worrying about performance, I am worrying about correctness, so I did not try to make any performance claims. Got that. Nope. Have not started worrying about that yet. Just trying to get the basic model laid out. Ok sorry to have bothered That is the point. It is not that the sandbox needs to know it, its that it needs to be hidden from the sandbox. So I think that's a simple step during the importing step. I have not looked at it. I am also not trying to build an RPC system *and* a security model for Python. That is just too much work right now. Ok sorry :-) Thanks, Lawrence. Thank you! -- Lawrence http://www.oluyede.org/blog ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for PyGetSetDefs in pydoc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Jul 13, 2006, at 12:12 PM, Barry Warsaw wrote: I've updated SF patch #1520294 and assigned it back to Georg for another quick review. Neal commented in the patch that it might help to explain the implementation a bit. I'd like to do that and also explain why I think this patch should go in 2.5. OTOH, I suspect most people just don't care, which is why I've gotten almost no comments on the patch (other than one or two mild nods of approval). As a reminder, this patch is really about providing useful help() at the interactive prompt for two specific types of C implemented descriptors, getset and member descriptors. Both objects can be given docstrings in the C code, but currently nothing in the tool chain pulls out and displays those docstrings, so help() on such objects is not very useful. While certainly a minor point, I think it improves the user experience to be able to display any existing docstrings for these types of objects. Of course, I'm scratching my own itch here because our app is deeply embedded and we have a lot of getset and member descriptors with docstrings. I'd like for our users to be able to type help(someobj.member) at our interactive prompt and get our docstring printed back at them. The implementation is fairly straightforward I add two new constants to types.py, a couple of is*() methods to inspect.py, and a bit of extra support to pydoc.py. Where it gets complicated is that I added a _types builtin module to provide a getset and member descriptor that could be turned into *Type constants in the types.py module. Why did I do this instead of trying to hunt down some existing getset or member descriptor? For one thing, there really aren't very good candidates for such objects in the built-in modules. You can't use objects like datetime.timedelta.days in types.py because datetime is not importable early enough for types.py to use it. Even if there were likely candidates, they would be accidents of implementation and it doesn't seem like a good idea to force force some future datetime maintainer to have to fix types.py when she decides that datetime.timedelta.days should be implemented in some other way. A 3rd party extension module doesn't work either because you really need the tie-in from types.py, inspect.py, and pydoc.py. You certainly don't want to go poking new values into types.py from the outside, and besides inspect.py and pydoc.py also need to learn about these fundamental built-in Python types. ISTM the most straightforward approach is to provide these types in a built-in C module, linked into the interpreter so types.py can get access to it as early as it needs to. Also, because the type that these descriptors are attached to cannot be instantiated from Python, they should be quite benign, existing only in order to give type() something to query. Neal also questions the reliance on the types.py module and whether we're moving away from it. My own feeling is that certainly for common types available as type constructors in the built-in namespace (e.g. list, set, dict, etc.), no, we certainly don't need types.py. But there are enough other tools in Pythonland that need to introspect about types that aren't readily available, and I see no reason why types.py shouldn't continue to fulfill those needs. How else would you find the type of frames or generators in a simple, documented, consistent way? Neal also asks about other implementations. I believe that I've patched types.py, inspect.py, and pydoc.py in such a way that if the implementation did not have these types available, those modules would not break. For example, GetSetDescriptorType and MemberDescriptorType are not defined if _types can't be imported. If those types aren't defined, the appropriate inspect.py is() methods will always return False. Thus, other implementations can do hasattr () tests on types.py or better yet, inspect.isgetsetdescriptor() and get an answer it can deal with. Further, by providing a _types built-in module, other implementations /could/ expose those types, if they were meaningful. I believe that an _types module provides a good place for those alternative implementations to hide types of their own should the need arise (e.g. if Jython had some implementation-specific JavaSpecialUnicornType, the object would live in Jython's _types.java and be exposed via a type(_type.javaspecialunicorn) in types.py). Could this wait until Python 2.6? Well, it /could/ but IMO it shouldn't. Improving the user experience by providing useful help() shouldn't be postponed when we have the opportunity to do so now, with low risk of breakage. Probably the biggest risk is whether I can modify the Windows build to DTRT with _types.c. I've been building enough Windows s/w lately that I think I can handle
Re: [Python-Dev] Support for PyGetSetDefs in pydoc
Barry Warsaw wrote: Why did I do this instead of trying to hunt down some existing getset or member descriptor? For one thing, there really aren't very good candidates for such objects in the built-in modules. You can't use objects like datetime.timedelta.days in types.py because datetime is not importable early enough for types.py to use it. Even if there were likely candidates, they would be accidents of implementation and it doesn't seem like a good idea to force force some future datetime maintainer to have to fix types.py when she decides that datetime.timedelta.days should be implemented in some other way. A 3rd party extension module doesn't work either because you really need the tie-in from types.py, inspect.py, and pydoc.py. You certainly don't want to go poking new values into types.py from the outside, and besides inspect.py and pydoc.py also need to learn about these fundamental built-in Python types. ISTM the most straightforward approach is to provide these types in a built-in C module, linked into the interpreter so types.py can get access to it as early as it needs to. Also, because the type that these descriptors are attached to cannot be instantiated from Python, they should be quite benign, existing only in order to give type() something to query. Perhaps you could put the objects into _testcapi. That way no new module has to be deployed (is _testcapi installed on every system?) Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Undocumented PEP 302 protocol change by need-for-speed sprint
At 12:28 PM 7/20/2006 -0700, Brett Cannon wrote: On 7/20/06, Phillip J. Eby mailto:[EMAIL PROTECTED][EMAIL PROTECTED] wrote: While investigating the need to apply http://python.org/sf/1525766http://python.org/sf/1525766 I found that there was a modification to pkgutil during the need-for-speed sprint that affects the PEP 302 protocol in a backwards incompatible way. Specifically, PEP 302 documents that path_importer_cache always contains either importer objects or None. Any code written to obtain importer objects is therefore now broken, because import.c is slapping False in for non-existent filesystem paths. The pkgutil module was then hacked to work around this problem, thereby hiding the breakage from at least the standard library, but not any external libraries that follow the PEP 302 protocol to find importers. There are several options as to how to proceed: 1. Revert the change 2. Document the breakage, update PEP 302, and make everybody update their code 3. Make it not break existing code, by using a NonexistentPathImporter or NullImporter type in place of False in sys.path_importer_cache. Any thoughts? Revert it. Is it really that much of a bonus to use False over None? Both evaluate to false and both are already singleton so you can use 'is' for testing. The changed code still uses None. PEP 302 defines None as meaning that a sys.path entry does not have an importer. It's just that the need-for-speed patch *adds* the use of True and False. None still means no importer, but True now means no importer, path exists and False now means no importer, path does not exist. The idea is that import.c can then skip checking the existence of the path when it sees True or False, but it then means that code that gets data from path_importer_cache needs to know about these new special values, or else it will get an attribute error when it tries to call True.find_module(). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Undocumented PEP 302 protocol change by need-for-speed sprint
On 7/20/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 12:28 PM 7/20/2006 -0700, Brett Cannon wrote:On 7/20/06, Phillip J. Ebymailto:[EMAIL PROTECTED][EMAIL PROTECTED] wrote:While investigating the need to applyhttp://python.org/sf/1525766http://python.org/sf/1525766 I foundthat there was a modification to pkgutil during the need-for-speed sprintthat affects the PEP 302 protocol in a backwards incompatible way.Specifically, PEP 302 documents that path_importer_cache always contains either importer objects or None.Any code written to obtain importerobjects is therefore now broken, because import.c is slapping False in fornon-existent filesystem paths. The pkgutil module was then hacked to work around this problem, therebyhiding the breakage from at least the standard library, but not anyexternal libraries that follow the PEP 302 protocol to find importers. There are several options as to how to proceed:1. Revert the change2. Document the breakage, update PEP 302, and make everybody update theircode3. Make it not break existing code, by using a NonexistentPathImporter or NullImporter type in place of False in sys.path_importer_cache.Any thoughts?Revert it.Is it really that much of a bonus to use False overNone?Both evaluate to false and both are already singleton so you can use 'is' for testing.The changed code still uses None.PEP 302 defines None as meaning that asys.path entry does not have an importer.It's just that theneed-for-speed patch *adds* the use of True and False.None still means no importer, but True now means no importer, path exists and False nowmeans no importer, path does not exist.Ah. Sounds like None is not really even needed with the change (although I am not suggesting the removal of None). The idea is that import.c can then skip checking the existence of the path when it sees True or False, but it then means that code that gets data frompath_importer_cache needs to know about these new special values, or elseit will get an attribute error when it tries to call True.find_module ().Well, I have not played with the PEP 302 stuff so I don't know how helpful they are to have around. But it is definitely a semantic change that either needs to be reverted or documented. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Undocumented PEP 302 protocol change by need-for-speed sprint
On Thu, 20 Jul 2006 14:57:07 -0400, Phillip J. Eby [EMAIL PROTECTED] wrote: While investigating the need to apply http://python.org/sf/1525766 I found that there was a modification to pkgutil during the need-for-speed sprint that affects the PEP 302 protocol in a backwards incompatible way. It just so happens that the bug that is reported was probably reported because I'm working on some controversial new functionality in Twisted - controversial because it replicates the functionality that bug is about in pkgutil. This functionality does make some use of PEP 302 functionality :). See http://twistedmatrix.com/trac/ticket/1940 Specifically, PEP 302 documents that path_importer_cache always contains either importer objects or None. Any code written to obtain importer objects is therefore now broken, because import.c is slapping False in for non-existent filesystem paths. Oddly, for once I'm going to say I don't care about this change. The code I've written so far doesn't depend on this, and I was pretty careful to be conservative about depending too much on the stuff described in PEP 302. It documents several features which don't exist (get_data, and methods in the imp module which don't exist in python2.3 or python2.4, where it was nominally accepted). There are several options as to how to proceed: 2. Document the breakage, update PEP 302, and make everybody update their code Personally I'd prefer it if PEP 302 were updated for a variety of reasons. It's very hard to use as a reference for writing actual code because so many features are optional or open issues, and there's no description in the PEP of what their status is. Better yet, this breakage (and other things) should be documented in the Python reference, and the PEP should link to the documentation for different versions, which can each describe the PEP's implementation status. The importing modules section of the library reference seems like a natural place to put it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for PyGetSetDefs in pydoc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Jul 20, 2006, at 3:32 PM, Georg Brandl wrote: Perhaps you could put the objects into _testcapi. That way no new module has to be deployed (is _testcapi installed on every system?) That doesn't seem importable in types.py either. You /could/ add that to Modules/config.c.in but I'm not sure it''s a good idea. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (Darwin) iQCVAwUBRL/rgHEjvBPtnXfVAQLnHwP/Tzy1Iy8nbByuVaYaGrhel9zsdlbRXXlj 8wvLLlmzctmnzBrPBpZKV7aJMdiBd6261SKqPCrDZCj2LEplZWRa7za0zm3O2iZu F1kbWCKTwWeI2/kVNpP1lTkm53u9yYp8grjxe1Q6fniB5nqQH11j79JehAYpBwdj /RsVYoeQxng= =lIEO -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] FW: Bug? Certainly a new *behavior* from subprocess in 2.5 on Win32
Larry Hastings wrote: I run the following script: -- from subprocess import * Popen(ls -l) -- (yeah, I have ls.exe on Windows) Under Python 2.4.2, this simply dumped the results of ls.exe to the terminal--sorry, to the command shell. Under Python 2.5, both beta 1 and beta 2, it dumps the results to the command shell, but *also* prints this: Exception exceptions.AttributeError: 'NoneType' object has no attribute 'append' in bound method Popen.__del__ of subprocess.Popen object at 0x00C04EB0 ignored Calling Popen() with a stdout = subprocess.PIPE does not throw this exception. I've asked Larry to raise this on SourceForge, but with the SF email problems I thought I'd better forward it here. Looks like there's a bug in Popen.__del__ in 2.5. I'm not in a position to have a look right now. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FW: Bug? Certainly a new *behavior* from subprocess in 2.5 on Win32
Reported to the list about a week ago, with analysis. Didn't get a response. Won't use sourceforge. Sorry about the top post.-KevinOn 7/20/06, Delaney, Timothy (Tim) [EMAIL PROTECTED] wrote:Larry Hastings wrote: I run the following script: -- from subprocess import * Popen(ls -l) -- (yeah, I have ls.exe on Windows) Under Python 2.4.2, this simply dumped the results of ls.exe to the terminal--sorry, to the command shell. Under Python 2.5, both beta 1 and beta 2, it dumps the results to the command shell, but *also* prints this: Exception exceptions.AttributeError: 'NoneType' object has no attribute 'append' in bound method Popen.__del__ of subprocess.Popen object at 0x00C04EB0 ignored Calling Popen() with a stdout = subprocess.PIPE does not throw this exception.I've asked Larry to raise this on SourceForge, but with the SF emailproblems I thought I'd better forward it here.Looks like there's a bug in Popen.__del__ in 2.5. I'm not in a positionto have a look right now.Tim Delaney___Python-Dev mailing listPython-Dev@python.org http://mail.python.org/mailman/listinfo/python-devUnsubscribe: http://mail.python.org/mailman/options/python-dev/jacobs%40bioinformed.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Behavior change in subprocess.py
Title: Message Hah - just found it. I even remember reading it... I'll update the SF tracker (1526203) with your analysis. Tim Delaney -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kevin Jacobs [EMAIL PROTECTED]Sent: Thursday, 13 July 2006 12:33 AMTo: python-dev@python.orgSubject: [Python-Dev] Behavior change in subprocess.pyDuring my testing of Python 2.5b2, I've found something that may be worthy of discussion. I suspect that recent GC and finalization changes have altered the behavior of the Popen object in subprocess.py. I am now getting many many many finalization warnings in my code like: Exception exceptions.AttributeError: "'NoneType' object has no attribute 'append'" in bound method Popen.__del__ of subprocess.Popen object at 0x2b910950 ignoredIs this a bug or a feature? Personally, I'd like to see these messages silenced, since it is being generated during interpreter shutdown. The following patch does the trick for me: --- /usr/local/lib/python2.5/subprocess.py 2006-07-11 14:11:59.0 -0400+++ subprocess.py 2006-07-12 10:17:09.0 -0400@@ -613,7 +613,7 @@ return # In case the child hasn't been waited on, check if it's done. self.poll(_deadstate=sys.maxint)- if self.returncode is None:+ if self.returncode is None and _active is not None: # Child is still running, keep us alive until we can wait on it. _active.append(self)Note that popen.py does something similar, though I am not convinced that the test is right or if it is doing something more subtle: def __del__(self): # In case the child hasn't been waited on, check if it's done. self.poll(_deadstate=sys.maxint) if self.sts 0: if _active: # Child is still running, keep us alive until we can wait on it. _active.append(self) Regards,-Kevin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Strategy for converting the decimal module to C
Nick Maclaren wrote: Now, interrupting into that level has to be transparent, in order to support TLB misses, clock interrupts, device interrupts, machine-check interrupts and so on. I thought we were just talking about counting the number of floating point exceptions that a particular piece of code generates. Surely that's deterministic, and isn't affected by any of that stuff? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] first draft of bug guidelines for www.python.org/dev/
Here is a first stab at writing up guidelines for people to follow when reporting bug. If this goes well I will also do ones for patches, committing, and PEPs. -Brett --- These sets of guidelines are to help you file a bug report for the Python programming language on SourceForge_. If your bug is not for the language but for a third-party application, please report the bug to that third-party. *Please make sure to follow every step as it will make the lives of the Python developers much easier!!!* .. contents:: Get a SourceForge account === In order to file a bug report, you must have an account_ on SourceForge_. We realize some people would like to have anonymous bug reporting for various reasons (anonymity, ease or reporting, etc.). But SourceForge does not support anonymous reporting. Plus, by registering, you are notified by email when any action is been taken on your report. This can be very important if a Python developer needs more information from you about the bug. Start a new bug === You must be logged into SourceForge to file a bug! See `Get a SourceForge account`_ if you do not have one. Go to the `SourceForge bug page`_ to start a new bug report. There you will find a link called `Submit New`_. Click on it and it will allow you to fill out a new bug report. Once you click on the link, you are presented with a page that has several fields for you to fill in. Here is what to do for each field: * Category Set this to the area that the bug is related to (e.g., documentation, build, etc.). * Group Usually this is set the major.minor version of Python that you found the bug in. * Assigned To Leave this alone * Priority Leave this alone * Summary A one-line describing the problem so as to make it easy for developers to spot whether they have the expertise needed to work on the bug. * Detailed Description Following sections of this document discuss what should go in here. * Check to Upload and Attach a File If you are going to upload a file, you *must* check this box. * File Location Field Click the Browse button to upload any file to accompany your bug report (usually a succinct way to reproduce the bug). * File Description A one-line describing the file; no date info is needed since the upload is timestamped. Specify Python version === It is important that we have the most accurate version number of the interpreter you are using in order to best diagnose the issue. There are two ways to get us the version information. If you can run your Python interpreter, execute the following lines at an interpreter and paste the result into the ``Detailed Description`` field of the bug report:: import sys print sys.version If you are running a version of Python newer than 2.4 and are working from a source checkout of Python, the please also report the Subversion revision number for the root of your checkout:: python/trunk$ svnversion . If your bug is preventing you from running the interpreter, execute Python with teh ``-V`` command-line flag and paste the output:: python/trunk$ python -V Special settings for your Python interpreter Sometimes your environment influences a bug and thus needs to be reported to help find the problem. This means we need to have reported: * Operating System * Environment Variables + PYTHONSTARTUP If this is set and might be causing the issue, please either upload the file or state what it does. + PYTHONCASEOK If your bug is on Windows and involves importing, please report if this environment variable is set or not. * site-packages If you have andy third-party packages installed that may be contributing to the bug, please report those. * Custom Patches Any differences between your code and the code the Python developers work off of needs to be reported. Sample code to reproduce bug If you can, please upload a file the demonstrates the bug. The more succinct the better! And please do not forget to check the upload checkbox in the bug report. Submit! At this point you should have a detailed bug report for developers to work off of. Click the ``Submit`` button and read on to see what you should do after the bug is reported. Respond to requests from developers No matter how detailed the bug report, there is always the off-chance that a developer will need more information to fix a bug. Please be prompt in replying to requests for information by submitting a reply on the bug report. You may be asked to test out a patch. It is very important that you help with this, especially if the bug is not reproducible by the developer working on it. Write
Re: [Python-Dev] first draft of bug guidelines for www.python.org/dev/
Brett Cannon: But SourceForge does not support anonymous reporting. SourceForge does support anonymous reporting. A large proportion of the fault reports I receive for Scintilla are anonymous as indicated by nobody in the Submitted By column. https://sourceforge.net/tracker/?group_id=2439atid=102439 Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] first draft of bug guidelines for www.python.org/dev/
On Friday 21 July 2006 00:10, Neil Hodgson wrote: Brett Cannon: But SourceForge does not support anonymous reporting. SourceForge does support anonymous reporting. A large proportion of the fault reports I receive for Scintilla are anonymous as indicated by nobody in the Submitted By column. SourceForge supports anonymous reporting, but the Python project determined that the management cost of anonymous reports was higher than the value they provided. It might be time to reconsider that decision (though my position hasn't changed). -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for PyGetSetDefs in pydoc
Barry Warsaw [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] think this patch should go in 2.5. OTOH, I suspect most people just don't care, which is why I've gotten almost no comments on the patch (other than one or two mild nods of approval). I use help(ob) quite a bit, have noticed that somethings don't give anything useful, and definitely approve of making help more useful. tjr ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
