Re: [Python-Dev] Criticism of execfile() removal in Python3
On 10 June 2014 12:23, Paul Sokolovsky pmis...@gmail.com wrote: 1. It hampers interactive mode - instead of short and easy to type execfile(file.py) one needs to use exec(open(file.py).read()). I'm sure that's not going to bother a lot of people - after all, the easiest way to execute a Python file is to drop back to shell and restart python with file name, using all wonders of tab completion. But now imagine that Python interpreter runs on bare hardware, and its REPL is the only shell. That's exactly what we have with MicroPython's Cortex-M port. But it's not really MicroPython-specific, there's CPython port to baremetal either - http://www.pycorn.org/ . https://docs.python.org/3/library/runpy.html#runpy.run_path import runpy file_globals = runpy.run_path(file.py) The standard implementation of run_path reads the whole file into memory, but MicroPython would be free to optimise that and do statement by statement execution instead (while that will pose some challenges in terms of handling encoding cookies, future imports, etc correctly, it's certainly feasible). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
On 10 June 2014 05:02, Ben Hoyt benh...@gmail.com wrote: To solve this problem, what do people think about adding an st_winattrs attribute to the object returned by os.stat() on Windows? +1. Given the precedent of Linux- and OS X-specific attributes, this seems like a no-brainer to me. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Criticism of execfile() removal in Python3
On 10 June 2014 08:36, Nick Coghlan ncogh...@gmail.com wrote: The standard implementation of run_path reads the whole file into memory, but MicroPython would be free to optimise that and do statement by statement execution instead (while that will pose some challenges in terms of handling encoding cookies, future imports, etc correctly, it's certainly feasible). ... and if they did optimise that way, I would imagine that the patch would be a useful contribution back to the core Python stdlib, rather than remaining a MicroPython-specific optimisation. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Criticism of execfile() removal in Python3
On 10 Jun 2014 18:41, Paul Moore p.f.mo...@gmail.com wrote: On 10 June 2014 08:36, Nick Coghlan ncogh...@gmail.com wrote: The standard implementation of run_path reads the whole file into memory, but MicroPython would be free to optimise that and do statement by statement execution instead (while that will pose some challenges in terms of handling encoding cookies, future imports, etc correctly, it's certainly feasible). ... and if they did optimise that way, I would imagine that the patch would be a useful contribution back to the core Python stdlib, rather than remaining a MicroPython-specific optimisation. I believe it's a space/speed trade-off, so I'd be surprised if it made sense for CPython in general. There are also some behavioural differences when it comes to handling syntax errors. Now that I think about the idea a bit more, if the MicroPython folks can get a low memory usage incremental file execution model working, the semantic differences mean it would likely make the most sense as a separate API in runpy, rather than as an implicit change to run_path. Cheers, Nick. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
2014-06-10 6:02 GMT+02:00 Ben Hoyt benh...@gmail.com: To solve this problem, what do people think about adding an st_winattrs attribute to the object returned by os.stat() on Windows? (...) FILE_ATTRIBUTE_HIDDEN = 2 # constant defined in Windows.h if hasattr(st, 'st_winattrs') and st.st_winattrs FILE_ATTRIBUTE_HIDDEN: I don't like such API, it requires to import constants, use masks, etc. I would prefer something like: if st.win_hidden: ... Or maybe: if st.winattrs.hidden: ... Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
On 2014-06-10 05:02, Ben Hoyt wrote: [snip] FILE_ATTRIBUTE_HIDDEN = 2 # constant defined in Windows.h def is_hidden(path): if startswith(os.path.basename(path), '.'): return True st = os.stat(path) if hasattr(st, 'st_winattrs') and st.st_winattrs FILE_ATTRIBUTE_HIDDEN: That could be written more succinctly as: if getattr(st, 'st_winattrs', 0) FILE_ATTRIBUTE_HIDDEN: return True return False ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
FILE_ATTRIBUTE_HIDDEN = 2 # constant defined in Windows.h if hasattr(st, 'st_winattrs') and st.st_winattrs FILE_ATTRIBUTE_HIDDEN: I don't like such API, it requires to import constants, use masks, etc. I would prefer something like: if st.win_hidden: ... Or maybe: if st.winattrs.hidden: ... Yes, fair call. However, it looks like the precent for the attributes in os.stat()'s return value has long since been set -- this is OS-specific stuff. For example, what's in st_flags? It's not documented, but comes straight from the OS. Same with st_rdev, st_type, etc -- the documentation doesn't define them, and it looks like they're OS-specific values. I don't think the st.win_hidden approach gains us much, because the next person is going to ask for the FILE_ATTRIBUTE_ENCRYPTED or FILE_ATTRIBUTE_COMPRESSED flag. So we really need all the bits or nothing. I don't mind the st.st_winattrs.hidden approach, except that we'd need 17 sub-attributes, and they'd all have to be documented. And if Windows added another attribute, Python wouldn't have it, etc. So I think the OS-defined constant is the way to go. Because these are fixed-forever constants, I suspect in library code and the like people would just KISS and use an integer literal and a comment, avoiding the import/constant thing: if getattr(st, 'st_winattrs', 0) 2: # FILE_ATTRIBUTE_HIDDEN ... -Ben ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
On 10 June 2014 13:19, Ben Hoyt benh...@gmail.com wrote: Because these are fixed-forever constants, I suspect in library code and the like people would just KISS and use an integer literal and a comment, avoiding the import/constant thing: The stat module exposes a load of constants - why not add the (currently known) ones there? Finding the values of Windows constants if you don't have access to the C headers can be a pain, so having them defined *somewhere* as named values is useful. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
The stat module exposes a load of constants - why not add the (currently known) ones there? Finding the values of Windows constants if you don't have access to the C headers can be a pain, so having them defined *somewhere* as named values is useful. So stat.FILE_ATTRIBUTES_HIDDEN and the like? Alternatively they could go in ctypes.wintypes, but I think stat makes more sense in this case. -Ben ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Criticism of execfile() removal in Python3
On Tue, 10 Jun 2014 19:07:40 +1000, Nick Coghlan ncogh...@gmail.com wrote: On 10 Jun 2014 18:41, Paul Moore p.f.mo...@gmail.com wrote: On 10 June 2014 08:36, Nick Coghlan ncogh...@gmail.com wrote: The standard implementation of run_path reads the whole file into memory, but MicroPython would be free to optimise that and do statement by statement execution instead (while that will pose some challenges in terms of handling encoding cookies, future imports, etc correctly, it's certainly feasible). ... and if they did optimise that way, I would imagine that the patch would be a useful contribution back to the core Python stdlib, rather than remaining a MicroPython-specific optimisation. I believe it's a space/speed trade-off, so I'd be surprised if it made sense for CPython in general. There are also some behavioural differences when it comes to handling syntax errors. Now that I think about the idea a bit more, if the MicroPython folks can get a low memory usage incremental file execution model working, the semantic differences mean it would likely make the most sense as a separate API in runpy, rather than as an implicit change to run_path. If it is a separate API, it seems like there's no reason it couldn't be contributed back to CPython. There might be other contexts in which low memory would be the right tradeoff. Although, if key bits end up working at the C level, contributing back might require writing separate C for CPython, so that might not happen. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Criticism of execfile() removal in Python3
On 10 June 2014 23:05, R. David Murray rdmur...@bitdance.com wrote: On Tue, 10 Jun 2014 19:07:40 +1000, Nick Coghlan ncogh...@gmail.com wrote: I believe it's a space/speed trade-off, so I'd be surprised if it made sense for CPython in general. There are also some behavioural differences when it comes to handling syntax errors. Now that I think about the idea a bit more, if the MicroPython folks can get a low memory usage incremental file execution model working, the semantic differences mean it would likely make the most sense as a separate API in runpy, rather than as an implicit change to run_path. If it is a separate API, it seems like there's no reason it couldn't be contributed back to CPython. There might be other contexts in which low memory would be the right tradeoff. Although, if key bits end up working at the C level, contributing back might require writing separate C for CPython, so that might not happen. Yeah, as a separate API it could make sense in CPython - I just didn't go back and revise the first paragraph after writing the second one :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
On 10 June 2014 13:58, Ben Hoyt benh...@gmail.com wrote: So stat.FILE_ATTRIBUTES_HIDDEN and the like? Yep. (Maybe WIN_FILE_ATTRIBUTES_HIDDEN, but the Unix ones don't have an OA name prefix, so I'd go with your original). Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python 3.5 on VC14 - update
For anyone who is interested in more details on the CRT changes, there's a blog post from my colleague who worked on most of them at http://blogs.msdn.com/b/vcblog/archive/2014/06/10/the-great-crt-refactoring.aspx I wanted to call out one section and add some details: In order to unify these different CRTs [desktop, phone, etc], we have split the CRT into three pieces: 1. VCRuntime (vcruntime140.dll): This DLL contains all of the runtime functionality required for things like process startup and exception handling, and functionality that is coupled to the compiler for one reason or another. We may need to make breaking changes to this library in the future. 2. AppCRT (appcrt140.dll): This DLL contains all of the functionality that is usable on all platforms. This includes the heap, the math library, the stdio and locale libraries, most of the string manipulation functions, the time library, and a handful of other functions. We will maintain backwards compatibility for this part of the CRT. 3. DesktopCRT (desktopcrt140.dll): This DLL contains all of the functionality that is usable only by desktop apps. Notably, this includes the functions for working with multibyte strings, the exec and spawn process management functions, and the direct-to-console I/O functions. We will maintain backwards compatibility for this part of the CRT. The builds of Python I've already made are indeed linked against these three DLLs, though it happens transparently. Most of the APIs are from the AppCRT, which is a good sign as it will simplify portability to other Windows-based platforms (though the direct references to the Win32 API will arise again to complicate this). Very few functions are imported from VCRuntime, which is the only part that *may* have breaking changes in the future (that's the current promise, and I'd expect it to be strengthened one way or the other by releas). Apart from the standard memcpy/strcpy type functions (which may be moved in later builds), these other imports are compiler helpers: * void terminate(void) (currently exported as a decorated C++ function, but that's going to be fixed) * __vcrt_TerminateProcess * __vcrt_UnhandledException * __vcrt_cleanup_type_info_names * _except_handler4_common * _local_unwind4 I've checked with our CRT dev and he says that these don't keep any state (and won't cause problems like we've seen in the past with FILE*), and are only there to deal with potential C++ exceptions - they are included at a point where it is impossible to tell whether C++ is involved, and so can't be removed. My builds pass almost all of regrtest.py and the only issues are with Tcl/tk and OpenSSL, which need to update their compiler version detection. I've built them with changes, though as usual Tcl/tk is a real pain. I ran a quick test with profile-guided optimization (PGO, pronounced pogo), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?) Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
On 06/09/2014 09:02 PM, Ben Hoyt wrote: To solve this problem, what do people think about adding an st_winattrs attribute to the object returned by os.stat() on Windows? +1 to the idea, whatever the exact implementation. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
On Tue, Jun 10, 2014 at 12:17 PM, Ethan Furman et...@stoneleaf.us wrote: On 06/09/2014 09:02 PM, Ben Hoyt wrote: To solve this problem, what do people think about adding an st_winattrs attribute to the object returned by os.stat() on Windows? +1 to the idea, whatever the exact implementation. Agreed. -- Zach ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 on VC14 - update
Le 10/06/2014 12:30, Steve Dower a écrit : I ran a quick test with profile-guided optimization (PGO, pronounced pogo), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?) I would recommend using the non-trivial suite of benchmarks at http://hg.python.org/benchmarks (both for the profiling and the benchmarking, though you may want to use additional workloads for profiling too) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
I wonder how a hybrid approach would work? Use a dynamically-created class, but then construct the __new__ method using exec and inject it into the new class. As far as I can see, it's only __new__ that benefits from the exec approach. Anyone tried this yet? Is it worth an experiment? I'm not sure what the benefit would be. Other than the ast manipulations for __new__, the rest of the non-exec code is easy to understand. I misread this, sorry. This might work for collections.namedtuple, but is probably not worth the hassle or churn of changing it. The main reason I switched to ast for namedlist is because generating the text version of __new__ or __init__ with default parameter values was extremely difficult, so an approach of exec-ing that one function wouldn't work for me. Eric. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 on VC14 - update
Antoine Pitrou wrote: Le 10/06/2014 12:30, Steve Dower a écrit : I ran a quick test with profile-guided optimization (PGO, pronounced pogo), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?) I would recommend using the non-trivial suite of benchmarks at http://hg.python.org/benchmarks (both for the profiling and the benchmarking, though you may want to use additional workloads for profiling too) Regards Antoine. Thanks. I knew there was a proper set somewhere, but didn't manage to track it down in the minute or so I spent looking :) Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Documentation Oversight
From the csv module pydoc: The optional dialect parameter is discussed below The discussion is actually above the method. Present in 2.7.6. -- H -- Sent from my mobile device Envoyé de mon portable ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
To solve this problem, what do people think about adding an st_winattrs attribute to the object returned by os.stat() on Windows? +1 to the idea, whatever the exact implementation. Cool. I think we should add a st_winattrs integer attribute (on Windows) and then also add the FILE_ATTRIBUTES_* constants to stat.py per Paul Moore. What would be the next steps to get this to happen? Open an issue on bugs.python.org and submit a patch with tests? -Ben ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning Windows file attribute information via os.stat()
On Tue, Jun 10, 2014 at 2:04 PM, Ben Hoyt benh...@gmail.com wrote: To solve this problem, what do people think about adding an st_winattrs attribute to the object returned by os.stat() on Windows? +1 to the idea, whatever the exact implementation. Cool. I think we should add a st_winattrs integer attribute (on Windows) and then also add the FILE_ATTRIBUTES_* constants to stat.py per Paul Moore. Add to _stat.c rather than stat.py. What would be the next steps to get this to happen? Open an issue on bugs.python.org and submit a patch with tests? Yep! -- Zach ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Documentation Oversight
On 6/10/2014 2:51 PM, Hasan Diwan wrote: From the csv module pydoc: The optional dialect parameter is discussed below The discussion is actually above the method. Present in 2.7.6. Bug reports should be posted on the tracker rather than sent here. Short doc reports like this can be sent to d...@python.org. Also, the docs are continuous updated. Reports should be based on the current version as docs.python.org. As it turns out, this sentence is not in the current Doc/library/csv.rst or the online version at https://docs.python.org/3/library/csv.html#module-csv If this is what you meant, something has been changed. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 on VC14 - update
2014-06-10 18:30 GMT+02:00 Steve Dower steve.do...@microsoft.com: I ran a quick test with profile-guided optimization (PGO, pronounced pogo), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?) If we upgrade the compiler on Windows, some optimizer options can maybe be enabled again. Previous Visual Studio (2010?) bugs: * http://bugs.python.org/issue15993 * http://bugs.python.org/issue8847#msg166935 Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
Am 07.06.14 01:01, schrieb Steve Dower: We keep the VS 2010 files around and make sure they keep working. This is the biggest risk of the whole plan, but I believe that there's enough of a gap between when VS 14 is planned to release (which I know, but can't share) and when Python 3.5 is planned (which I don't know, but have a semi-informed guess). By keep around, I'd be fine with in a subdirectory of PC. PCbuild should either switch for sure, or not switch at all. People had proposed to come up with a PCbuildN directory (N=10, N=14, or whatever) to maintain two build environments simultaneously; I'd be -1 on such a plan. There needs to be one official toolset to build Python X.Y with, and it needs to be either VS 2010 or VS 2014, but not both. Is Python 3.5b1 being built with VS 14 RC (hypothetically) a blocking issue? Do we need to resolve that now or can it wait until it happens? It's up to the release manager, but I'd personally see it as a blocking issue: we shouldn't use a beta compiler for the final release, and we shouldn't switch compilers (back) after b1. The RM *could* opt to bet on VS 14 RTM appearing before 3.5rc1 is released (or otherwise blocking rc1 until VS 14 is released); I would consider this risy, but possibly worth it. We certainly don't need to resolve this now. We should discuss it again when the release schedule for 3.5 is proposed. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
Am 07.06.14 17:38, schrieb Steve Dower: One more possible concern that I just thought of is the availability of the build tools on Windows Vista and Windows 7 RTM (that is, without SP1). I'd have to check, but I don't believe anything after VS 2012 is supported on Vista and it's entirely possible that installation is blocked. I wouldn't worry about that. People can be asked to update their build machines (within reason), as long as the resulting binaries should work on older systems still. There are testing issues, of course, but they show up even in other cases, like testing whether a 32-bit installer actually runs on a 32-bit system when the build system is a 64-bit system; such issues will always exist. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 on VC14 - update
Am 10.06.14 18:30, schrieb Steve Dower: I ran a quick test with profile-guided optimization (PGO, pronounced pogo), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?) You probably ran too little Python code. See PCbuild/build_pgo.bat for what used to be part of the release process. It takes quite some time, but it rebuilt more than 1% (IIRC). FWIW, I stopped using PGO for the official releases when it was demonstrated to generate bad code. In my experience, a compiler that generates bad code has lost trust forever, so it will be hard to justify re-enabling PGO (like but it really works this time). I wasn't sad when I found a justification to skip the profiling, since it significantly held up the release process. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 on VC14 - update
Martin v. Löwis wrote: Am 10.06.14 18:30, schrieb Steve Dower: I ran a quick test with profile-guided optimization (PGO, pronounced pogo), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?) You probably ran too little Python code. See PCbuild/build_pgo.bat for what used to be part of the release process. It takes quite some time, but it rebuilt more than 1% (IIRC). That's almost certainly the case. I didn't run anywhere near enough to call it good, though I'd only really expect the size to get worse and the speed to get better. FWIW, I stopped using PGO for the official releases when it was demonstrated to generate bad code. In my experience, a compiler that generates bad code has lost trust forever, so it will be hard to justify re-enabling PGO (like but it really works this time). I wasn't sad when I found a justification to skip the profiling, since it significantly held up the release process. Yeah, and it seems the bad code is still there. I suspect it's actually due to optimizing for space rather than speed, and not due to PGO directly, but either way I'll be trying to get it fixed. [EARLIER EMAIL] By keep around, I'd be fine with in a subdirectory of PC. PCbuild should either switch for sure, or not switch at all. People had proposed to come up with a PCbuildN directory (N=10, N=14, or whatever) to maintain two build environments simultaneously; I'd be -1 on such a plan. There needs to be one official toolset to build Python X.Y with, and it needs to be either VS 2010 or VS 2014, but not both. That's what I have planned. Right now it's in my sandbox and I've just replaced the existing PCbuild contents (rather wholesale - I took the opportunity to simplify the files, which is important to me as I spend most of my time editing them by hand rather than through VS). When/if I merge, the version in PC\VS10.0 will be exactly what was there at merge time. Regards, Martin And thanks, I appreciate the context and suggestions. Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 on VC14 - update
On Tue, Jun 10, 2014 at 9:30 AM, Steve Dower steve.do...@microsoft.com wrote: I ran a quick test with profile-guided optimization (PGO, pronounced pogo), which has supposedly been improved since VC9, and saw a very unscientific 20% speed improvement on pybench.py and 10% size reduction in python35.dll. I'm not sure what we used to get from VC9, but it certainly seems worth enabling provided it doesn't break anything. (Interestingly, PGO decided that only 1% of functions needed to be compiled for speed. Not sure if I can find out which ones those are but if anyone's interested I can give it a shot?) For what it's worth, we build Google's internal Python interpreters with gcc's flavour of PGO and are seeing somewhat more than 20% performance increase for Python 2.7. (We train using most of the testsuite, not pybench, and I believe the Debian/Ubuntu packages also do this.) I believe almost all of that is from speedups to the main eval loop, which is a huge switch in a bigger loop with complicated jump logic. It wouldn't surprise me if VS's PGO only decided to optimize that eval loop :) -- Thomas Wouters tho...@python.org Hi! I'm an email virus! Think twice before sending your email to help me spread! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Why does IOBase.__del__ call .close?
Hello,
I recently noticed (after some rather protacted debugging) that the
io.IOBase class comes with a destructor that calls self.close():
[0] nikratio@vostro:~/tmp$ cat test.py
import io
class Foo(io.IOBase):
def close(self):
print('close called')
r = Foo()
del r
[0] nikratio@vostro:~/tmp$ python3 test.py
close called
To me, this came as quite a surprise, and the best documentation of
this feature seems to be the following note (from the io library
reference):
The abstract base classes also provide default implementations of some
methods in order to help implementation of concrete stream classes. For
example, BufferedIOBase provides unoptimized implementations of
readinto() and readline().
For me, having __del__ call close() does not qualify as a reasonable
default implementation unless close() is required to be idempotent
(which one could deduce from the documentation if one tries to, but it's
far from clear).
Is this behavior an accident, or was that a deliberate decision?
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does IOBase.__del__ call .close?
On 2014-06-11 02:30, Nikolaus Rath wrote:
Hello,
I recently noticed (after some rather protacted debugging) that the
io.IOBase class comes with a destructor that calls self.close():
[0] nikratio@vostro:~/tmp$ cat test.py
import io
class Foo(io.IOBase):
def close(self):
print('close called')
r = Foo()
del r
[0] nikratio@vostro:~/tmp$ python3 test.py
close called
To me, this came as quite a surprise, and the best documentation of
this feature seems to be the following note (from the io library
reference):
The abstract base classes also provide default implementations of some
methods in order to help implementation of concrete stream classes. For
example, BufferedIOBase provides unoptimized implementations of
readinto() and readline().
For me, having __del__ call close() does not qualify as a reasonable
default implementation unless close() is required to be idempotent
(which one could deduce from the documentation if one tries to, but it's
far from clear).
Is this behavior an accident, or was that a deliberate decision?
To me, it makes sense. You want to make sure that it's closed, releasing
any resources it might be holding, even if you haven't done so
explicitly.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does IOBase.__del__ call .close?
Le 10/06/2014 21:30, Nikolaus Rath a écrit : For me, having __del__ call close() does not qualify as a reasonable default implementation unless close() is required to be idempotent (which one could deduce from the documentation if one tries to, but it's far from clear). close() should indeed be idempotent on all bundled IO class implementations (otherwise it's a bug), and so should it preferably on third-party IO class implementations. If you want to improve the documentation on this, you're welcome to provide a patch! Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
