Re: [Python-Dev] __file__ is not always an absolute path
Antoine Pitrou wrote: Barry Warsaw barry at python.org writes: exarkun at boson:~$ python -m timeit -s 'from os import getcwd' 'getcwd()' 100 loops, best of 3: 1.02 usec per loop [...] I'd like to see the effect on command line scripts that are run often and then exit, e.g. Bazaar or Mercurial. Start up time due to import overhead seems to be a constant battle for those types of projects. If os.getcwd() is only called once when normalizing sys.path, and if it just takes one microsecond, I don't really see the point. :-) The problem is that having '' as the first entry in sys.path currently means do the import relative to the current directory. Unless we want to change the language semantics so we stick os.getcwd() at the front instead of '', then __file__ is still going to be relative sometimes. Alternatively, we could special case those specific imports to do os.getcwd() at the time of the import. That won't affect the import speed significantly for imports from locations other than '' (i.e. most of them) and will more accurately reflect the true meaning of __file__ in that case (since we put the module in sys.modules, future imports won't see different versions of that module even if the working directory is changed, so the relative value for __file__ becomes a lie as soon as the working directory changes) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__ is not always an absolute path
Nick Coghlan ncoghlan at gmail.com writes: The problem is that having '' as the first entry in sys.path currently means do the import relative to the current directory. Unless we want to change the language semantics so we stick os.getcwd() at the front instead of '', then __file__ is still going to be relative sometimes. Changing the language semantics is actually what I was thinking about :) Do some people actually rely on the fact that changing the current directory will also change the import path? cheers Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3147: PYC Repository Directories
Ron Adam wrote: To tell the truth in most cases I hardly notice the extra time the first run takes compared to later runs with the precompiled byte code. Yes it may be a few seconds at start up, but after that it's usually not a big part of the execution time. Hmmm, I wonder if there's a threshold in file size where it really doesn't make a significant difference? It's relative to runtime for the application itself (long-running applications aren't going to notice as much of a percentage effect on runtime) as well as to how many Python files are actually imported at startup (only importing a limited number of modules, importing primarily extension modules or effective use of a lazy module loading mechanism will all drastically reduce the proportional impact of precompiled bytecode) We struggle enough with startup time that doing anything that makes it slower is rather undesirable though. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__ is not always an absolute path
Antoine Pitrou wrote: Nick Coghlan ncoghlan at gmail.com writes: The problem is that having '' as the first entry in sys.path currently means do the import relative to the current directory. Unless we want to change the language semantics so we stick os.getcwd() at the front instead of '', then __file__ is still going to be relative sometimes. Changing the language semantics is actually what I was thinking about :) Do some people actually rely on the fact that changing the current directory will also change the import path? I've learned that no matter how insane our current semantics for something may be, someone, somewhere will be relying on them :) In this case, the current semantics aren't even all that insane. A bit odd maybe, but not insane. I think they're even documented, but I couldn't say exactly where without some digging. I think we also use the trick of checking for an empty string in sys.path[0] in a couple of places before deciding whether or not to remove it (I seem to recall applying a patch to pydoc along those lines so it worked properly with the -m switch). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__ is not always an absolute path
On Mon, Feb 08, 2010 at 12:51:22PM +, Antoine Pitrou wrote: Do some people actually rely on the fact that changing the current directory will also change the import path? On the interactive prompt, yes. But I guess that's a habit that could be easily un-learnt. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3147: PYC Repository Directories
On 2/8/2010 7:54 AM, Nick Coghlan wrote: Ron Adam wrote: To tell the truth in most cases I hardly notice the extra time the first run takes compared to later runs with the precompiled byte code. Yes it may be a few seconds at start up, but after that it's usually not a big part of the execution time. Hmmm, I wonder if there's a threshold in file size where it really doesn't make a significant difference? It's relative to runtime for the application itself (long-running applications aren't going to notice as much of a percentage effect on runtime) as well as to how many Python files are actually imported at startup (only importing a limited number of modules, importing primarily extension modules or effective use of a lazy module loading mechanism will all drastically reduce the proportional impact of precompiled bytecode) We struggle enough with startup time that doing anything that makes it slower is rather undesirable though. Definitely. I have even wondered whether it would be possible to cache not just the bytecode for initializing a module, but also the initialized module itself (perhaps minus the name bindings for other imported modules). Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385 progress report
On Sun, Feb 7, 2010 at 22:51, Benjamin Peterson benja...@python.org wrote: How about a week after, so we have more time to adjust release procedures? Sounds fine to me. Will you do test conversions of the sandbox projects, too? Got any particular projects in mind? Also I think we should have some document (perhaps the dev FAQ) explaining exactly how to do common tasks in mercurial. For example - A bug fix, which needs to be in 4 branches. - A bug fix, which only belongs in 2.7 and 2.6 or 3.2 and 3.1. - Which way do we merge (What's a subset of what?) Yes, writing lots of docs is part of the plan. Cheers, Dirkjan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385 progress report
On Sun, Feb 7, 2010 at 22:58, Mark Hammond skippy.hamm...@gmail.com wrote: Isn't setting a date premature while outstanding issues remain without a timetable for their resolution? If we set a date, that would imply a timetable for their resolution. See http://mercurial.selenic.com/wiki/EOLTranslationPlan#TODO - of particular note: * There are transient errors in the tests which Martin is yet to identify. These tests do not require windows to reproduce or fix. * The mercurial tests do not run on Windows. Given the above, most sane Windows developers would hold off on live testing of the extension until at least the first issue is resolved - but the second issue makes it very difficult for them to help resolve that. The Mercurial tests can actually run on Windows -- and I've updated the page to that effect. They require something called pysh, though. I've also asked Patrick Mezard to include the eol extension in his nightly test run on Windows. I guess since some of the test errors do not require Windows to reproduce or fix, I'd invite anyone to jump in and help fix these issues. Cheers, Dirkjan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385 progress report
2010/2/8 Dirkjan Ochtman dirk...@ochtman.nl: On Sun, Feb 7, 2010 at 22:51, Benjamin Peterson benja...@python.org wrote: Will you do test conversions of the sandbox projects, too? Got any particular projects in mind? 2to3. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython
Hi Craig, On Tue, Feb 2, 2010 at 4:42 PM, Craig Citro craigci...@gmail.com wrote: Done. The diff is at http://codereview.appspot.com/186247/diff2/5014:8003/7002. I listed Cython, Shedskin and a bunch of other alternatives to pure CPython. Some of that information is based on conversations I've had with the respective developers, and I'd appreciate corrections if I'm out of date. Well, it's a minor nit, but it might be more fair to say something like Cython provides the biggest improvements once type annotations are added to the code. After all, Cython is more than happy to take arbitrary Python code as input -- it's just much more effective when it knows something about types. The code to make Cython handle closures has just been merged ... hopefully support for the full Python language isn't so far off. (Let me know if you want me to actually make a comment on Rietveld ...) Indeed, you're quite right. I've corrected the description here: http://codereview.appspot.com/186247/diff2/7005:9001/10001 Now what's more interesting is whether or not U-S and Cython could play off one another -- take a Python program, run it with some generic input data under Unladen and record info about which functions are hot, and what types they tend to take, then let Cython/gcc -O3 have a go at these, and lather, rinse, repeat ... JIT compilation and static compilation obviously serve different purposes, but I'm curious if there aren't other interesting ways to take advantage of both. Definitely! Someone approached me about possibly reusing the profile data for a feedback-enhanced code coverage tool, which has interesting potential, too. I've added a note about this under the Future Work section: http://codereview.appspot.com/186247/diff2/9001:10002/9003 Thanks, Collin Winter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385 progress report
Benjamin Peterson wrote: 2010/2/8 Dirkjan Ochtman dirk...@ochtman.nl: On Sun, Feb 7, 2010 at 22:51, Benjamin Peterson benja...@python.org wrote: Will you do test conversions of the sandbox projects, too? Got any particular projects in mind? 2to3. Does Mercurial even support merge tracking the way we are doing it for 2to3 right now? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385 progress report
2010/2/8 Martin v. Löwis mar...@v.loewis.de: Benjamin Peterson wrote: 2010/2/8 Dirkjan Ochtman dirk...@ochtman.nl: On Sun, Feb 7, 2010 at 22:51, Benjamin Peterson benja...@python.org wrote: Will you do test conversions of the sandbox projects, too? Got any particular projects in mind? 2to3. Does Mercurial even support merge tracking the way we are doing it for 2to3 right now? I don't believe so. My plan was to manually sync updates or use subrepos. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
