Re: [Python-Dev] __file__ and bytecode-only

2010-03-22 Thread Barry Warsaw
On Mar 16, 2010, at 07:44 PM, Nick Coghlan wrote: In Python 3.1, *invoking* py_compile.compile() will create 2.x style bytecode. Similarly, when force==False, compileall.compile_dir() and compileall.compile_path() will check for 2.x style bytecode in order to decide whether or not to compile the

Re: [Python-Dev] __file__ and bytecode-only

2010-03-16 Thread Nick Coghlan
Barry Warsaw wrote: On Mar 14, 2010, at 12:17 AM, Nick Coghlan wrote: While it's probably OK if the import side-effects only create files using the new scheme, the standard library modules will likely need to support both schemes (although I'm not sure if same as import system or same as

Re: [Python-Dev] __file__ and bytecode-only

2010-03-15 Thread Barry Warsaw
On Mar 14, 2010, at 12:17 AM, Nick Coghlan wrote: Hmm - methinks the PEP actually needs to talk explicitly about the py_compile and compileall modules. These compile the files directly rather than using the import system's side-effect, so they'll need to understand the intricacies of the new

Re: [Python-Dev] __file__ and bytecode-only

2010-03-15 Thread Barry Warsaw
On Mar 14, 2010, at 04:37 PM, Paul Moore wrote: The bdist_wininst installer also compiles modules explicitly on install (as does the python.org Windows MSI installer). I've always assumed that this worked via compileall, but haven't checked. Regardless, these should probably also be covered in

Re: [Python-Dev] __file__ and bytecode-only

2010-03-15 Thread Barry Warsaw
On Mar 15, 2010, at 07:43 AM, Nick Coghlan wrote: He did (in favour of keeping the directory visible). http://www.mail-archive.com/python-dev@python.org/msg45203.html (added to pep) -Barry signature.asc Description: PGP signature ___ Python-Dev

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Jon Ribbens
On Fri, Mar 12, 2010 at 05:56:57PM -0500, Barry Warsaw wrote: Nope, sorry I should have been clearer. *creation* is the key here. As per BDFL pronouncement, we'll support reading pyc-only modules just like we do today. This is in PEP 3147. We won't support creating them though. BTW,

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Steven D'Aprano
On Sun, 14 Mar 2010 11:59:50 pm Jon Ribbens wrote: Sorry if I missed it, but why on earth is the bytecode directory __pycache__ and not .pycache? (Or indeed anything else that starts with a '.') Surely this is a classic ideal use case for a hidden directory? I disagree with your assumption

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Antoine Pitrou
Le Sun, 14 Mar 2010 12:59:50 +, Jon Ribbens jon+python-...@unequivocal.co.uk a écrit : Sorry if I missed it, but why on earth is the bytecode directory __pycache__ and not .pycache? (Or indeed anything else that starts with a '.') Surely this is a classic ideal use case for a hidden

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Paul Moore
On 13 March 2010 14:17, Nick Coghlan ncogh...@gmail.com wrote: The creation side could be made a little more explicit in the PEP. We could also do something via the compileall module. (Pause while Nick goes and reads the source code for compileall for the first time ever...) Hmm - methinks

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread R. David Murray
On Sun, 14 Mar 2010 12:59:50 -, Jon Ribbens jon+python-...@unequivocal.co.uk wrote: On Fri, Mar 12, 2010 at 05:56:57PM -0500, Barry Warsaw wrote: Nope, sorry I should have been clearer. *creation* is the key here. As per BDFL pronouncement, we'll support reading pyc-only modules

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Nick Coghlan
R. David Murray wrote: On Sun, 14 Mar 2010 12:59:50 -, Jon Ribbens jon+python-...@unequivocal.co.uk wrote: Sorry if I missed it, but why on earth is the bytecode directory __pycache__ and not .pycache? (Or indeed anything else that starts with a '.') Surely this is a classic ideal use

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Greg Ewing
Jon Ribbens wrote: Sorry if I missed it, but why on earth is the bytecode directory __pycache__ and not .pycache? (Or indeed anything else that starts with a '.') Surely this is a classic ideal use case for a hidden directory? Please don't try to hide it. On MacOSX, it makes it impossible to

Re: [Python-Dev] __file__ and bytecode-only

2010-03-13 Thread Nick Coghlan
Barry Warsaw wrote: *creation* is the key here. As per BDFL pronouncement, we'll support reading pyc-only modules just like we do today. This is in PEP 3147. We won't support creating them though. The creation side could be made a little more explicit in the PEP. We could also do something

Re: [Python-Dev] __file__ and bytecode-only

2010-03-12 Thread Barry Warsaw
On Mar 04, 2010, at 11:34 PM, Nick Coghlan wrote: The remaining open question to my mind is whether or not there should be a -X option to control the bytecode generation. E.g.: -Xcache_bytecode=no (don't write bytecode files at all) -B and $PYTHONDONTWRITEBYTECODE will still be supported and

Re: [Python-Dev] __file__ and bytecode-only

2010-03-12 Thread Michael Foord
On 12/03/2010 19:53, Barry Warsaw wrote: On Mar 04, 2010, at 11:34 PM, Nick Coghlan wrote: The remaining open question to my mind is whether or not there should be a -X option to control the bytecode generation. E.g.: -Xcache_bytecode=no (don't write bytecode files at all) -B and

Re: [Python-Dev] __file__ and bytecode-only

2010-03-12 Thread Michael Foord
On 12/03/2010 22:48, Michael Foord wrote: On 12/03/2010 19:53, Barry Warsaw wrote: On Mar 04, 2010, at 11:34 PM, Nick Coghlan wrote: The remaining open question to my mind is whether or not there should be a -X option to control the bytecode generation. E.g.: -Xcache_bytecode=no (don't

Re: [Python-Dev] __file__ and bytecode-only

2010-03-12 Thread Barry Warsaw
On Mar 12, 2010, at 10:48 PM, Michael Foord wrote: On 12/03/2010 19:53, Barry Warsaw wrote: I believe we've decided /not/ to support creation of bytecode-only distributions out of the box. I thought Guido said on this topic [1]: FWIW, I started at -1 and am still -1. I think the PEP is

Re: [Python-Dev] __file__

2010-03-04 Thread Nick Coghlan
Henning von Bargen wrote: If the ZIP contains only bytecode files, it is just not intended for changing any code, so I don't think this is an argument. If you have access to the source code, you still can use that instead of messing around with byte code. That doesn't apply when it is the app

Re: [Python-Dev] __file__ and bytecode-only

2010-03-04 Thread Nick Coghlan
Barry Warsaw wrote: On Mar 03, 2010, at 07:37 PM, Jim Jewett wrote: I understand the need to ship without source -- but why does that require supporting .pyc (or .pyo) -only? Couldn't vendors just replace the real .py files with empty files? Yes, I think that's a possibility. What would

Re: [Python-Dev] __file__

2010-03-04 Thread Barry Warsaw
On Mar 04, 2010, at 11:25 PM, Nick Coghlan wrote: It's a moot point anyway - Guido has pronounced that the bytecode-only support will be left alone by PEP 3147 even if the other caching changes are eventually accepted. Right. I should have stopped while I was ahead. We'll just keep what's in

Re: [Python-Dev] __file__

2010-03-03 Thread Henning von Bargen
Nick Coghlan wrote: Another option is to remove bytecode-only support from the default filesystem importer, but keep it for zipimport (since the stat call savings don't apply in the latter case). +1 Baptiste Carvello wrote: However, making a difference between zipimport and the filesystem

Re: [Python-Dev] __file__

2010-03-03 Thread Barry Warsaw
On Mar 02, 2010, at 09:06 PM, Steven D'Aprano wrote: (1) What happens if the __cache__ directory doesn't exist and the enclosing directory is unwriteable, or if it does exist, but is unreadable? I expect that the byte code files will simply not be created, and everything will continue without

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Barry Warsaw
On Mar 03, 2010, at 07:37 PM, Jim Jewett wrote: I understand the need to ship without source -- but why does that require supporting .pyc (or .pyo) -only? Couldn't vendors just replace the real .py files with empty files? Yes, I think that's a possibility. What would people think about that?

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Brett Cannon
On Wed, Mar 3, 2010 at 16:37, Jim Jewett jimjjew...@gmail.com wrote: I understand the need to ship without source -- but why does that require supporting .pyc (or .pyo) -only? Couldn't vendors just replace the real .py files with empty files? Because if someone screws up the mod time on the

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Glenn Linderman
On approximately 3/3/2010 5:49 PM, came the following characters from the keyboard of Barry Warsaw: On Mar 03, 2010, at 07:37 PM, Jim Jewett wrote: I understand the need to ship without source -- but why does that require supporting .pyc (or .pyo) -only? Couldn't vendors just replace the

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Greg Ewing
Barry Warsaw wrote: On Mar 03, 2010, at 07:37 PM, Jim Jewett wrote: Couldn't vendors just replace the real .py files with empty files? Yes, I think that's a possibility. What would people think about that? Seems like a perverse thing to have to do to me. Also a bit fragile, since you

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Greg Ewing
Glenn Linderman wrote: In this scenario, the .pyc files would still live in __pycache__ ? Complete with the foo.token.pyc naming ? It might be neater to have a separate cache directory for each bytecode version, named __cache.token__ or some such. -- Greg

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Glyph Lefkowitz
On Mar 3, 2010, at 10:22 PM, Greg Ewing wrote: Glenn Linderman wrote: In this scenario, the .pyc files would still live in __pycache__ ? Complete with the foo.token.pyc naming ? It might be neater to have a separate cache directory for each bytecode version, named __cache.token__ or

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Antoine Pitrou
Le Wed, 3 Mar 2010 17:51:04 -0800, Brett Cannon br...@python.org a écrit : On Wed, Mar 3, 2010 at 16:37, Jim Jewett jimjjew...@gmail.com wrote: I understand the need to ship without source -- but why does that require supporting .pyc (or .pyo) -only? Couldn't vendors just replace the

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Antoine Pitrou
Le Thu, 04 Mar 2010 16:22:13 +1300, Greg Ewing greg.ew...@canterbury.ac.nz a écrit : Glenn Linderman wrote: In this scenario, the .pyc files would still live in __pycache__ ? Complete with the foo.token.pyc naming ? It might be neater to have a separate cache directory for each

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Greg Ewing
Glyph Lefkowitz wrote: if we're going to have it be something.something-else, can we please make sure that .something-else is a common extension that means python bytecode cache? Maybe something like __bytecode-version__.pycache ? -- Greg ___

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Greg Ewing
Antoine Pitrou wrote: Unless the .py files arrange to raise a syntax error on compiling. I guess that prevents a total disaster, but the program is still broken and you have to hunt down the offending files and fix the timestamps -- if it's even evident what the problem is and how to fix it.

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Greg Ewing
Antoine Pitrou wrote: Actually, I find it neater to have a single cache directory. It makes for much less clutter, and simpler ignore rules. Another possibility would be to have a single top-level cache directory with a subdirectory for each version:

Re: [Python-Dev] __file__ and bytecode-only

2010-03-03 Thread Antoine Pitrou
Le Thu, 04 Mar 2010 16:39:03 +1300, Greg Ewing greg.ew...@canterbury.ac.nz a écrit : Antoine Pitrou wrote: Unless the .py files arrange to raise a syntax error on compiling. I guess that prevents a total disaster, but the program is still broken and you have to hunt down the offending

Re: [Python-Dev] __file__

2010-03-02 Thread Steven D'Aprano
On Tue, 2 Mar 2010 11:41:52 am Barry Warsaw wrote: After PEP 3147 is implemented, and the default, you'll have to byte-compile the files, then find the pycs in the __pycache__ directory, move them up a level and rename them. Then of course remove the .py files. It's not insurmountable of

Re: [Python-Dev] __file__

2010-03-02 Thread Steven D'Aprano
On Tue, 2 Mar 2010 11:59:55 am Barry Warsaw wrote: Thanks everybody for providing great input on this aspect of the PEP. I've updated the open issues section to include a list of the possible resolutions for bytecode-only imports. Unless anybody has more ideas, it might just be time to get a

Re: [Python-Dev] __file__

2010-03-02 Thread Brett Cannon
On Tue, Mar 2, 2010 at 02:06, Steven D'Aprano st...@pearwood.info wrote: On Tue, 2 Mar 2010 11:59:55 am Barry Warsaw wrote: Thanks everybody for providing great input on this aspect of the PEP. I've updated the open issues section to include a list of the possible resolutions for

Re: [Python-Dev] __file__

2010-03-01 Thread Baptiste Carvello
Antoine Pitrou a écrit : Le Sun, 28 Feb 2010 21:45:56 +0100, Baptiste Carvello a écrit : bytecode-only in a zip is used by py2exe, cx_freeze and the like, for space reasons. Disabling it would probably hurt them. Source code compresses quite well. I'm not sure it would make much of a

Re: [Python-Dev] __file__

2010-03-01 Thread Nick Coghlan
Brett Cannon wrote: So there are a total of five to six depending on the OS (actually, VMS goes up to eight!) before a search path is considered not to contain a module. The windows list is actually going to be slightly different (dir, pyd, py, pyw, py[co]). It looks for .pyd files rather than

Re: [Python-Dev] __file__

2010-03-01 Thread Antoine Pitrou
Le Mon, 01 Mar 2010 09:09:09 +0100, Baptiste Carvello baptiste...@free.fr a écrit : I did a quick check on the stdlib: a zip with .py and .pyc is about 80% bigger than one with .pyc only. If you use only the bytecode, this can be seen as waisted space. On the other hand, if you ever need to

Re: [Python-Dev] __file__

2010-03-01 Thread Ron Adam
Nick Coghlan wrote: Michael Foord wrote: Can't it look for a .py file in the source directory first (1st stat)? When it's there check for the .pyc in the cache directory (2nd stat, magic number encoded in filename), if it's not check for .pyc in the source directory (2nd stat + read for magic

Re: [Python-Dev] __file__

2010-03-01 Thread Brett Cannon
On Mon, Mar 1, 2010 at 08:30, Ron Adam r...@ronadam.com wrote: Nick Coghlan wrote: Michael Foord wrote: Can't it look for a .py file in the source directory first (1st stat)? When it's there check for the .pyc in the cache directory (2nd stat, magic number encoded in filename), if it's

Re: [Python-Dev] __file__

2010-03-01 Thread Greg Ewing
Ron Adam wrote: What if ... a bytecode-only mode is triggered by __main__ loading from a bytecode file, otherwise the .py files are needed and are checked to make sure the bytecode files are current. That would preclude having a bytecode-only library that could be used by a sourceful

Re: [Python-Dev] __file__

2010-03-01 Thread Barry Warsaw
On Feb 28, 2010, at 10:40 PM, Antoine Pitrou wrote: File extensions exist for a reason, even if you find that kooky and have strong ideas about the psychology of text editors. Having some binary files named foobar.py would certainly annoy a lot of people, including me. I completely agree.

Re: [Python-Dev] __file__

2010-03-01 Thread Barry Warsaw
On Feb 28, 2010, at 02:51 PM, Greg Ewing wrote: A solution might be to look for the presence of the cache directory, and only look for a .pyc in the source directory if there is no cache directory. Testing for the cache directory would only have to be done once per package and the result

Re: [Python-Dev] __file__

2010-02-28 Thread Floris Bruynooghe
On Sun, Feb 28, 2010 at 02:51:16PM +1300, Greg Ewing wrote: Floris Bruynooghe wrote: (But even then I'm not convinced that would double the stat calls for normal users, only for those who only ship .pyc files) It would increase the number of stat calls for normal users by 50%. You would

Re: [Python-Dev] __file__

2010-02-28 Thread Michael Foord
-- http://www.ironpythoninaction.com On 28 Feb 2010, at 12:19, Floris Bruynooghe floris.bruynoo...@gmail.com wrote: On Sun, Feb 28, 2010 at 02:51:16PM +1300, Greg Ewing wrote: Floris Bruynooghe wrote: (But even then I'm not convinced that would double the stat calls for normal users,

Re: [Python-Dev] __file__

2010-02-28 Thread Nick Coghlan
Michael Foord wrote: Can't it look for a .py file in the source directory first (1st stat)? When it's there check for the .pyc in the cache directory (2nd stat, magic number encoded in filename), if it's not check for .pyc in the source directory (2nd stat + read for magic number check). Or

Re: [Python-Dev] __file__

2010-02-28 Thread Floris Bruynooghe
On Sun, Feb 28, 2010 at 11:07:27PM +1000, Nick Coghlan wrote: Michael Foord wrote: Can't it look for a .py file in the source directory first (1st stat)? When it's there check for the .pyc in the cache directory (2nd stat, magic number encoded in filename), if it's not check for .pyc in the

Re: [Python-Dev] __file__

2010-02-28 Thread Brett Cannon
On Sun, Feb 28, 2010 at 05:07, Nick Coghlan ncogh...@gmail.com wrote: Michael Foord wrote: Can't it look for a .py file in the source directory first (1st stat)? When it's there check for the .pyc in the cache directory (2nd stat, magic number encoded in filename), if it's not check for

Re: [Python-Dev] __file__

2010-02-28 Thread Robert Collins
On Sun, 2010-02-28 at 12:21 -0800, Brett Cannon wrote: Actually it's four: name/__init__.py, name/__init__.pyc, name.py, and then name.pyc. And just so people have terminology to go with all of this, this search is what the finder does to say whether it can or cannot handle the requested

Re: [Python-Dev] __file__

2010-02-28 Thread Baptiste Carvello
Nick Coghlan a écrit : Another option is to remove bytecode-only support from the default filesystem importer, but keep it for zipimport (since the stat call savings don't apply in the latter case). bytecode-only in a zip is used by py2exe, cx_freeze and the like, for space reasons.

Re: [Python-Dev] __file__

2010-02-28 Thread Nick Coghlan
Brett Cannon wrote: Actually it's four: name/__init__.py, name/__init__.pyc, name.py, and then name.pyc. And just so people have terminology to go with all of this, this search is what the finder does to say whether it can or cannot handle the requested module. Huh, I thought we checked for

Re: [Python-Dev] __file__

2010-02-28 Thread Antoine Pitrou
Le Sun, 28 Feb 2010 21:45:56 +0100, Baptiste Carvello a écrit : bytecode-only in a zip is used by py2exe, cx_freeze and the like, for space reasons. Disabling it would probably hurt them. Source code compresses quite well. I'm not sure it would make much of a difference. AFAIR, when you create

Re: [Python-Dev] __file__

2010-02-28 Thread Greg Ewing
Glenn Linderman wrote: if the command line/runpy can do it, the importer could do it. Just a matter of desire and coding. Whether it is worth pursuing further depends on people's perceptions of kookiness vs. functional and performance considerations. Having .py files around that aren't

Re: [Python-Dev] __file__

2010-02-28 Thread Floris Bruynooghe
On Sun, Feb 28, 2010 at 09:45:56PM +0100, Baptiste Carvello wrote: However, making a difference between zipimport and the filesystem importer means the application will stop working if I unzip the library zip file, which is surprising. Unzipping the zip file can be handy when debugging a bug

Re: [Python-Dev] __file__

2010-02-28 Thread Greg Ewing
Floris Bruynooghe wrote: Can't it look for a .py file in the source directory first (1st stat)? When it's there check for the .pyc in the cache directory (2nd stat, magic number encoded in filename), if it's not check for .pyc in the source directory (2nd stat + read for magic number check).

Re: [Python-Dev] __file__

2010-02-28 Thread Robert Collins
On Mon, 2010-03-01 at 12:35 +1300, Greg Ewing wrote: Yes, although that would then incur higher stat overheads for people distributing .pyc files. There doesn't seem to be a way of pleasing everyone. This is all assuming that the extra stat calls are actually a problem. Does anyone have

Re: [Python-Dev] __file__

2010-02-28 Thread Greg Ewing
Robert Collins wrote: In the special case of probing for $name.$ext1, ...$ext2, ...$ext3, you generally hit the same pages and don't incur additional page in costs. So then looking for a .pyc alongside a .py or vice versa should be almost free, and we shouldn't be worrying about it. hot

Re: [Python-Dev] __file__

2010-02-28 Thread Brett Cannon
On Sun, Feb 28, 2010 at 16:31, Greg Ewing greg.ew...@canterbury.ac.nzwrote: Robert Collins wrote: In the special case of probing for $name.$ext1, ...$ext2, ...$ext3, you generally hit the same pages and don't incur additional page in costs. So then looking for a .pyc alongside a .py or

Re: [Python-Dev] __file__

2010-02-28 Thread Brett Cannon
On Sun, Feb 28, 2010 at 12:46, Nick Coghlan ncogh...@gmail.com wrote: Brett Cannon wrote: Actually it's four: name/__init__.py, name/__init__.pyc, name.py, and then name.pyc. And just so people have terminology to go with all of this, this search is what the finder does to say whether it

Re: [Python-Dev] __file__

2010-02-28 Thread Brett Cannon
On Sun, Feb 28, 2010 at 12:45, Baptiste Carvello baptiste...@free.frwrote: Nick Coghlan a écrit : Another option is to remove bytecode-only support from the default filesystem importer, but keep it for zipimport (since the stat call savings don't apply in the latter case). bytecode-only

Re: [Python-Dev] __file__

2010-02-28 Thread Glenn Linderman
On approximately 2/28/2010 3:22 PM, came the following characters from the keyboard of Greg Ewing: Glenn Linderman wrote: if the command line/runpy can do it, the importer could do it. Just a matter of desire and coding. Whether it is worth pursuing further depends on people's perceptions of

Re: [Python-Dev] __file__

2010-02-28 Thread Antoine Pitrou
Le Sun, 28 Feb 2010 19:32:09 -0800, Glenn Linderman v+pyt...@g.nevcal.com a écrit : If your text editor blows up because it is binary, it is a sad text editor. If you have .py mapped to a text editor, that's sort of kooky too; I have it mapped to Python. File extensions exist for a

Re: [Python-Dev] __file__

2010-02-28 Thread Greg Ewing
Glenn Linderman wrote: If your text editor blows up because it is binary, it is a sad text editor. Blow up is probably an exaggeration, but even just getting a screen full of gibberish when I think I'm opening a text file is a jarring experience. If you have .py mapped to a text editor,

Re: [Python-Dev] __file__

2010-02-28 Thread Glyph Lefkowitz
On Feb 27, 2010, at 9:38 AM, Nick Coghlan wrote: I do like the idea of pulling .pyc only imports out into a separate importer, but would go so far as to suggest keeping them as a command line option rather than as a separately distributed module. One advantage of doing this as a separately

Re: [Python-Dev] __file__

2010-02-28 Thread Bugbee, Larry
Greg Ewing greg.ew...@canterbury.ac.nz wrote: Having .py files around that aren't source text could lead to a lot of confusion, given that most platforms these days decide which application to open for a given file based solely on the filename extension. I wouldn't enjoy trying to open a

Re: [Python-Dev] __file__

2010-02-28 Thread Baptiste Carvello
Brett Cannon a écrit : However, making a difference between zipimport and the filesystem importer means the application will stop working if I unzip the library zip file, which is surprising. Unzipping the zip file can be handy when debugging a bug caused by a forgotten module.

Re: [Python-Dev] __file__

2010-02-27 Thread Nick Coghlan
Brett Cannon wrote: On Fri, Feb 26, 2010 at 20:08, Glenn Linderman v+pyt...@g.nevcal.com I'm not sure why what you did is different than what I did, -M uses runpy which is not directly equivalent to importing. It's actually execution which is different from importing. Direct execution

Re: [Python-Dev] __file__

2010-02-27 Thread Nick Coghlan
Glenn Linderman wrote: But if the technique can work from the command line, it seems the same technique could be re-used in the importer. Not really - we only get away with the fun and games at execution time because __main__ is a bit special (and always has been). Those tricks would be a lot

Re: [Python-Dev] __file__

2010-02-27 Thread Nick Coghlan
Steven D'Aprano wrote: On Sat, 27 Feb 2010 09:09:26 am Brett Cannon wrote: I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about

Re: [Python-Dev] __file__

2010-02-27 Thread Barry Warsaw
On Feb 28, 2010, at 01:38 AM, Nick Coghlan wrote: I think the use case of keep the user from fiddling casually with our application is a valid one. Doesn't the existing support for zipimport satisfy that use case already, and probably better so? Heck you can even name your zip file

Re: [Python-Dev] __file__

2010-02-27 Thread Barry Warsaw
On Feb 26, 2010, at 10:59 PM, Michael Foord wrote: There are several companies who currently ship bytecode only. (There was someone on the IronPython mailing list only last week asking if IronPython could support pyc files for this reason). For many pointy-haired-bosses 'some' protection is

Re: [Python-Dev] __file__

2010-02-27 Thread Barry Warsaw
On Feb 26, 2010, at 08:30 PM, Ron Adam wrote: It does not make sense (to me) to have byte code only modules and packages in python's lib directory. The whole purpose (as far as I know) is for modules and packages located there to be shared. And as such, the source file becomes a source of

Re: [Python-Dev] __file__

2010-02-27 Thread Barry Warsaw
On Feb 26, 2010, at 05:11 PM, Guido van Rossum wrote: Barry's PEP would fix this even if we kept supporting .pyc-only files: the lingering .pyc files will be in the __pycache__ directory which is *not* searched -- only .pyc files directly in the source directory will be found -- where the PEP

Re: [Python-Dev] __file__

2010-02-27 Thread Floris Bruynooghe
On Sat, Feb 27, 2010 at 10:56:13AM -0500, Barry Warsaw wrote: On Feb 26, 2010, at 10:59 PM, Michael Foord wrote: There are several companies who currently ship bytecode only. (There was someone on the IronPython mailing list only last week asking if IronPython could support pyc files for

Re: [Python-Dev] __file__

2010-02-27 Thread David Bolen
Steven D'Aprano st...@pearwood.info writes: Personally, I can't imagine ever wanting to ship a .pyc module without the .py, but since Python already gives people the opportunity to shoot themselves in the foot, meh, we're all adults here. Not sure I've seen it mentioned in this thread, but

Re: [Python-Dev] __file__

2010-02-27 Thread Greg Ewing
Glenn Linderman wrote: What I did was: python -m test ren test.pyc foo.py foo.py and it worked. Source files mentioned on the command line aren't required to have a .py extension. I think what's happening is that the interpreter ignores the filename altogether in that case and examines the

Re: [Python-Dev] __file__

2010-02-27 Thread Antoine Pitrou
Le Fri, 26 Feb 2010 14:29:03 -0800, Guido van Rossum a écrit : Byte-code only wasn't always supported. We added it knowing full well it had all those problems (plus, it locks in the Python version), simply because a certain class of developers won't stop asking for it. Their users are

Re: [Python-Dev] __file__

2010-02-27 Thread Michael Foord
On 28/02/2010 01:22, Antoine Pitrou wrote: Le Fri, 26 Feb 2010 14:29:03 -0800, Guido van Rossum a écrit : Byte-code only wasn't always supported. We added it knowing full well it had all those problems (plus, it locks in the Python version), simply because a certain class of developers

Re: [Python-Dev] __file__

2010-02-27 Thread Antoine Pitrou
Le Sun, 28 Feb 2010 01:25:38 +, Michael Foord fuzzy...@voidspace.org.uk a écrit : Well if we'd *never* had this feature this argument would be very strong indeed. On the other hand if we want them to switch to Python 3 - but by the way we cut one of the features you rely on, but don't

Re: [Python-Dev] __file__

2010-02-27 Thread Greg Ewing
Floris Bruynooghe wrote: (But even then I'm not convinced that would double the stat calls for normal users, only for those who only ship .pyc files) It would increase the number of stat calls for normal users by 50%. You would need to look for a .pyc in the source directory, then .py in the

Re: [Python-Dev] __file__

2010-02-27 Thread Nick Coghlan
Glenn Linderman wrote: Thanks for the explanation. Brett mentioned something like runpy vs import using different techniques. Which is OK, I guess, but if the command line/runpy can do it, the importer could do it. Just a matter of desire and coding. Whether it is worth pursuing further

Re: [Python-Dev] __file__

2010-02-26 Thread Brett Cannon
On Thu, Feb 25, 2010 at 16:13, Greg Ewing greg.ew...@canterbury.ac.nzwrote: Michael Foord wrote: I thought we agreed at the language summit that if a .pyc was in the place of the source file it *could* be imported from - making pyc only distributions possible. Ah, that's okay, then.

Re: [Python-Dev] __file__

2010-02-26 Thread Guido van Rossum
On Fri, Feb 26, 2010 at 2:09 PM, Brett Cannon br...@python.org wrote: And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Bytecode contains so much data that disassembling it gives you a very clear picture of what the original code

Re: [Python-Dev] __file__

2010-02-26 Thread Barry Warsaw
On Feb 26, 2010, at 02:09 PM, Brett Cannon wrote: But a benefit of no longer supporting bytecode-only modules by default is it cuts back on possible stat calls which slows down Python's startup time (a complaint I hear a lot). Performance issues become even more acute if you try to come up with

Re: [Python-Dev] __file__

2010-02-26 Thread Brett Cannon
On Fri, Feb 26, 2010 at 14:29, Guido van Rossum gu...@python.org wrote: On Fri, Feb 26, 2010 at 2:09 PM, Brett Cannon br...@python.org wrote: And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Bytecode contains so much data

Re: [Python-Dev] __file__

2010-02-26 Thread Michael Foord
On 26/02/2010 22:09, Brett Cannon wrote: On Thu, Feb 25, 2010 at 16:13, Greg Ewing greg.ew...@canterbury.ac.nz mailto:greg.ew...@canterbury.ac.nz wrote: Michael Foord wrote: I thought we agreed at the language summit that if a .pyc was in the place of the source file it

Re: [Python-Dev] __file__

2010-02-26 Thread Glenn Linderman
On approximately 2/26/2010 2:55 PM, came the following characters from the keyboard of Brett Cannon: Maybe Greg's and my response to the mention of dropping this feature is too strong -- after all we're both dinosaurs. And maybe the developers who want the feature can write their

Re: [Python-Dev] __file__

2010-02-26 Thread Ian Bicking
The one issue I thought would be resolved by not easily allowing .pyc-only distributions is the case when you rename a file (say module.py to newmodule.py) and there is a module.pyc laying around, and you don't get the ImportError you would expect from import module -- and to make it worse

Re: [Python-Dev] __file__

2010-02-26 Thread Brett Cannon
On Fri, Feb 26, 2010 at 16:58, Ian Bicking i...@colorstudy.com wrote: The one issue I thought would be resolved by not easily allowing .pyc-only distributions is the case when you rename a file (say module.py to newmodule.py) and there is a module.pyc laying around, and you don't get the

Re: [Python-Dev] __file__

2010-02-26 Thread Guido van Rossum
On Fri, Feb 26, 2010 at 4:58 PM, Ian Bicking i...@colorstudy.com wrote: The one issue I thought would be resolved by not easily allowing .pyc-only distributions is the case when you rename a file (say module.py to newmodule.py) and there is a module.pyc laying around, and you don't get the

Re: [Python-Dev] __file__

2010-02-26 Thread Brett Cannon
On Fri, Feb 26, 2010 at 15:35, Glenn Linderman v+pyt...@g.nevcal.comv%2bpyt...@g.nevcal.com wrote: On approximately 2/26/2010 2:55 PM, came the following characters from the keyboard of Brett Cannon: Maybe Greg's and my response to the mention of dropping this feature is too strong

Re: [Python-Dev] __file__

2010-02-26 Thread Doug Hellmann
On Feb 26, 2010, at 5:59 PM, Michael Foord wrote: On 26/02/2010 22:09, Brett Cannon wrote: On Thu, Feb 25, 2010 at 16:13, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Michael Foord wrote: I thought we agreed at the language summit that if a .pyc was in the place of the source file

Re: [Python-Dev] __file__

2010-02-26 Thread Steven D'Aprano
On Sat, 27 Feb 2010 09:09:26 am Brett Cannon wrote: I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about their code not being

Re: [Python-Dev] __file__

2010-02-26 Thread Brett Cannon
On Fri, Feb 26, 2010 at 17:20, Doug Hellmann doug.hellm...@gmail.comwrote: On Feb 26, 2010, at 5:59 PM, Michael Foord wrote: On 26/02/2010 22:09, Brett Cannon wrote: On Thu, Feb 25, 2010 at 16:13, Greg Ewing greg.ew...@canterbury.ac.nzwrote: Michael Foord wrote: I thought we agreed

Re: [Python-Dev] __file__

2010-02-26 Thread Ron Adam
Barry Warsaw wrote: On Feb 26, 2010, at 02:09 PM, Brett Cannon wrote: But a benefit of no longer supporting bytecode-only modules by default is it cuts back on possible stat calls which slows down Python's startup time (a complaint I hear a lot). Performance issues become even more acute if

Re: [Python-Dev] __file__

2010-02-26 Thread Glenn Linderman
On approximately 2/26/2010 5:13 PM, came the following characters from the keyboard of Brett Cannon: On Fri, Feb 26, 2010 at 15:35, Glenn Linderman v+pyt...@g.nevcal.com mailto:v%2bpyt...@g.nevcal.com wrote: On approximately 2/26/2010 2:55 PM, came the following characters from the

Re: [Python-Dev] __file__

2010-02-26 Thread Brett Cannon
On Fri, Feb 26, 2010 at 20:08, Glenn Linderman v+pyt...@g.nevcal.comv%2bpyt...@g.nevcal.com wrote: On approximately 2/26/2010 5:13 PM, came the following characters from the keyboard of Brett Cannon: On Fri, Feb 26, 2010 at 15:35, Glenn Linderman

Re: [Python-Dev] __file__

2010-02-26 Thread Guido van Rossum
On Fri, Feb 26, 2010 at 5:13 PM, Brett Cannon br...@python.org wrote: On Fri, Feb 26, 2010 at 15:35, Glenn Linderman v+pyt...@g.nevcal.com When a .pyc is renamed to .py, Python (3.1 at least) recognizes and uses it... I assume by design, rather than accident, but I don't know the history.

  1   2   >