Re: [Python-Dev] contributor to committer
Hello, +1 Thanks all, for your warm welcome. The usual caveats apply though: - don't get carried away with the privileges - even core devs still put patches on the tracker sometimes - if in doubt, ask for advice on python-dev (or IRC) - make sure to subscribe to python-checkins Usually I tend to be cautious. -- Florent ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another version of Python
2010/2/24 skip s...@pobox.com: Some of you have probably already seen this, but in case you haven't: http://www.staringispolite.com/likepython/ :-) I'm reminded of LOLPython: http://bit.ly/271rt. -- Cheers, Simon B. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pickling named tuples on IronPython
Hello Raymond, Named tuples have compatibility code to enable them to work on IronPython without frame support, but unfortunately this doesn't allow pickling / unpickling of named tuples. One fix is to manually set __module__ on the named tuples once created, but I wonder if it would be possible to change the API to better support this - perhaps a default __module__ or providing an optional argument to specify it at creation time? Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another version of Python
  http://www.staringispolite.com/likepython/ Simon I'm reminded of LOLPython: http://bit.ly/271rt. You know, I'm thinking while both are obviously tongue-in-cheek we should probably include them on the /dev/implementations page of python.org, probably in a separate section at the end of the page. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another version of Python
On 26/02/2010 17:26, s...@pobox.com wrote: Â Â http://www.staringispolite.com/likepython/ Simon I'm reminded of LOLPython:http://bit.ly/271rt. You know, I'm thinking while both are obviously tongue-in-cheek we should probably include them on the /dev/implementations page of python.org, probably in a separate section at the end of the page. They're certainly fun - but they seem to be fly-by-night projects (i.e. unlikely to be maintained in the long run). The risk is that we end up with even more outdated links / material on the website. Anyway, if the consensus is that it would be good to link to them then I will update the page (which could already do with some updating by the looks of it). All the best, Michael Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another version of Python
On Fri, Feb 26, 2010 at 12:53 PM, Michael Foord fuzzy...@voidspace.org.uk wrote: On 26/02/2010 17:26, s...@pobox.com wrote: Â Â http://www.staringispolite.com/likepython/ Simon I'm reminded of LOLPython:http://bit.ly/271rt. You know, I'm thinking while both are obviously tongue-in-cheek we should probably include them on the /dev/implementations page of python.org, probably in a separate section at the end of the page. They're certainly fun - but they seem to be fly-by-night projects (i.e. unlikely to be maintained in the long run). The risk is that we end up with even more outdated links / material on the website. Anyway, if the consensus is that it would be good to link to them then I will update the page (which could already do with some updating by the looks of it). Sure, we can link to them, and then add titles to all job postings like Ninja and Wizard so we can really seem awesome. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Fri, Feb 19, 2010 at 19:37, s...@pobox.com wrote: Lennart I would like if we could look into making a timezone module Lennart that works on Python 2.5 to 3.2 that uses system data... 2.5, 2.6 and 3.1 are completely off the radar screen at this point. The best you could hope for is that someone backports whatever is created for 2.7 or 3.2 and distributes it outside the normal distribution channel (say, as a patch on PyPI). My argument was that we should create a module distributed on PyPI, and once that's stable, move it into stdlib. The suggestions in this thread of moving things into stdlib has included a lot of new features, and are as such not stable. I'm worrying that adding such a thing to stdlib will do so in an unfinished state, and we'll just en up with yet another state of semi-brokenness. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Fri, Feb 26, 2010 at 3:59 PM, Lennart Regebro rege...@gmail.com wrote: I'm worrying that adding such a thing to stdlib will do so in an unfinished state, and we'll just en up with yet another state of semi-brokenness. I valid worry, and compelling. As I've alluded to before, leaving it out and allowing applications to just use pytz (or whatever else) is entirely reasonable. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] The fate of Distutils in Python 2.7
Hello, This is a follow-up of the Pycon summit + sprints on packaging. This is what we have planned to do: 1. refactor distutils in a new standalone version called distutils2 [this is done already and we are actively working in the code] 2. completely revert distutils in Lib/ and Doc/ so the code + doc is the same than the current 2.6.x branch 3. leave the new sysconfig module, that is used by the Makefile and the site module The rest of the work will happen in distutils2 and we will try to release a version asap for Python 2.x and 3.x (2.4 to 3.2), and the goal is to put it back in the stdlib in Python 3.3 Distutils in Python will be feature-frozen and I will only do bug fixes there. All feature requests will be redirected to Distutils2. I think the easiest way to manage this for me and for the feedback of the community is to add in bugs.python.org a Distutils2 component, so I can start to reorganize the issues in there and reassign new issues to Distutils2 when it applies. Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3188: Implementation Questions
Meador Inge wrote: 3. Using Decimal keeps the desired precision, Well, sort of, but then you end up doing arithmetic in decimal instead of binary, which could give different results. Maybe the solution is to give ctypes long double objects the ability to do arithmetic? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Thu, Feb 25, 2010 at 16:13, Greg Ewing greg.ew...@canterbury.ac.nzwrote: Michael Foord wrote: I thought we agreed at the language summit that if a .pyc was in the place of the source file it *could* be imported from - making pyc only distributions possible. Ah, that's okay, then. Sorry about the panic! Michael is right about what as discussed at the language summit, but Barry means what he says; if you look at the PEP as it currently stands it does not support bytecode-only modules. Barry and I discussed how to implement the PEP at PyCon after the summit and supporting bytecode-only modules quickly began to muck with the semantics and made it harder to explain (i.e. what to set __file__ vs. __compiled__ based on what is or is not available and how to properly define get_paths for loaders). But a benefit of no longer supporting bytecode-only modules by default is it cuts back on possible stat calls which slows down Python's startup time (a complaint I hear a lot). Performance issues become even more acute if you try to come up with even a remotely proper way to have backwards-compatible support in importlib for its ABCs w/o forcing caching on all implementors of the ABCs. As for having a dependency on a loader, I don't see how that is obscure; it's just a dependency your package has that you handle at install-time. And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Bytecode contains so much data that disassembling it gives you a very clear picture of what the original code was like. I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about their code not being easy to examine by someone else. The only perk I can see is space-saving, but that's dangerous as that ties you to a specific VM with a specific magic number (let alone that it leads to people tying themselves to CPython and ignoring the other VMs that simply do not support bytecode). -Brett -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The fate of Distutils in Python 2.7
On Fri, Feb 26, 2010 at 13:44, Tarek Ziadé ziade.ta...@gmail.com wrote: Hello, This is a follow-up of the Pycon summit + sprints on packaging. This is what we have planned to do: 1. refactor distutils in a new standalone version called distutils2 [this is done already and we are actively working in the code] 2. completely revert distutils in Lib/ and Doc/ so the code + doc is the same than the current 2.6.x branch 3. leave the new sysconfig module, that is used by the Makefile and the site module The rest of the work will happen in distutils2 and we will try to release a version asap for Python 2.x and 3.x (2.4 to 3.2), and the goal is to put it back in the stdlib in Python 3.3 Distutils in Python will be feature-frozen and I will only do bug fixes there. All feature requests will be redirected to Distutils2. I think the easiest way to manage this for me and for the feedback of the community is to add in bugs.python.org a Distutils2 component, so I can start to reorganize the issues in there and reassign new issues to Distutils2 when it applies. I assume you want the Distutils2 component to auto-assign to you like Distutils currently does? If so I can add the component for you if people don't object to the new component. -Brett Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The fate of Distutils in Python 2.7
On Fri, Feb 26, 2010 at 11:13 PM, Brett Cannon br...@python.org wrote: [..] I assume you want the Distutils2 component to auto-assign to you like Distutils currently does? If so I can add the component for you if people don't object to the new component. Sounds good -- Thanks ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Fri, Feb 26, 2010 at 2:09 PM, Brett Cannon br...@python.org wrote: And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Bytecode contains so much data that disassembling it gives you a very clear picture of what the original code was like. I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about their code not being easy to examine by someone else. Byte-code only wasn't always supported. We added it knowing full well it had all those problems (plus, it locks in the Python version), simply because a certain class of developers won't stop asking for it. Their users are apparently too dumb to decode bytecode but smart enough to read source code, even if they don't understand it, and this knowledge could hurt them. Presumably users smart enough to decode bytecode will know enough not to hurt themselves. Maybe Greg's and my response to the mention of dropping this feature is too strong -- after all we're both dinosaurs. And maybe the developers who want the feature can write their own loader. But given that this feature takes an entirely different path through import.c anyway, I still don't see how dropping it is necessary in order to implement the PEP. If you have separate motivation to drop the feature, you should deprecate it properly. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Feb 26, 2010, at 02:09 PM, Brett Cannon wrote: But a benefit of no longer supporting bytecode-only modules by default is it cuts back on possible stat calls which slows down Python's startup time (a complaint I hear a lot). Performance issues become even more acute if you try to come up with even a remotely proper way to have backwards-compatible support in importlib for its ABCs w/o forcing caching on all implementors of the ABCs. And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Brett really hits the nail on the head, and yes I'm sorry for not being clear about what we discussed this at Pycon meant. The we being Brett and I of course (and Chris Withers IIRC). Bytecode-only deployments are a bit of a sham, and definitely a minority use case, so why should all of Python pay for the extra stat calls to support this by default? How many people would actually be hurt if this wasn't available out of the box, especially since you can still support it if you really want it and can't convince your manager that it provides essentially zero useful obfuscation of your code? I say this having been down that road myself with a previous employer. Management was pretty adamant about wanting this until I explained how easy it was to defeat and convinced them that the engineering resources to do it were better spent elsewhere. Having said that, I'd be all for including a reference implementation of a bytecode-only loader in the PEP for demonstration purposes. Greg, would you like to contribute that? -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Fri, Feb 26, 2010 at 14:29, Guido van Rossum gu...@python.org wrote: On Fri, Feb 26, 2010 at 2:09 PM, Brett Cannon br...@python.org wrote: And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Bytecode contains so much data that disassembling it gives you a very clear picture of what the original code was like. I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about their code not being easy to examine by someone else. Byte-code only wasn't always supported. We added it knowing full well it had all those problems (plus, it locks in the Python version), simply because a certain class of developers won't stop asking for it. Their users are apparently too dumb to decode bytecode but smart enough to read source code, even if they don't understand it, and this knowledge could hurt them. Presumably users smart enough to decode bytecode will know enough not to hurt themselves. Maybe it should be made optional much like the talk of frozen modules eventually becoming an optional thing. Maybe Greg's and my response to the mention of dropping this feature is too strong -- after all we're both dinosaurs. And maybe the developers who want the feature can write their own loader. We could also provide if necessary. But given that this feature takes an entirely different path through import.c anyway, I still don't see how dropping it is necessary in order to implement the PEP. It's not necessary at all. I think what Barry was going for was simply cleaning up semantics once instead of having to drag it out. If you have separate motivation to drop the feature, you should deprecate it properly. Fine by me. It would be easy enough to raise ImportWarning in the bytecode-only case if Barry decides to push for this. Here is a question for Barry to think about if he decides to move forward with all of this: would mixed support for both bytecode-only and source/bytecode be required for the same directory, or could it be one or the other but not both? Differing semantics based on what is found in the directory would make the path hook more expensive (which is a one-time cost per directory), but it would cut stat calls in the finder in half (which is a cost made per import). -Brett -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On 26/02/2010 22:09, Brett Cannon wrote: On Thu, Feb 25, 2010 at 16:13, Greg Ewing greg.ew...@canterbury.ac.nz mailto:greg.ew...@canterbury.ac.nz wrote: Michael Foord wrote: I thought we agreed at the language summit that if a .pyc was in the place of the source file it *could* be imported from - making pyc only distributions possible. Ah, that's okay, then. Sorry about the panic! Michael is right about what as discussed at the language summit, but Barry means what he says; if you look at the PEP as it currently stands it does not support bytecode-only modules. Barry and I discussed how to implement the PEP at PyCon after the summit and supporting bytecode-only modules quickly began to muck with the semantics and made it harder to explain (i.e. what to set __file__ vs. __compiled__ based on what is or is not available and how to properly define get_paths for loaders). But a benefit of no longer supporting bytecode-only modules by default is it cuts back on possible stat calls which slows down Python's startup time (a complaint I hear a lot). Performance issues become even more acute if you try to come up with even a remotely proper way to have backwards-compatible support in importlib for its ABCs w/o forcing caching on all implementors of the ABCs. As for having a dependency on a loader, I don't see how that is obscure; it's just a dependency your package has that you handle at install-time. And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Bytecode contains so much data that disassembling it gives you a very clear picture of what the original code was like. Well, understanding bytecode is *still* requires a higher level of understanding than the *majority* of Python programmers. Added to which there are no widely available tools that *I'm* aware of for decompiling recent versions of Python (decompyle worked up to Python 2.4 but then went closed source as a commercial service [1]. The situation is analagous to .NET assemblies by the way (which *can* be trivially decompiled by several widely available tools). Having a non-source distribution prevents your users from changing things and then calling you for support without them having to go to a lot more effort than it is worth. There are several companies who currently ship bytecode only. (There was someone on the IronPython mailing list only last week asking if IronPython could support pyc files for this reason). For many pointy-haired-bosses 'some' protection is enough and having Python not support this (out of the box) would be a black mark against Python for them. I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about their code not being easy to examine by someone else. For many use-cases some protection is enough. After all *any* DRM or source-code obfuscation is breakable in the medium / long term - so just enough to discourage the casual looker is probably sufficient. The fact that bytecode only distributions exist speaks to that. Whether you believe that allowing companies who ship bytecode is a disservice to them or not is fundamentally irrelevant. If they believe it is a service to them then it is... :-) As you can tell, I would be disappointed to see bytecode only distributions be removed from the out-of-the-box functionality. All the best, Michael The only perk I can see is space-saving, but that's dangerous as that ties you to a specific VM with a specific magic number (let alone that it leads to people tying themselves to CPython and ignoring the other VMs that simply do not support bytecode). [1] http://www.crazy-compilers.com/decompyle/ -Brett -- Greg ___ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing
[Python-Dev] Python 2.6.5 rc 1
Hello everybody! I hope you all had as great a time at Pycon 2010 as I did. No time to begin recovering though, we're on to Python 2.6.5 rc 1, which I would like to release on Monday. We have one showstopper still open, and I'll try to respond to that asap. http://bugs.python.org/issue7250 If there is anything else that absolutely should go into 2.6.5, now's the time to let me know. If there are no patches ready to be reviewed and landed though, you're probably running out of time. I will be very conservative about landing patches after rc1. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On approximately 2/26/2010 2:55 PM, came the following characters from the keyboard of Brett Cannon: Maybe Greg's and my response to the mention of dropping this feature is too strong -- after all we're both dinosaurs. And maybe the developers who want the feature can write their own loader. We could also provide if necessary. So if the implementation stores .pyc by default in a version-specific place, then it seems there are only two things needed to make a python byte-code only distribution... 1) rename all the .pyc to .py 2) packaging When a .pyc is renamed to .py, Python (3.1 at least) recognizes and uses it... I assume by design, rather than accident, but I don't know the history. I didn't experiment to discover what __file__ and __cached__ get set to in this case (especially since I don't have a version with the latter :) ). I speculate that packaging a distribution in this manner would be slightly different that how it is currently done, but I also suspect that it would avoid the same half of the stat calls, to aid performance. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
The one issue I thought would be resolved by not easily allowing .pyc-only distributions is the case when you rename a file (say module.py to newmodule.py) and there is a module.pyc laying around, and you don't get the ImportError you would expect from import module -- and to make it worse everything basically works, except there's two versions of the module that slowly become different. This regularly causes problems for me, and those problems would get more common and obscure if the pyc files were stashed away in a more invisible location. I can't even tell what the current proposal is; maybe this is resolved? If distributing bytecode required renaming pyc files to .py as Glenn suggested that would resolve the problem quite nicely from my perspective. (Frankly I find the whole use case for distributing bytecodes a bit specious, but whatever.) -- Ian Bicking | http://blog.ianbicking.org | http://twitter.com/ianbicking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Fri, Feb 26, 2010 at 16:58, Ian Bicking i...@colorstudy.com wrote: The one issue I thought would be resolved by not easily allowing .pyc-only distributions is the case when you rename a file (say module.py to newmodule.py) and there is a module.pyc laying around, and you don't get the ImportError you would expect from import module -- and to make it worse everything basically works, except there's two versions of the module that slowly become different. Yes, that problem would go away if bytecode-only modules were no longer supported. This regularly causes problems for me, and those problems would get more common and obscure if the pyc files were stashed away in a more invisible location. That has never been an issue with this proposal. The bytecode pulled from the __pycache__ directory only occurs if source exists. What we have been discussing is whether bytecode-only files in the directory of a package or something exists. -Brett I can't even tell what the current proposal is; maybe this is resolved? If distributing bytecode required renaming pyc files to .py as Glenn suggested that would resolve the problem quite nicely from my perspective. (Frankly I find the whole use case for distributing bytecodes a bit specious, but whatever.) -- Ian Bicking | http://blog.ianbicking.org | http://twitter.com/ianbicking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Fri, Feb 26, 2010 at 4:58 PM, Ian Bicking i...@colorstudy.com wrote: The one issue I thought would be resolved by not easily allowing .pyc-only distributions is the case when you rename a file (say module.py to newmodule.py) and there is a module.pyc laying around, and you don't get the ImportError you would expect from import module -- and to make it worse everything basically works, except there's two versions of the module that slowly become different. This regularly causes problems for me, and those problems would get more common and obscure if the pyc files were stashed away in a more invisible location. I can't even tell what the current proposal is; maybe this is resolved? If distributing bytecode required renaming pyc files to .py as Glenn suggested that would resolve the problem quite nicely from my perspective. (Frankly I find the whole use case for distributing bytecodes a bit specious, but whatever.) Barry's PEP would fix this even if we kept supporting .pyc-only files: the lingering .pyc files will be in the __pycache__ directory which is *not* searched -- only .pyc files directly in the source directory will be found -- where the PEP will never place them, at least not by default. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Fri, Feb 26, 2010 at 15:35, Glenn Linderman v+pyt...@g.nevcal.comv%2bpyt...@g.nevcal.com wrote: On approximately 2/26/2010 2:55 PM, came the following characters from the keyboard of Brett Cannon: Maybe Greg's and my response to the mention of dropping this feature is too strong -- after all we're both dinosaurs. And maybe the developers who want the feature can write their own loader. We could also provide if necessary. So if the implementation stores .pyc by default in a version-specific place, then it seems there are only two things needed to make a python byte-code only distribution... 1) rename all the .pyc to .py 2) packaging When a .pyc is renamed to .py, Python (3.1 at least) recognizes and uses it... I assume by design, rather than accident, but I don't know the history. This does not work for me (nor should it): touch temp.py python3 -c import temp rm temp.py mv temp.pyc temp.py python3 -c import temp Traceback (most recent call last): File string, line 1, in module File temp.py, line 2 SyntaxError: Non-UTF-8 code starting with '\x95' in file temp.py on line 2, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details -Brett I didn't experiment to discover what __file__ and __cached__ get set to in this case (especially since I don't have a version with the latter :) ). I speculate that packaging a distribution in this manner would be slightly different that how it is currently done, but I also suspect that it would avoid the same half of the stat calls, to aid performance. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Feb 26, 2010, at 5:59 PM, Michael Foord wrote: On 26/02/2010 22:09, Brett Cannon wrote: On Thu, Feb 25, 2010 at 16:13, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Michael Foord wrote: I thought we agreed at the language summit that if a .pyc was in the place of the source file it *could* be imported from - making pyc only distributions possible. Ah, that's okay, then. Sorry about the panic! Michael is right about what as discussed at the language summit, but Barry means what he says; if you look at the PEP as it currently stands it does not support bytecode-only modules. Barry and I discussed how to implement the PEP at PyCon after the summit and supporting bytecode-only modules quickly began to muck with the semantics and made it harder to explain (i.e. what to set __file__ vs. __compiled__ based on what is or is not available and how to properly define get_paths for loaders). But a benefit of no longer supporting bytecode-only modules by default is it cuts back on possible stat calls which slows down Python's startup time (a complaint I hear a lot). Performance issues become even more acute if you try to come up with even a remotely proper way to have backwards-compatible support in importlib for its ABCs w/o forcing caching on all implementors of the ABCs. As for having a dependency on a loader, I don't see how that is obscure; it's just a dependency your package has that you handle at install-time. And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Bytecode contains so much data that disassembling it gives you a very clear picture of what the original code was like. Well, understanding bytecode is *still* requires a higher level of understanding than the *majority* of Python programmers. Added to which there are no widely available tools that *I'm* aware of for decompiling recent versions of Python (decompyle worked up to Python 2.4 but then went closed source as a commercial service [1]. The situation is analagous to .NET assemblies by the way (which *can* be trivially decompiled by several widely available tools). Having a non-source distribution prevents your users from changing things and then calling you for support without them having to go to a lot more effort than it is worth. There are several companies who currently ship bytecode only. (There was someone on the IronPython mailing list only last week asking if IronPython could support pyc files for this reason). For many pointy- haired-bosses 'some' protection is enough and having Python not support this (out of the box) would be a black mark against Python for them. We ship bytecode only, basically for the reason Michael states here (keeping support costs under control from ambitious users). I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about their code not being easy to examine by someone else. For many use-cases some protection is enough. After all *any* DRM or source-code obfuscation is breakable in the medium / long term - so just enough to discourage the casual looker is probably sufficient. The fact that bytecode only distributions exist speaks to that. Right. We're more concerned with not having users muck with stuff than with keeping the implementation a secret, although having a bit of obfuscation doesn't hurt. Whether you believe that allowing companies who ship bytecode is a disservice to them or not is fundamentally irrelevant. If they believe it is a service to them then it is... :-) As you can tell, I would be disappointed to see bytecode only distributions be removed from the out-of-the-box functionality. +1 Doug ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Sat, 27 Feb 2010 09:09:26 am Brett Cannon wrote: I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about their code not being easy to examine by someone else. You say that as if it were a bad thing. *wink* Personally, I can't imagine ever wanting to ship a .pyc module without the .py, but since Python already gives people the opportunity to shoot themselves in the foot, meh, we're all adults here. I do recall a poster on comp.lang.python pulling his hair out over a customer who was too big to fire, but who had the obnoxious habit of making random so-called fixes to the poster's .py files, so perhaps byte-code only distribution isn't all bad. But I don't care much either way. -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Fri, Feb 26, 2010 at 17:20, Doug Hellmann doug.hellm...@gmail.comwrote: On Feb 26, 2010, at 5:59 PM, Michael Foord wrote: On 26/02/2010 22:09, Brett Cannon wrote: On Thu, Feb 25, 2010 at 16:13, Greg Ewing greg.ew...@canterbury.ac.nzwrote: Michael Foord wrote: I thought we agreed at the language summit that if a .pyc was in the place of the source file it *could* be imported from - making pyc only distributions possible. Ah, that's okay, then. Sorry about the panic! Michael is right about what as discussed at the language summit, but Barry means what he says; if you look at the PEP as it currently stands it does not support bytecode-only modules. Barry and I discussed how to implement the PEP at PyCon after the summit and supporting bytecode-only modules quickly began to muck with the semantics and made it harder to explain (i.e. what to set __file__ vs. __compiled__ based on what is or is not available and how to properly define get_paths for loaders). But a benefit of no longer supporting bytecode-only modules by default is it cuts back on possible stat calls which slows down Python's startup time (a complaint I hear a lot). Performance issues become even more acute if you try to come up with even a remotely proper way to have backwards-compatible support in importlib for its ABCs w/o forcing caching on all implementors of the ABCs. As for having a dependency on a loader, I don't see how that is obscure; it's just a dependency your package has that you handle at install-time. And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Bytecode contains so much data that disassembling it gives you a very clear picture of what the original code was like. Well, understanding bytecode is *still* requires a higher level of understanding than the *majority* of Python programmers. Added to which there are no widely available tools that *I'm* aware of for decompiling recent versions of Python (decompyle worked up to Python 2.4 but then went closed source as a commercial service [1]. The situation is analagous to .NET assemblies by the way (which *can* be trivially decompiled by several widely available tools). Having a non-source distribution prevents your users from changing things and then calling you for support without them having to go to a lot more effort than it is worth. There are several companies who currently ship bytecode only. (There was someone on the IronPython mailing list only last week asking if IronPython could support pyc files for this reason). For many pointy-haired-bosses 'some' protection is enough and having Python not support this (out of the box) would be a black mark against Python for them. We ship bytecode only, basically for the reason Michael states here (keeping support costs under control from ambitious users). I think it's almost a dis-service to support bytecode-only files as it leads people who are misinformed or simply don't take the time to understand what is contained in a .pyc file into a false sense of security about their code not being easy to examine by someone else. For many use-cases some protection is enough. After all *any* DRM or source-code obfuscation is breakable in the medium / long term - so just enough to discourage the casual looker is probably sufficient. The fact that bytecode only distributions exist speaks to that. Right. We're more concerned with not having users muck with stuff than with keeping the implementation a secret, although having a bit of obfuscation doesn't hurt. Whether you believe that allowing companies who ship bytecode is a disservice to them or not is fundamentally irrelevant. If they believe it is a service to them then it is... :-) As you can tell, I would be disappointed to see bytecode only distributions be removed from the out-of-the-box functionality. +1 So what is the burden of including a single source file that added the support to load from bytecode-only modules? I am not saying you shouldn't be able to have this functionality, just that I personally don't want to pay for the overhead (both performance-wise and development-wise) by default just because you and some other people want this functionality for some clients. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another version of Python
On Sat, 27 Feb 2010 04:53:36 am Michael Foord wrote: On 26/02/2010 17:26, s...@pobox.com wrote: Â Â http://www.staringispolite.com/likepython/ Simon I'm reminded of LOLPython:http://bit.ly/271rt. You know, I'm thinking while both are obviously tongue-in-cheek we should probably include them on the /dev/implementations page of python.org, probably in a separate section at the end of the page. They're certainly fun - but they seem to be fly-by-night projects (i.e. unlikely to be maintained in the long run). The risk is that we end up with even more outdated links / material on the website. Anyway, if the consensus is that it would be good to link to them then I will update the page (which could already do with some updating by the looks of it). For what it's worth, I have compiled a list of between 14 and 27 implementations of Python, depending on how conservative you are at defining implementation. I then went to the wiki and discovered my list was nowhere near complete. Obviously this information is extensive and rapidly changing, so I think it would be better to have the current implementation page be fairly minimal but link to the wiki for more details: http://wiki.python.org/moin/implementation http://www.python.org/dev/implementations/ -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
Barry Warsaw wrote: On Feb 26, 2010, at 02:09 PM, Brett Cannon wrote: But a benefit of no longer supporting bytecode-only modules by default is it cuts back on possible stat calls which slows down Python's startup time (a complaint I hear a lot). Performance issues become even more acute if you try to come up with even a remotely proper way to have backwards-compatible support in importlib for its ABCs w/o forcing caching on all implementors of the ABCs. And personally, I don't see what bytecode-only modules buy you. The obfuscation argument is bunk as we all know. Brett really hits the nail on the head, and yes I'm sorry for not being clear about what we discussed this at Pycon meant. The we being Brett and I of course (and Chris Withers IIRC). Bytecode-only deployments are a bit of a sham, and definitely a minority use case, so why should all of Python pay for the extra stat calls to support this by default? How many people would actually be hurt if this wasn't available out of the box, especially since you can still support it if you really want it and can't convince your manager that it provides essentially zero useful obfuscation of your code? I say this having been down that road myself with a previous employer. Management was pretty adamant about wanting this until I explained how easy it was to defeat and convinced them that the engineering resources to do it were better spent elsewhere. Having said that, I'd be all for including a reference implementation of a bytecode-only loader in the PEP for demonstration purposes. Greg, would you like to contribute that? -Barry Micheal Foords view point on this strikes me as the most realistic. Some people do find it to be a value for their particular needs and circumstance. Michael Foord Wrote: For many use-cases some protection is enough. After all *any* DRM or source-code obfuscation is breakable in the medium / long term - so just enough to discourage the casual looker is probably sufficient. The fact that bytecode only distributions exist speaks to that. Whether you believe that allowing companies who ship bytecode is a disservice to them or not is fundamentally irrelevant. If they believe it is a service to them then it is... :-) To possibly qualify it a bit more: It does not make sense (to me) to have byte code only modules and packages in python's lib directory. The whole purpose (as far as I know) is for modules and packages located there to be shared. And as such, the source file becomes a source of documentation. Not supporting bytecode only python modules and packages in pythons lib directory may be good. For python programs located and installed elsewhere I think Michaels view point is applicable. For some files that are not meant to be shared, some form of discouragement can be a feature. Ron Adam ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On approximately 2/26/2010 5:13 PM, came the following characters from the keyboard of Brett Cannon: On Fri, Feb 26, 2010 at 15:35, Glenn Linderman v+pyt...@g.nevcal.com mailto:v%2bpyt...@g.nevcal.com wrote: On approximately 2/26/2010 2:55 PM, came the following characters from the keyboard of Brett Cannon: Maybe Greg's and my response to the mention of dropping this feature is too strong -- after all we're both dinosaurs. And maybe the developers who want the feature can write their own loader. We could also provide if necessary. So if the implementation stores .pyc by default in a version-specific place, then it seems there are only two things needed to make a python byte-code only distribution... 1) rename all the .pyc to .py 2) packaging When a .pyc is renamed to .py, Python (3.1 at least) recognizes and uses it... I assume by design, rather than accident, but I don't know the history. This does not work for me (nor should it): touch temp.py python3 -c import temp rm temp.py mv temp.pyc temp.py python3 -c import temp Traceback (most recent call last): File string, line 1, in module File temp.py, line 2 SyntaxError: Non-UTF-8 code starting with '\x95' in file temp.py on line 2, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details -Brett I'll admit to not doing exhaustive testing, but I'll not admit to not doing any testing... because it was sort of a wild idea. Someone else called it kooky, which is fair. What I did was: python -m test ren test.pyc foo.py foo.py and it worked. Then I posted, knowing that I'd also tested, the other day, several .py into a .zip named .py, and once that worked, then I changed to putting all .pyc into the .zip named .py and that worked too... including imports of the several modules from the __main__.pyc. Of course, all those were still named .pyc inside the .zip named .py. So I'm not sure what the difference is... .pyc as .py works from the command line, but not from import? Some specialty because of using -c ? I'd guess the technique could be made to work, probably not require extensive changes, if Python developers wanted to make it work. I think it could be efficient and that same someone that called it kooky admitted it would solve their use case, at least. I'm not sure why what you did is different than what I did, nor why you state without justification that it shouldn't work... I might be able to figure out the former if I spend enough time with the documentation, if it is documented, but I'm too new to Python to understand the latter without explanation. Could you supply at least the latter explanation? I'd like to understand the issue here, whether or not the kooky idea goes forward. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Fri, Feb 26, 2010 at 20:08, Glenn Linderman v+pyt...@g.nevcal.comv%2bpyt...@g.nevcal.com wrote: On approximately 2/26/2010 5:13 PM, came the following characters from the keyboard of Brett Cannon: On Fri, Feb 26, 2010 at 15:35, Glenn Linderman v+pyt...@g.nevcal.comv%2bpyt...@g.nevcal.commailto: v%2bpyt...@g.nevcal.com v%252bpyt...@g.nevcal.com wrote: On approximately 2/26/2010 2:55 PM, came the following characters from the keyboard of Brett Cannon: Maybe Greg's and my response to the mention of dropping this feature is too strong -- after all we're both dinosaurs. And maybe the developers who want the feature can write their own loader. We could also provide if necessary. So if the implementation stores .pyc by default in a version-specific place, then it seems there are only two things needed to make a python byte-code only distribution... 1) rename all the .pyc to .py 2) packaging When a .pyc is renamed to .py, Python (3.1 at least) recognizes and uses it... I assume by design, rather than accident, but I don't know the history. This does not work for me (nor should it): touch temp.py python3 -c import temp rm temp.py mv temp.pyc temp.py python3 -c import temp Traceback (most recent call last): File string, line 1, in module File temp.py, line 2 SyntaxError: Non-UTF-8 code starting with '\x95' in file temp.py on line 2, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details -Brett I'll admit to not doing exhaustive testing, but I'll not admit to not doing any testing... because it was sort of a wild idea. Someone else called it kooky, which is fair. What I did was: python -m test ren test.pyc foo.py foo.py and it worked. Then I posted, knowing that I'd also tested, the other day, several .py into a .zip named .py, and once that worked, then I changed to putting all .pyc into the .zip named .py and that worked too... including imports of the several modules from the __main__.pyc. Of course, all those were still named .pyc inside the .zip named .py. So I'm not sure what the difference is... .pyc as .py works from the command line, but not from import? Some specialty because of using -c ? I'd guess the technique could be made to work, probably not require extensive changes, if Python developers wanted to make it work. I think it could be efficient and that same someone that called it kooky admitted it would solve their use case, at least. I'm not sure why what you did is different than what I did, -M uses runpy which is not directly equivalent to importing. nor why you state without justification that it shouldn't work... It just is not supposed to happen that way. Masquerading a bytecode file as a source file shouldn't work; imp.get_suffixes() controls how files should be interpreted based on their file extension. -Brett I might be able to figure out the former if I spend enough time with the documentation, if it is documented, but I'm too new to Python to understand the latter without explanation. Could you supply at least the latter explanation? I'd like to understand the issue here, whether or not the kooky idea goes forward. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On Fri, Feb 26, 2010 at 5:13 PM, Brett Cannon br...@python.org wrote: On Fri, Feb 26, 2010 at 15:35, Glenn Linderman v+pyt...@g.nevcal.com When a .pyc is renamed to .py, Python (3.1 at least) recognizes and uses it... I assume by design, rather than accident, but I don't know the history. This does not work for me (nor should it): touch temp.py python3 -c import temp rm temp.py mv temp.pyc temp.py python3 -c import temp Traceback (most recent call last): File string, line 1, in module File temp.py, line 2 SyntaxError: Non-UTF-8 code starting with '\x95' in file temp.py on line 2, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details Try python temp.py though. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__
On approximately 2/26/2010 8:31 PM, came the following characters from the keyboard of Brett Cannon: I'm not sure why what you did is different than what I did, -M uses runpy which is not directly equivalent to importing. OK, that gives me some good keywords for searching documentation. What I (thought I) knew so far, was that it seemed to be equivalent, but that could easily be the 10,000' view instead of the reality. Thanks. nor why you state without justification that it shouldn't work... It just is not supposed to happen that way. Masquerading a bytecode file as a source file shouldn't work; imp.get_suffixes() controls how files should be interpreted based on their file extension. Well, since a .py can be a .zip, why not a .pyc? Just because no one thought of doing it before? Of course, I realize that I only know that a .py can be a .zip on the command line (is that runpy again, I'll bet it is), not for importing, which probably doesn't work, from what you imply. But if the technique can work from the command line, it seems the same technique could be re-used in the importer. A bytecode only .py would result in identical values for __file__ and __cached__ methinks. -- Glenn -- http://nevcal.com/ === A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
