[Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread anatoly techtonik
On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: This is the strongest reason why I recommend to everyone I know that they not use pickle for storage they'd like to keep working after upgrades [not just of stdlib, but other 3rd party software or their own

Re: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py

2010-11-04 Thread Victor Stinner
On Wednesday 03 November 2010 23:12:01 Nick Coghlan wrote: On Thu, Nov 4, 2010 at 1:01 AM, Benjamin Peterson benja...@python.org wrote: 2010/11/3 Nick Coghlan ncogh...@gmail.com: On Wed, Nov 3, 2010 at 10:19 PM, Benjamin Peterson benja...@python.org wrote: Warnings is loaded every time

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread exarkun
On 06:28 am, techto...@gmail.com wrote: On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: This is the strongest reason why I recommend to everyone I know that they not use pickle for storage they'd like to keep working after upgrades [not just of stdlib, but

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Nick Coghlan
On Thu, Nov 4, 2010 at 2:44 PM, Allan McRae al...@archlinux.org wrote: The second case was particularly interesting.  These software would change some of their #! to point at the python2 symlink and leave the rest pointing at python.  Note that python-2.7 itself falls into this category as many

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Nick Coghlan
On Thu, Nov 4, 2010 at 4:28 PM, anatoly techtonik techto...@gmail.com wrote: On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: This is the strongest reason why I recommend to everyone I know that they not use pickle for storage they'd like to keep working after

Re: [Python-Dev] Code coverage doesn't show .py stats

2010-11-04 Thread Walter Dörwald
On 03.11.10 19:21, anatoly techtonik wrote: Hi, Python code coverage doesn't include any .py files. What happened? http://coverage.livinglogic.de/ Did it work before? It did, however currently the logfile http://coverage.livinglogic.de/testlog.txt shows the following exception:

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Antoine Pitrou
On Thu, 4 Nov 2010 23:33:38 +1000 Nick Coghlan ncogh...@gmail.com wrote: Tools also had a few discrepancies: scripts/2to3.py: /usr/bin/env python (necessary, I think - I believe 2to3 is a 2.x only program) scripts/gprof2html.py: /usr/bin/env python32.3 (Huh? Automated correction gone

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Benjamin Peterson
2010/11/4 Nick Coghlan ncogh...@gmail.com: On Thu, Nov 4, 2010 at 2:44 PM, Allan McRae al...@archlinux.org wrote: The second case was particularly interesting.  These software would change some of their #! to point at the python2 symlink and leave the rest pointing at python.  Note that

Re: [Python-Dev] [Python-checkins] r85987 - python/branches/py3k/Lib/test/test_os.py

2010-11-04 Thread Hirokazu Yamamoto
On 2010/11/02 1:30, Nick Coghlan wrote: On Tue, Nov 2, 2010 at 2:10 AM, Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp wrote: Does this really cause resource warning? I think os.popen instance won't be into traceback because it's not declared as variable. So I suppose it will be deleted by

Re: [Python-Dev] [Python-checkins] r85987 - python/branches/py3k/Lib/test/test_os.py

2010-11-04 Thread Antoine Pitrou
On Thu, 04 Nov 2010 23:09:39 +0900 Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp wrote: On 2010/11/02 1:30, Nick Coghlan wrote: On Tue, Nov 2, 2010 at 2:10 AM, Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp wrote: Does this really cause resource warning? I think os.popen instance won't be

Re: [Python-Dev] [Python-checkins] r85987 - python/branches/py3k/Lib/test/test_os.py

2010-11-04 Thread Hirokazu Yamamoto
On 2010/11/04 23:23, Antoine Pitrou wrote: You can use all the usual means of controlling emission of warnings, so for example python -Wi would work to silence them all. Also, ResourceWarning is silenced by default in release builds. Regards Antoine. Thank you, this works. (I couldn't find

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Guido van Rossum
On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: This is the strongest reason why I recommend to everyone I know that they not use pickle for storage they'd like to keep working after upgrades [not just of stdlib, but other 3rd party software or their own

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Alexander Belopolsky
On Thu, Nov 4, 2010 at 10:51 AM, Guido van Rossum gu...@python.org wrote: .. Twisted actually tried to preserve pickle compatibility in the bad old days, but it was impossible.  Pickles should never really be saved to disk unless they contain nothing but lists, ints, strings, and dicts. But

Re: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py

2010-11-04 Thread Nick Coghlan
On Thu, Nov 4, 2010 at 9:16 PM, Victor Stinner victor.stin...@haypocalc.com wrote: So do you still think that I should patch the os module to use a global import or not? I'm actually more inclined to suggest we avoid triggering the warning under -bb in the first place by iterating over the

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Barry Warsaw
On Nov 04, 2010, at 02:44 PM, Allan McRae wrote: While this is not strictly related to python development, I thought that developers of python might be interested in some of the lessons provided by this. So forgive me if this is really wrong for this list... Recently Arch Linux did a big

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Barry Warsaw
On Nov 04, 2010, at 11:33 PM, Nick Coghlan wrote: world: /usr/bin/env python (I have no idea what this script is even for) It's basically a front-end to ISO 3166 country codes. IOW, it prints the expansion of top-level domain names and can do some reverse lookups too. E.g. %

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread anatoly techtonik
On Thu, Nov 4, 2010 at 3:38 PM, Nick Coghlan ncogh...@gmail.com wrote: On Thu, Nov 4, 2010 at 4:28 PM, anatoly techtonik techto...@gmail.com wrote: On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: This is the strongest reason why I recommend to everyone I know

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Guido van Rossum
On Thu, Nov 4, 2010 at 9:15 AM, anatoly techtonik techto...@gmail.com wrote: pickle is insecure, marshal too. What's the attack you're thinking of on marshal? It never executes any code while unmarshalling (although it can unmarshal code objects -- but the receiving program has to do something

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Thomas Wouters
On Thu, Nov 4, 2010 at 05:44, Allan McRae al...@archlinux.org wrote: According to #python, we are all idiots To clarify (but I dont speak for the rest of #python, just myself), I think the move was premature, but I don't use Arch and I don't know what typical Arch users expect. The reason

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Martin v. Löwis
To clarify (but I dont speak for the rest of #python, just myself), I think the move was premature, but I don't use Arch and I don't know what typical Arch users expect. The reason I think it's premature is that 'python2' just doesn't work everywhere, and I would have gone for a transitionary

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Glyph Lefkowitz
On Nov 4, 2010, at 12:49 PM, Guido van Rossum wrote: What's the attack you're thinking of on marshal? It never executes any code while unmarshalling (although it can unmarshal code objects -- but the receiving program has to do something additionally to execute those). These issues may have

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Matthias Klose
On 04.11.2010 21:12, Martin v. Löwis wrote: To clarify (but I dont speak for the rest of #python, just myself), I think the move was premature, but I don't use Arch and I don't know what typical Arch users expect. The reason I think it's premature is that 'python2' just doesn't work everywhere,

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Thomas Wouters
On Thu, Nov 4, 2010 at 21:12, Martin v. Löwis mar...@v.loewis.de wrote: As for #python, well, we got this storm of people utterly confused about how their stuff doesn't work anymore, and putting the blame in the wrong place. I don't think a distribution should ever cause that (even though

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Guido van Rossum
On Thu, Nov 4, 2010 at 1:25 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: On Nov 4, 2010, at 12:49 PM, Guido van Rossum wrote: What's the attack you're thinking of on marshal? It never executes any code while unmarshalling (although it can unmarshal code objects -- but the receiving

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Laurens Van Houtven
On Thu, Nov 4, 2010 at 5:44 AM, Allan McRae al...@archlinux.org wrote: According to #python, we are all idiots I realize this is not really what your message was about and for sake of brevity you used a bit of a hyperbole, but like Thomas I would still like to nip in right there. #python is

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Allan McRae
On 05/11/10 08:40, Laurens Van Houtven wrote: On Thu, Nov 4, 2010 at 5:44 AM, Allan McRaeal...@archlinux.org wrote: According to #python, we are all idiots I realize this is not really what your message was about and for sake of brevity you used a bit of a hyperbole, but like Thomas I

[Python-Dev] Help with warnings not being raised

2010-11-04 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all. I just committed r86180, but there is something I don't like. If you read the tests I did (by hand)at http://bugs.python.org/issue9675#msg120462 , python should show the unraisable and THEN the C API unavailable warning, but it is not showing

Re: [Python-Dev] Help with warnings not being raised

2010-11-04 Thread Benjamin Peterson
2010/11/4 Jesus Cea j...@jcea.es: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all. I just committed r86180, but there is something I don't like. If you read the tests I did (by hand)at http://bugs.python.org/issue9675#msg120462 , python should show the unraisable and THEN the C API

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Marcel Hellkamp
Am 04.11.2010 17:15, schrieb anatoly techtonik: pickle is insecure, marshal too. If the transport or storage layer is not save, you should cryptographically sign the data anyway:: def pickle_encode(data, key): msg = base64.b64encode(pickle.dumps(data, -1)) sig =

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Stephen J. Turnbull
Thomas Wouters writes: To clarify (but I dont speak for the rest of #python, just myself), I think the move was premature, but I don't use Arch and I don't know what typical Arch users expect. All of the Arch users I know expect Arch to occasionally do radical things because they're the

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Devin Cook
On Thu, Nov 4, 2010 at 7:19 PM, Allan McRae al...@archlinux.org wrote: I also agree with the NO ARCH topic at the moment. I was fairly surprised so many people went to #python for help given we had made news posts and had a topic in our IRC channel pointing to how to start fixing issues.

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Steven D'Aprano
Nick Coghlan wrote: As a tool for communicating between different instances of the *same* version of Python though, pickle is fine. I'm using pickle to pass a list and dict of floats and strings from Python 2.6 to 3.1. I've never had any problems with it. Am I living in a state of sin or is

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread James Y Knight
On Nov 4, 2010, at 8:43 PM, Stephen J. Turnbull wrote: All of the Arch users I know expect Arch to occasionally do radical things because they're the right things to do in the long run. But the previous consensus (at least, as I, and presumably many other people understood it) was that

Re: [Python-Dev] Help with warnings not being raised

2010-11-04 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/11/10 01:36, Benjamin Peterson wrote: I don't know why. Are you passing -3 -Wall? I am passing -3 -Werror, to induce the error control I have committed. - -- Jesus Cea Avion _/_/ _/_/_/_/_/_/

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/11/10 05:44, Allan McRae wrote: The second case was particularly interesting. These software would change some of their #! to point at the python2 symlink and leave the rest pointing at python. Note that python-2.7 itself falls into this

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/11/10 15:57, Alexander Belopolsky wrote: On Thu, Nov 4, 2010 at 10:51 AM, Guido van Rossum gu...@python.org wrote: .. Twisted actually tried to preserve pickle compatibility in the bad old days, but it was impossible. Pickles should never

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Steven D'Aprano
James Y Knight wrote: But the previous consensus (at least, as I, and presumably many other people understood it) was that python2 would remain the owner of the name /usr/bin/python for the indefinite future, and python3 would be invoked with /usr/bin/python3. Given that, it's not at all clear

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread exarkun
On 12:21 am, m...@gsites.de wrote: Am 04.11.2010 17:15, schrieb anatoly techtonik: pickle is insecure, marshal too. If the transport or storage layer is not save, you should cryptographically sign the data anyway:: def pickle_encode(data, key): msg =

Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

2010-11-04 Thread Bob Ippolito
On Friday, November 5, 2010, exar...@twistedmatrix.com wrote: On 12:21 am, m...@gsites.de wrote: Am 04.11.2010 17:15, schrieb anatoly techtonik: pickle is insecure, marshal too. If the transport or storage layer is not save, you should cryptographically sign the data anyway::    def

Re: [Python-Dev] Python-3 transition in Arch Linux

2010-11-04 Thread Allan McRae
On 05/11/10 11:20, Jesus Cea wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/11/10 05:44, Allan McRae wrote: The second case was particularly interesting. These software would change some of their #! to point at the python2 symlink and leave the rest pointing at python. Note that