Re: [Python-Dev] peps: Update PEP 399 to include comments from python-dev.
On Apr 12, 2011, at 7:50 PM, Tres Seaver wrote: Trying to accelerate existing code which doesn't have the coverage is insane: how can you know that the accelerator doesn't subtly change the semantics without tests? But even if you do have 100% python source code branch coverage, that's not nearly enough. There are thousands of branches inside the python interpreter, which you also need to have full coverage on to *really* ensure that the behavior of the code does not subtly change. Good luck with that. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 399: Pure Python/C Accelerator Module Compatibiilty Requirements
On Apr 6, 2011, at 10:08 AM, Nick Coghlan wrote: On Wed, Apr 6, 2011 at 5:57 AM, Raymond Hettinger raymond.hettin...@gmail.com wrote: [Brett] This PEP requires that in these instances that both the Python and C code must be semantically identical Are you talking about the guaranteed semantics promised by the docs or are you talking about every possible implementation detail? ISTM that even with pure python code, we get problems with people relying on implementation specific details. Indeed. Argument handling is certainly a tricky one - getting positional only arguments requires a bit of a hack in pure Python code (accepting *args and unpacking the arguments manually), but it comes reasonably naturally when parsing arguments directly using the C API. Perhaps the argument handling for C functions ought to be enhanced to work like python's argument handling, instead of trying to hack it the other way around? James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 399: Pure Python/C Accelerator Module Compatibiilty Requirements
On Apr 6, 2011, at 4:44 PM, s...@pobox.com wrote: Brett How about the test suite needs to have 100% test coverage (or as Brett close as possible) on the pure Python version? Works for me, but you will have to define what 100% is fairly clearly. 100% of the lines get executed? All the branches are taken? Under what circumstances might the 100% rule be relaxed? And...does that include all branches taken within the interpreter too? :) E.g. check whether all possible exceptions are thrown in all possible places an exception could be thrown? (As per the exception compatibility subthread) And what about all the possible crazy stuff you could do in callbacks back to user code (e.g. mutating arguments passed to the initial function, or installing a trace hook or...)? Does use of the function as a class attribute need to be covered? (see previous discussion on differences in behavior due to descriptors). Etcetc. I'd love it if CPython C modules acted equivalently to python code, but there is almost an endless supply of differences...100% test coverage of the behavior seems completely infeasible if interpreted strictly; some explicit subset of all possible behavior needs to be defined for what users cannot reasonably depend on. (sys.settrace almost certainly belonging on that list :).) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API deprecations in Python 3, from a Python 2 perspective
On Mar 17, 2011, at 4:07 PM, Raymond Hettinger wrote: Agreeing with Guido is always a good move :-) In addition, any new deprecations in Py3.x can be marked with py3k warnings in Py2.7 point releases. That would give users the maximum chance to make updates before porting, even if they wait five years to migrate to Py3.x. And of course, updating 2to3, if possible, would also be appreciated. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] public visibility of python-dev decisions before it's too late
On Mar 15, 2011, at 10:14 PM, Lennart Regebro wrote: On Tue, Mar 15, 2011 at 21:54, Antoine Pitrou solip...@pitrou.net wrote: I don't know what other core devs, but I don't think this discussion is going anywhere. If porting the ZTK is a burden for you, perhaps you should try to find some financial support for it (or let other people do it for you), rather than accusing the python-dev community. Up until the reactions from the core Python developers on these real world problems, it was hard work, but also fun. It isn't anymore, and I hear your message, loud and clear, so indeed, somebody else will have to do it. I've lost interest. It seems a bit odd for you to be so surprised about this, considering Python3 is a *huge* compatibility break that broke basically every single thing ever written in Python. And I'm not sure why you're making such a big deal about this one corner of incompatibility -- PyCObject vs. PyCapsule seems like such a trivial and easy to work around issue compared to the massive pain-in-the-ass porting job for all the incompatibilities on the Python-language side of the fence. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] GPL'd python code vs Python2.6 linked against OpenSSL
It's well known that OpenSSL is incompatible with the GPL. [1] Python (from 2.6) is *always* linked against openssl, instead of waiting for you to import ssl. Doesn't this mean it's now impossible (rather, a license violation) to distribute a GPL'd python program (or to use a GPL'd library in your python program)? This seems like a problem... Thanks to Ulrik Sverdrup (http://lists.debian.org/debian-python/2011/03/msg00082.html) for pointing this out... James [1] http://people.gnome.org/~markmc/openssl-and-the-gpl.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GPL'd python code vs Python2.6 linked against OpenSSL
On Mar 9, 2011, at 6:45 PM, Sandro Tosi wrote:
It seems introduced by the patch debian/patches/setup-modules-ssl.diff
with description # DP: Modules/Setup.dist: patch to build _hashlib
and _ssl extensions statically
Indeed you're right -- out of the box, python still builds _ssl.so as a
separate module. Sorry for the misinformation -- indeed *nothing* has changed
here upstream, it was just a build patch added to debian's python packaging.
[[
BTW, the actual thing causing that special behavior is not that diff, but
rather is in debian/rules:
egrep \
^#($$(awk '$$2 ~ /^extension$$/ {print $$1}'
debian/PVER-minimal.README.Debian.in | tr '\012' '|')XX) \
Modules/Setup.dist \
| sed -e 's/^#//' -e 's/-Wl,-Bdynamic//;s/-Wl,-Bstatic//' \
$(buildd_shared)/Modules/Setup.local
which causes all the modules listed as extension in
PVER-minimal.README.Debian.in to be compiled in to the python interpreter
instead of built as separate .so files.
]]
Despite that not having changed recently, it still seems like there's a problem
in upstream python, which perhaps nobody has thought much about: even if you
don't use SSL, many modules indirectly import either _hashlib or _ssl, and thus
your program is linked against openssl.
You can of course prevent that from happening:
sys.modules['_ssl'] = None; sys.modules['_hashlib'] = None
Luckily, basically everything is already written to keep working if 'import
ssl' fails, so that's fine. (assuming you don't want SSL support, of course).
But you're also left with not being able to 'import hashlib'. While python has
fallback code, those modules (_md5, _sha, _sha256, _sha512) aren't built if
openssl was found at build time. So you can't just select at runtime that you
didn't want to use openssl.
Not being able to import hashlib unfortunately makes urllib2 (and a lot of 3rd
party packages) fail to import.
Possibly distributors like Debian should just cause _hashopenssl to never be
used, and always use the fallback code, for maximal license compatibility?
On Mar 9, 2011, at 8:27 PM, Joao S. O. Bueno wrote:
Any libraries commonly avaliable with a CPython instalation can be
considered as system libraries for GPL purposes - and so
this would fall in the system library exception as described by the FAQ:
The system library exception is often held not to apply when you are
distributing your program as part of the same system as said system library.
The system library exception would also not be applicable if it is the
license of a system library itself that you're violating, e.g. if I have a
BSD-licensed program which starts with import hashlib, readline.
See also:
http://curl.haxx.se/legal/distro-dilemma.html
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream
On Mar 7, 2011, at 3:49 PM, Paul Moore wrote: The launcher could also (as per Mark's suggestion) interpret a shebang line in the script, so that scripts could specify their required version without needing a different command,or multiple version-specific extensions. Note that, on Unix, python file.py doesn't choose the correct version of python to run by looking at a shebang, it just runs the version of python installed as python. Only ./file.py looks at the shebang (assuming that file is marked executable). Is the proposal to make python.exe do that on windows? That is a rather significant difference from the unix behavior. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEPs] Support the /usr/bin/python2 symlink upstream
On Mar 4, 2011, at 4:21 PM, Martin v. Löwis wrote: and setting PYTHONPATH will continue to break installations). Indeed, it's really *quite* unfortunate that the proposal to make python3 use PYTHON3PATH instead of PYTHONPATH was rejected. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream
On Mar 3, 2011, at 3:55 AM, Piotr Ożarowski wrote: I don't really mind adding /usr/bin/python2 symlink just to clean Arch mess Is there any chance you would add the symlink in the next Debian stable point release? If both Debian and Python upstream added the python2 symlink in the next stable update rather than waiting for the next major release (which for Python itself is of course never), that could reduce the problem of systems not having the symlink installed quite significantly. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream
On Mar 2, 2011, at 8:23 AM, Sandro Tosi wrote: On Wed, Mar 2, 2011 at 13:56, Piotr Ożarowski pi...@debian.org wrote: [Sandro Tosi, 2011-03-02] On Wed, Mar 2, 2011 at 10:01, Piotr Ożarowski pi...@debian.org wrote: I co-maintain with Matthias a package that provides /usr/bin/python symlink in Debian and I can confirm that it will always point to Python 2.X. We also do not plan to add /usr/bin/python2 symlink (and I guess only accepted PEP can change that) Can you please explain why you NACK this proposed change? it encourages people to change /usr/bin/python symlink to point to python3.X which I'm strongly against (how can I tell that upstream author meant python3.X and not python2.X without checking the code?) with 'people' do you mean 'users'? if so, isn't this risk already present? If you, user, change the python symlink (provided by python-minimal in Debian) to something else than what's shipped, it's still a local change, and will never be supported; but with python2 *Debian is free* to decide if python can be pointed to python3, if the time will come. I suspect he's saying it'd be better if the time didn't come (if so, I'd agree). Python3 *is* unfortunately a new and incompatible programming language, it makes sense for it to have it have its own interpreter name. Eventually /usr/bin/python might no longer be installed, but that doesn't mean python3 shouldn't simply be called python3 forever. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream
On Mar 2, 2011, at 9:54 AM, Allan McRae wrote: That way in ?? years when python-3.x is the python and python-2.x is obsolete, and it is decided that /usr/bin/python will be python-3.x (which I believe is the only logical outcome), But that's not the only logical outcome. A perfectly logical outcome is that /usr/bin/python disappears completely if python2.X isn't installed, and python3 is always called python3. That is the outcome I find sensible. And that is the crux of the disagreement in this thread. Those who think python3.X should stay /usr/bin/python3 forever do not see any reason to make everyone rewrite their existing python scripts to say /usr/bin/python2 instead of /usr/bin/python. So, there's no point in adding a /usr/bin/python2 now. Scripts that want python2 can remain using /usr/bin/python forever, and that will either be installed, or not installed, depending on whether that OS has a copy of python2.X. Those who think python3 should (eventually someday, or maybe immediately, depending) be named or have an alias of /usr/bin/python want to make everyone rewrite their scripts to say /usr/bin/python2 now. For that position, it's unfortunate that python source doesn't install itself with an alias of /usr/bin/python2, and some distros don't install that alias either. So they want to fix that. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream
On Mar 2, 2011, at 12:14 PM, Barry Warsaw wrote: I don't have a problem with adding such a symlink, and I think it should be done by Informational PEP, not Standards Track PEP. Since there will be no Python 2.8, our own build system shouldn't ever be changed to add such a link, but we can recommend it for consistency among distros, which would be free to adopt it or not. Why not? 2.7 is supposed to be in long term maintenance mode. Surely if it's a good idea for everyone else to ship a python2 binary, 2.7.next should also install it when building from source... James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream
On Mar 2, 2011, at 11:42 AM, R. David Murray wrote: Well, I personally won't use a distribution that makes this choice. For whatever that's worth :) This ***shouldn't*** be a choice distros have to make. There should be a standard upstream recommended way to install python, and that's also what make install should do. That distros are having to make a choice here is a problem in communication from python core developers -- it sucks that we've gotten this far without consensus on a proper transition plan for moving from Python 2.X to Python 3.X. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream
On Mar 2, 2011, at 5:04 PM, Martin v. Löwis wrote: Am 02.03.2011 20:49, schrieb James Y Knight: On Mar 2, 2011, at 12:14 PM, Barry Warsaw wrote: I don't have a problem with adding such a symlink, and I think it should be done by Informational PEP, not Standards Track PEP. Since there will be no Python 2.8, our own build system shouldn't ever be changed to add such a link, but we can recommend it for consistency among distros, which would be free to adopt it or not. Why not? 2.7 is supposed to be in long term maintenance mode. Surely if it's a good idea for everyone else to ship a python2 binary, 2.7.next should also install it when building from source... I agree with Barry that this would be a new feature, and, by default, cannot be added to the 2.7 release which is in maintenance mode. IMO, an accepted PEP could override the policy, though. That sounds like an entirely reasonable position to take. All the more reason for someone who's in favor of python3 being called python in the future to write the PEP outlining how to ease the transition by providing a python2 link now. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream
On Mar 2, 2011, at 7:01 PM, Kerrick Staley wrote: As an aside, this whole thing started when I tried installing ROS, only to find that it made assumptions about /usr/bin/python, which points to python3 on my Arch Linux system. Yep, exactly that kind of problem is why I think it's an absolutely terrible idea to switch the /usr/bin/python link to point to python3 (ever). When python 2.x is dead, I really don't see what the problem is with only having a python3 binary, and no python binary. That said, since the conclusion here is that it *IS* a good idea to point python to a python3 interpreter eventually, I guess it's better to add the python2 link 4 years late than never. A lot of pain could've been spared if python 2.x had started installing python2 years ago, so by now everyone would depend on it existing. But oh well, too late for that, unless someone has a time machine handy. I went in search of documentation for the python2/python3 convention so that I could make the ROS developers aware of it and help them to follow it, only to find out that no such convention actually exists. I'm sure there are many other users out there that are frustrated by similar issues; Well, so far, only those unfortunate users of Arch Linux...but considering the consensus here, I'm sure there will be more in the future. supporting a python2/python3 convention on all distros as well as in scripts would solve these issues without creating further problems and without the need for a slow consensus to be reached on what /usr/bin/python should be. Well, it will definitely will create problems: scripts may start using the python2 name to be compatible with Arch Linux (or anyone else who sets python-python3), but the python2 link won't exist on any existing from-source Python install, or OSX, or Debian, or Ubuntu. And it likely will not start existing on some of those systems for years to come, even if the PEP is accepted today. Perhaps that problem is considered less of a problem than the problem Arch Linux users have today (as you point out, sysadmins can create the link themselves), but it still is a problem. As to the PEP itself: you should specify an action item that the Python 2.7.N upstream makefile be modified to install a python2 symlink, as well. James___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support the /usr/bin/python2 symlink upstream
On Mar 1, 2011, at 5:06 PM, Guido van Rossum wrote: On Tue, Mar 1, 2011 at 1:26 PM, Eric Smith e...@trueblade.com wrote: On 3/1/2011 4:19 PM, Kerrick Staley wrote: Hello, There is a need for the default Python2 install to place a symlink at /usr/bin/python2 that points to /usr/bin/python, or for the documentation to recommend that packagers ensure that python2 is defined. Also, all documentation should be changed to recommend that #!/usr/bin/env python2 be used as the shebang for Python 2 scripts. This is needed because some distributions (Arch Linux, in particular), point /usr/bin/python to /usr/bin/python3, while others (including Slackware, Debian, and the BSDs, probably more) do not even define the python2 command. This means that a script has no way of achieving cross-platform compatibility. The point at which many distributions begin to alias /usr/bin/python to /usr/bin/python3 is due soon, and for the next couple of years, it would be best to use a python2 or python3 shebang in all scripts, making no assumptions about plain python, which should only be invoked interactively. This email from about 3 years ago seems relevant: : http://mail.python.org/pipermail/python-3000/2008-March/012421.html Again, this issue needs to be addressed by the Python developers themselves so that different *nix distributions will handle it consistently, allowing Python scripts to continue to be cross-platform. I believe we agreed at the language summit last year (or maybe even the year before) that python would always be python2.x, and python3 would be python3.x. And by always we indeed meant forever. To do otherwise would break scripts even many, many years from now. Unfortunately distros are not following these guidelines. As long as we still have the pythonX.Y links I think it's better to have python2, python3 and python than total anarchy. If python upstream would make it clear that that *IS* the policy, distros might follow it. Right now, there is no clear guidance, as far as I can tell. If you do not want distros making python be a link to python3, please say so loudly, preferably on a webpage on python.org that users can point the distros to. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Link to issue tracker
On Feb 23, 2011, at 3:05 PM, Brett Cannon wrote: On Wed, Feb 23, 2011 at 11:52, Martin v. Löwis mar...@v.loewis.de wrote: Am 23.02.2011 19:30, schrieb Brett Cannon: I won't add the link back Why not? It's a useful link apparently. The Developer's Guide link does not hint that it will be the only way to find the bug tracker. But python.org/dev/ is a dead page. I was trying to avoid adding a redirect for python.org/dev/ as I was afraid that it would lead to the website doing something silly like redirecting everything below that URL, but obviously this can continue since people seem to think that python.org/dev/ has anything useful on it (which it does not). It seems unfortunate that the Core Development link now points directly to the devguide, since it unexpectedly breaks the navigation UI. It seemed rather more useful and consistent to have Core Development show the page with quick links. James___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Link to issue tracker
On Feb 23, 2011, at 4:00 PM, Brett Cannon wrote: On Wed, Feb 23, 2011 at 12:38, James Y Knight f...@fuhm.net wrote: On Feb 23, 2011, at 3:05 PM, Brett Cannon wrote: On Wed, Feb 23, 2011 at 11:52, Martin v. Löwis mar...@v.loewis.de wrote: Am 23.02.2011 19:30, schrieb Brett Cannon: I won't add the link back Why not? It's a useful link apparently. The Developer's Guide link does not hint that it will be the only way to find the bug tracker. But python.org/dev/ is a dead page. I was trying to avoid adding a redirect for python.org/dev/ as I was afraid that it would lead to the website doing something silly like redirecting everything below that URL, but obviously this can continue since people seem to think that python.org/dev/ has anything useful on it (which it does not). It seems unfortunate that the Core Development link now points directly to the devguide, since it unexpectedly breaks the navigation UI. It seemed rather more useful and consistent to have Core Development show the page with quick links. Honestly, working on pydotorg is too bloody painful to even conceive of doing this purely to keep a single link onwww.python.org. Well, presumably at least 5 links: Devguide, PEP index, buildbot, issue tracker, link to source code browser. But sure, I'm just a user, I have no idea how hard it is to edit a page on pydotorg (and no, I don't really want to know). But I seriously cannot imagine it's so hard that you can't leave a page with a few links there, that had already been written! Note how clicking everything else in the left navbar makes useful links appear below the item, and the rest of the page keep the same theme/general layout. Core Development is now the unexpected exception to that UI consistency, and is less useful because of it. James___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] w9xpopen.exe is still in 3.2
On Feb 20, 2011, at 4:10 AM, Martin v. Löwis wrote: Am 20.02.2011 07:43, schrieb anatoly techtonik: Python definitely needs a development Roadmap to avoid things like w9xpopen.exe slipping off radar from release to release. We don't support Windows 9x since Python 2.6. What this file does in 3.x distributions? http://bugs.python.org/issue2405 Read the report carefully. It can't be removed to support installations that have changed COMSPEC. Does a modern windows installation actually even *work* if you change COMSPEC to command.com instead of cmd.exe? And why would anyone ever do that? Hey, I have a good idea: python can just ignore COMSPEC and always run cmd.exe. Then you can delete w9xpopen, hooray. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] short fetch for NEXTARG macro (was: one byte byte code arguments)
On Jan 31, 2011, at 7:45 AM, Antoine Pitrou wrote: On Mon, 31 Jan 2011 13:28:39 +0100 Jurjen N.E. Bos jurjen@hetnet.nl wrote: I just did it: my first python source code hack. I replaced the NEXTARG and PEEKARG macros in ceval.c using a cast to short pointer, and lo and behold, a crude measurement indicates one to two percent speed increase. That isn't much, but it is virtually for free! Here are the macro's I used: #define NEXTARG() (next_instr +=2, *(short*)next_instr[-2]) #define PEEKARG() (*(short*)next_instr[1]) Some architectures forbid unaligned access, so this can't be used as-is. It could perhaps be #ifdef'd in on x86/x86-64, though, which is by far the most common architecture to run python on. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 393: Flexible String Representation
On Jan 27, 2011, at 2:06 PM, Stefan Behnel wrote: Martin v. Löwis, 24.01.2011 21:17: The Py_UNICODE type is still supported but deprecated. It is always defined as a typedef for wchar_t, so the wstr representation can double as Py_UNICODE representation. It's too bad this isn't initialised by default, though. Py_UNICODE is the only representation that can be used efficiently from C code and Cython relies on it for fast text processing. This proposal will therefore likely have a pretty negative performance impact on extensions written in Cython as the compiler could no longer expect this representation to be available instantaneously. But the whole point of the exercise is so that it doesn't have to store a 4byte-per-char representation when a 1byte-per-char rep would do. If cython wants to work most efficiently with this proposal, it should learn to deal with the three possible raw representations. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Import and unicode: part two
On Jan 26, 2011, at 4:40 AM, Victor Stinner wrote: During Python 3.2 development, we tried to be able to use a filesystem encoding different than the locale encoding (PYTHONFSENCODING environment variable): but it doesn't work simply because Python is not alone in the OS. Except Python, all programs speak the same language: the locale encoding. Let's try to give you an example: if create a module with a name encoded to UTF-8, your file browser will display mojibake. Is that really true? I'm pretty sure GTK+ treats all filenames as UTF-8 no matter what the locale says. (over-rideable by G_FILENAME_ENCODING or G_BROKEN_FILENAMES) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Import and unicode: part two
On Jan 26, 2011, at 11:47 AM, Victor Stinner wrote: Not exactly. Gtk+ uses the glib library, and to encode/decode filenames, the glib library uses: - UTF-8 on Windows - G_FILENAME_ENCODING environment variable if set (comma-separated list of encodings) - UTF-8 if G_BROKEN_FILENAMES env var is set - or the locale encoding But the documentation says: On Unix, the character sets are determined by consulting the environment variables G_FILENAME_ENCODING and G_BROKEN_FILENAMES. On Windows, the character set used in the GLib API is always UTF-8 and said environment variables have no effect. G_FILENAME_ENCODING may be set to a comma-separated list of character set names. The special token @locale is taken to mean the character set for thecurrent locale. If G_FILENAME_ENCODING is not set, but G_BROKEN_FILENAMES is, the character set of the current locale is taken as the filename encoding. If neither environment variable is set, UTF-8 is taken as the filename encoding, but the character set of the current locale is also put in the list of encodings. Which indicates to me that (unless you override the behavior with env vars) it encodes filenames in UTF-8 regardless of the locale, and attempts decoding in UTF-8 primarily. And that only when the filename doesn't make sense in UTF-8, it will also try decoding it in the locale encoding. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mail archive line wrapping (Was: Import and unicode: part two)
On Jan 21, 2011, at 6:31 AM, Antoine Pitrou wrote:
On Thu, 20 Jan 2011 22:25:17 -0500
James Y Knight f...@fuhm.net wrote:
On Jan 20, 2011, at 3:55 PM, Antoine Pitrou wrote:
(by the way, it would be nice if your text/mail editor wrapped lines at
80 characters or something)
You could complain to Apple, but it seems unlikely that they'd change it.
They broke it intentionally in OSX 10.6.2 for better compatibility with MS
Outlook.
(for the technically inclined: It still wraps lines at 80 characters in the
raw message, but it uses quoted-printable encoding to escape the
line-breaks, so mail readers which decode quoted-printable but can't flow
text are now S.O.L. Apple used to use the nice format=flowed standard
instead.)
I think most mail readers are able to word-wrap raw text correctly
(even though it still makes your messages look bad amongst a thread of
nicely-formatted 80-column messages).
The real annoyance is when reading Web archives of mailing-lists, e.g.
http://twistedmatrix.com/pipermail/twisted-python/2011-January/023346.html
Well, yes, that's a pretty annoying bug in mailman, isn't it? If only anyone
around here was involved in mailman and could fix it! :) [I've attempted to cc
this to mailman-users with this message, but since I'm not subscribed I dunno
if it'll make it or not.]
I have this in my user CSS override file to fix the issue for myself globally
on all such archives out in the world:
/* Mailing list archives */
htmlbodypre { white-space: pre-wrap !important; }
But really, pipermail should just output a suitable style itself, e.g.: pre
style=white-space: pre-wrap or a stylepre { white-space: pre-wrap;
}/style in the header.
That's supported on all browsers since FF3.0, IE8, Safari 3, Opera 8. There are
various nonstandard CSS selectors for reaching older browsers (IE5.5+, Firefox
pre-1.0+, Opera 4+)...But by the time this change gets made in mailman, and
released, and gets into the distros that the various list hosts around the web
use, and those hosts get upgraded, I doubt anyone will actually even be able to
run those old browsers anymore.
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Import and unicode: part two
On Jan 19, 2011, at 11:39 PM, Toshio Kuratomi wrote: Tangent: This is not true about Linux. UTF-8 is a matter of the interpretation of the filesystem bytes that the user specifies by setting their system locale. Setting system locale to ASCII for use in system-wide scripts, is quite common as is changing locale settings in other parts of the world (as I can tell you from the bug reports colleagues CC me on to fix for the problems with unicode support in their python2 programs). Fortunately, there's been some (slow) movement towards adding a C.UTF-8 locale and using that by default where C (ASCII) is currently used. So that may be less of a problem in a few years time. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87815 - peps/trunk/pep-3333.txt
On Jan 20, 2011, at 9:31 PM, Ezio Melotti wrote:
Modified: peps/trunk/pep-.txt
==
--- peps/trunk/pep-.txt (original)
+++ peps/trunk/pep-.txt Fri Jan 7 16:39:27 2011
@@ -310,9 +310,9 @@
elif not headers_sent:
# Before the first output, send the stored headers
status, response_headers = headers_sent[:] = headers_set
- sys.stdout.write('Status: %s\r\n' % status)
+ sys.stdout.buffer.write('Status: %s\r\n' % status)
for header in response_headers:
- sys.stdout.write('%s: %s\r\n' % header)
+ sys.stdout.buffer.write('%s: %s\r\n' % header)
Also note that .buffer might not be available in some cases (i.e. when
sys.stdout has been replaced with other objects).
Do you have a recommendation for a better way to do bytes I/O on stdin/sydout,
then?...just saying that .buffer might not be available isn't a very useful
comment unless there's a replacement idiom...
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Import and unicode: part two
On Jan 20, 2011, at 3:55 PM, Antoine Pitrou wrote: On Thu, 20 Jan 2011 15:27:08 -0500 Glyph Lefkowitz gl...@twistedmatrix.com wrote: To support the latter, could we just make sure that zipimport has a consistent, non-locale-or-operating-system-dependent interpretation of encoding? It already has, but it's dependent on a flag in the zip file itself (actually, one flag per archived file in the zip it seems). (by the way, it would be nice if your text/mail editor wrapped lines at 80 characters or something) You could complain to Apple, but it seems unlikely that they'd change it. They broke it intentionally in OSX 10.6.2 for better compatibility with MS Outlook. (for the technically inclined: It still wraps lines at 80 characters in the raw message, but it uses quoted-printable encoding to escape the line-breaks, so mail readers which decode quoted-printable but can't flow text are now S.O.L. Apple used to use the nice format=flowed standard instead.) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Import and unicode: part two
On Jan 19, 2011, at 6:44 PM, Toshio Kuratomi wrote: This problem of which encoding to use is a problem that can be seen on UNIX systems even now. Try this: echo 'print(hi)' café.py convmv -f utf-8 -t latin1 café.py python3 -c 'import café' ASCII seems very sensible to me when faced with these ambiguities. Other options I can brainstorm that could be explored: * Specify an encoding per platform and stick to that. (So, for instance, all module names on posix platforms would have to be utf-8). Force translation between encoding when installing packages (But that doesn't help for people that are creating their modules using their own build scripts rather than distutils, copying the files using raw tar, etc.) * Change import semantics to allow specifying the encoding of the module on the filesystem (seems really icky). None of this is unique to import -- the same exact issue occurs with open(u'café'). I don't see any reason why import café should be though of as more of a problem, or treated any differently. It's reasonable to recommend that people use ASCII in their module names if they want wide portability, but it should still be supported to use non-ASCII. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add sendfile() to core?
If you're gonna wrap sendfile, it might be nice to also wrap the splice, tee, and vmsplice syscalls on linux, since they're a lot more flexible. Also note that sendfile on BSD has a completely different signature to sendfile on linux. The BSD one has the rather odd functionality of a built-in writev() before and after the sending of the file itself, with an extra struct argument to specify that, while on linux, if you want to write some other buffers, you're just expected to call writev yourself. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3333: wsgi_string() function
On Jan 7, 2011, at 6:51 AM, Victor Stinner wrote: I don't understand why you are attached to this horrible hack (bytes-in-unicode). It introduces more work and more confusing than using raw bytes unchanged. It doesn't work and so something has to be changed. It's gross but it does work. This has been discussed ad-nausium on web-sig over a period of years. I'd like to reiterate that it is only even a potential issue for the PATH_INFO/SCRIPT_NAME keys. Those two keys are required to have been urldecoded already, into byte-data in some encoding. For all the other keys (including the ones from os.environ), they are either *properly* decoded in 8859-1 or are just ascii (possibly still urlencoded, so the app needs to urldecode and decode into a string with the correct encoding). James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3333: wsgi_string() function
On Jan 6, 2011, at 8:16 PM, Glenn Linderman wrote: On 1/6/2011 3:50 PM, And Clover wrote: ISO-8859-1 is the encoding specified by the HTTP RFC Please could I have the reference to that specification? I only recall ASCII and UTF-8 in my readings of various things HTTP and HTML, for headers, and form data. Naturally data pages can have any encoding they please, as there are headers and meta tags to describe their encodings. Did you try google? http://www.google.com/search?http+rfc James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable
On Dec 17, 2010, at 7:55 PM, Victor Stinner wrote: Hi, In 2008, I proposed a patch to raise a Python exception on SIGSEVG signal. In some cases, it's possible to catch the exception, log it, and continue the execution. But because in some cases, the Python internal state is corrupted, the idea was rejected (see the issue #3999). Someone asked me to display the Python backtrace on SIGSEGV, instead of raising an exception. I implemented this idea in issue #8863. After 9 versions, I think that the patch is ready for inclusion. It catchs SIGSEGV, SIGFPE, SIGBUS and SIGILL signals, and also display the Python backtrace on fatal errors. Because some operating systems have their own fault handler (eg. Ubuntu with apport), Dave Malcolm asked me to add an option disable the Python handler. I think instead of calling abort() to kill the process, you should: - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing upon entering the handler), and then, - at the end of the handler, kill(getpid(), orig_signal) in order to abort the process. This has two advantages: 1) the process's exit code will actually show the correct signal, 2) it might let the OS fault handlers work properly as well -- I'm not sure. If it does, you may want to experiment with whether having or omitting SA_NODEFER gives a better backtrace (from the OS mechanism) in that case. If #2 actually works, you may not even need the env var, which would be nice. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Dec 16, 2010, at 3:14 AM, Senthil Kumaran wrote: Even HTTP 0.9 says that response SHOULD start with status line, but gives a suggestion that clients can tolerate bad server server behaviors when they don't send the status line and in that the case response is the body. http://www.w3.org/Protocols/HTTP/Response.html So, It cannot be associated with the behavior most webservers, back then and even more so now. Actually no. That document is describing almost-HTTP 1.0. Here is the actual document you were looking for: http://www.w3.org/Protocols/HTTP/AsImplemented.html HTTP 0.9 had no headers, no status line, nothing but GET $url crlf and a stream of data in response. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
On Dec 3, 2010, at 5:52 PM, Martin v. Löwis wrote: Am 03.12.2010 23:48, schrieb Éric Araujo: But I'm not interested at all in having it in distutils2. I want the Python build itself to use it, and alas, I can't because of the freeze. You can’t in 3.2, true. Neither can you in 3.1, or any previous version. If you implement it in distutils2, you have very good chances to get it for 3.3. Isn’t that a win? It is, unfortunately, a very weak promise. Until distutils2 is integrated in Python, I probably won't spend any time on it. It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: gc is implementation specific. CPython uses ref counting + cycle gc. A constraint on all implementations is that objects have a fixed, unique id during their lifetime. CPython uses the address as the id, so it cannot move objects. Other implementations do differently. Compacting gc requires an id to current address table or something. It's somewhat unfortuante that python has this constraint, instead of the looser: objects have a fixed id during their lifetime, which is much easier to implement, and practically as useful. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
On Dec 3, 2010, at 7:05 PM, Terry Reedy wrote: I left out that the id must be an int. It's somewhat unfortuante that python has this constraint, instead of the looser: objects have a fixed id during their lifetime, which is much easier to implement, and practically as useful. Given that the only different between 'fixed and unique' and 'fixed' is the uniqueness part, I do not understand 'practically as useful'. Duplicate ids (in the extreme, that same for all objects) hardly seem useful at all. Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this using identityHashCode()), you can still make an identity hashtable. You simply need to *also* check using is that the two objects really are the same one after finding the hash bin using id. It'd be a quality of implementation issue whether an implementation gives you the same value for every object. It should not, of course, if it wants programs to have reasonable performance. Same sort of thing as how __hash__ should not return 0 for everything. Besides identity-hashtables, the main other thing id gets used for is to have some identifier string in a printer (e.g. class Foo at 0x1234567890). There, it's generally good enough to use an id which is not guaranteed to be, but often is, unique. It works for Java... James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote: On 12/3/2010 7:46 PM, James Y Knight wrote: Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this using identityHashCode()), you can still make an identity I do not see the point of calling a (non-unique) hash value the identity My point was simply that a) it's an unfortunate constraint on potential GC implementations that objects need to have a fixed and unique id in Python, and b) that it's not actually necessary to have such a constraint (in the abstract sense of required; obviously it's a requirement upon Python *today*, due to existing code which depends upon that promise). Would you be happier if I had said it's unfortunate that Python has an id function instead of an identityHashValue function? I suppose that's what I really meant. Python the language would not have been harmed had it had from the start an identityHashValue() function instead of an id() function. In the CPython implementation, it may even have had the exact same behavior, but would've allowed other implementations more flexibility. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ICU
On Dec 1, 2010, at 11:45 PM, Alexander Belopolsky wrote: On Tue, Nov 30, 2010 at 3:13 PM, Antoine Pitrou solip...@pitrou.net wrote: Oh, about ICU: Actually, I remember you saying that locale should ideally be replaced with a wrapper around the ICU library. By that, I stand - however, I have given up the hope that this will happen anytime soon. Perhaps this could be made a GSOC topic. Incidentally, this may also address another Python's Achilles' heel: the timezone support. http://icu-project.org/download/icutzu.html Does ICU do anything regarding timezones that datetime + pytz doesn't already do? Wouldn't it make more sense to integrate the already-existing-and-pythonic pytz into Python than to make a new wrapper based on ICU? James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 final review
On Nov 29, 2010, at 8:58 AM, Nick Coghlan wrote: The http read only URLs didn't work (no diff returned, just svn: OPTIONS of 'http://svn.python.org/python/branches/pep-0384': 200 OK (http://svn.python.org)), That was the wrong url: you should've used http://svn.python.org/projects/python/branches/pep-0384 James___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] len(chr(i)) = 2?
On Nov 23, 2010, at 6:49 PM, Greg Ewing wrote: Maybe Python should have used UTF-8 as its internal unicode representation. Then people who were foolish enough to assume one character per string item would have their programs break rather soon under only light unicode testing. :-) You put a smiley, but, in all seriousness, I think that's actually the right thing to do if anyone writes a new programming language. It is clearly the right thing if you don't have to be concerned with backwards-compatibility: nobody really needs to be able to access the Nth codepoint in a string in constant time, so there's not really any point in storing a vector of codepoints. Instead, provide bidirectional iterators which can traverse the string by byte, codepoint, or by grapheme (that is: the set of combining characters + base character that go together, making up one thing which a human would think of as a character). James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] len(chr(i)) = 2?
On Nov 24, 2010, at 12:07 AM, Stephen J. Turnbull wrote: Or you can give user programs memory indicies, and enjoy the fun as the poor developers do things like pos += 1 which works fine on the ASCII data they have lying around, then wonder why they get Unicode errors when they take substrings. a) You seem to be hung up implementation details of emacs. But yes, positions should be stored as an byte offset into the utf8 string. NOT as number of codepoints since the beginning of the string. Probably you want it to be somewhat opaque, so that you actually have to specify whether you wanted to go to +1 byte, codepoint, or grapheme. b) Those poor developers are *already* screwed if they're using pos += 1 when pos is a codepoint index and they then take a substring based on that! They will get half a character when the string contains combining characters... Pretending that codepoints are a useful abstraction just makes poor developers get by without doing the correct thing (incrementing to the next grapheme boundary) for a little bit longer. But once you [the language implementor] are providing correct abstractions for grapheme movement, it's just as easy to also provide an abstraction for codepoint movement, and make your low-level implementation of the iterator object be a byte-offset into a UTF8 buffer. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] len(chr(i)) = 2?
On Nov 24, 2010, at 12:07 AM, Stephen J. Turnbull wrote: By the way, to send the ball back into your court, I have this feeling that the demand for UTF-8 is once again driven by native English speakers who are very shortly going to find themselves, and the data they are most familiar with, very much in the minority. Of course the market that benefits from UTF-8 compression will remain very large for the immediate future, but in the grand scheme of things, most of the world is going to prefer UTF-16 by a substantial margin. No, the demand for UTF-8 is because that's what much of the internet (and not coincidentally, unix) world has standardized on. The main pieces of software using UTF-16 (Windows, Java) started doing so before it became apparent that 16 bits wasn't enough to actually hold a unicode codepoint, so they were actually implementing UCS-2. In those days, UCS-2 was a fairly sensible choice. But, now, if your choices are UTF-8 or UTF-16, UTF-8 is clearly superior. Not because it's smaller -- it's pretty much a tossup -- but because it is an ASCII superset, and thus more easily compatible with other software. That also makes it most commonly used for internet communication. (So, there's a huge advantage for using it internally as well right there: no transcoding necessary for writing your HTML output). UTF-16 is incompatible with ASCII, and furthermore, it's still a variable-width encoding, with all the same issues that causes. As such, there's really very little to be said in favor of it. If you really want a fixed-width encoding, you have to go to UTF-32, which is excessively large. UTF-32 is a losing choice, simply because of the wasted memory usage. But that's all a side issue: even if you do choose UTF-16 as your underlying encoding, you *still* need to provide iterators that work by byte (only now bytes are 16-bits), by codepoint, and by grapheme. Of course, people who implement UTF-16 (such as python, java, and windows) often pretend they're still implementing UCS-2, and don't bother even providing their users with the necessary APIs to do things correctly. Which, you can often get away with...just so long as you don't mind that you sometimes end up splitting a string in the middle of a codepoint and causing a unicode error! James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] len(chr(i)) = 2?
Why don't ya'll just call them --unichar-width=16/32. That describes precisely what the options do, and doesn't invite any quibbling over definitions. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On Nov 17, 2010, at 9:19 AM, Nick Coghlan wrote: (and is a little trickier in the case of module level globals, since those can't be deprecated properly) People keep saying this, but there have already been examples shown of how to do it. I actually think that python should include a way to do so standard -- it's a reasonable enough desire, as shown by how many times in this thread the inability to do so has been mentioned. If the existing working 3rd-party mechanisms aren't good enough for python-dev standards, come up with a new way... James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On Nov 17, 2010, at 10:30 AM, Guido van Rossum wrote: On Wed, Nov 17, 2010 at 7:24 AM, James Y Knight f...@fuhm.net wrote: On Nov 17, 2010, at 9:19 AM, Nick Coghlan wrote: (and is a little trickier in the case of module level globals, since those can't be deprecated properly) People keep saying this, but there have already been examples shown of how to do it. I actually think that python should include a way to do so standard -- it's a reasonable enough desire, as shown by how many times in this thread the inability to do so has been mentioned. If the existing working 3rd-party mechanisms aren't good enough for python-dev standards, come up with a new way... That's quite the distraction from the current thread though. Start discussing it on python-ideas, or submit a code fix, or something in between. But the hackish way that some 3rd party frameworks use (replacing the module object with a class instance in sys.modules) is clearly not right for the standard library (I'll explain on python-ideas if you insist). I just don't want people to use the current lack as an excuse to simply remove module attributes without prior deprecation (or make a compatibility policy which recommends doing such a thing). I'll leave it up to the experts on this list (or python-ideas...) to determine how to implement a module-level deprecation in a way that isn't considered hackish. (Or, if there is no such way, there's also the alternative of simply never removing module-level names.) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On Nov 17, 2010, at 11:38 AM, Guido van Rossum wrote: Deprecation doesn't *require* logging a warning or raising an exception. You can also add a note to the docs, or if it is undocumented, just add a comment to the code. (Though if it is in widespread use despite being undocumented, a better way would be to document it first -- as immediately deprecated if necessary.) Deprecation is in the end a way to give people advance warning about future changes. The mechanism of the warning doesn't always have to be implemented by the interpreter/compiler/parser or whatever other tool. Well, that's certainly a possible policy. I'd suggest that adding notes to the docs after-the-fact is a singularly ineffective way of giving people advance warning of feature removal compared to having the interpreter/compiler/parser or whatever other tool warn you. And if that's to be python's policy, when it's possible to do better, I'm disappointed. (But won't respond further, my point is made.) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r86441 - python/branches/py3k/Lib/test/test_nntplib.py
On Nov 13, 2010, at 7:08 AM, Antoine Pitrou wrote: Funny, it shows that the NNTP SSL tests don't check the certificate, then. Unsurprising, given that you need 140 lines of pretty non-obvious python code to do so... James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On Nov 10, 2010, at 8:47 AM, Michael Foord wrote: How about making this explicit (either pep 8 or our developer docs): If a module or package defines __all__ that authoritatively defines the public interface. Modules with __all__ SHOULD still respect the naming conventions (leading underscore for private members) to avoid confusing users. Modules SHOULD NOT export private members in __all__. I don't like the idea of the authoritative definition of a public interface being defined based on __all__, because that provides users almost no warning that they're using a private API: the __all__ attribute doesn't do anything if you aren't using import *. If there was some proposal to make it so that accessing an attribute not in __all__ did prevent or somehow warn users that they're doing something dangerous, that'd be different, but there isn't such a proposal, and I don't even know what such a proposal would look like... On the other hand, if you make the primary mechanism to indicate privateness be a leading underscore, that's obvious to everyone. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
On Nov 8, 2010, at 6:08 PM, Lennart Regebro wrote: 2010/11/8 James Y Knight f...@fuhm.net: On Nov 8, 2010, at 4:42 AM, Lennart Regebro wrote: So it can be done, but the question is Why? To keep the batteries included? But they'll only be included in 2.7, which won't be used much, [...] If there was going to be an official python.org sanctioned Python 2.8 release, I'm not at all sure that'd be the case. Since there isn't going to be one, then yes, that's probably true. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
On Nov 8, 2010, at 4:42 AM, Lennart Regebro wrote: Except for making releases that start backporting Python 3 features and breaking backwards compatibility gradually (which may or may not be a good idea) I don't see the point. There isn't much to do when it comes to improving the language, and there is a moratorium anyway. Improvements in the standard library can be more easily done in external libraries anyway, and then you can release the improved libraries for everything from Python 2.4 and forwards if you like. So it can be done, but the question is Why? To keep the batteries included? James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-3 transition in Arch Linux
On Nov 6, 2010, at 9:41 AM, Martin v. Löwis wrote: So I don't recall a decision that there shouldn't be a python2 binary, The decision to make one would have to be an active decision, since Python has never installed one before. If there should be one, then the Python Makefile should make one by default. nor a decision that anything is done indefinitely (it may be that the decision was actually just about 3.1 - changing it again for 3.2 would require another decision, but certainly can't be ruled out categorically). When I said indefinite, I meant until some point in the future not yet determined, with an implied undertone of not anytime soon. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-3 transition in Arch Linux
On Nov 4, 2010, at 8:43 PM, Stephen J. Turnbull wrote: All of the Arch users I know expect Arch to occasionally do radical things because they're the right things to do in the long run. But the previous consensus (at least, as I, and presumably many other people understood it) was that python2 would remain the owner of the name /usr/bin/python for the indefinite future, and python3 would be invoked with /usr/bin/python3. Given that, it's not at all clear that Arch's actions are the right thing to do. IMO, moving away from that consensus should've been brought up on python-dev rather than just one distro just doing it all alone, causing incompatibilities and annoyance. If python-dev wants python3 to inherit the name /usr/bin/python, then python2 should've been installing a binary called /usr/bin/python2 for a couple years ahead of time, and recommending that everyone use that in their #! lines, so that the switch could've been done without breaking everything... Sure, and Guido should have exercised the Time Machine a little harder so that Python 3 never needed to happen. IOW, this is the price of success and wide distribution. Well, other programming languages seem to have avoided making sweeping bidirectionally-incompatible changes, despite being successful and widely distributed. But that's a whole other discussion. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/
On Nov 3, 2010, at 11:25 AM, Eric Smith wrote: On 11/3/10 10:53 AM, Eric Smith wrote: The problem is that there is no unittest.loader in 2.4, and unittest.loader.TestLoader is the name that the 2.7 pickle creates. We see this problem every time we try and move anything in the stdlib. And BTW: for me, this is the strongest reason not to break up modules into packages or otherwise reorganize the stdlib. This is the strongest reason why I recommend to everyone I know that they not use pickle for storage they'd like to keep working after upgrades [not just of stdlib, but other 3rd party software or their own software]. :) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
On Oct 27, 2010, at 10:22 PM, Kristján Valur Jónsson wrote: Hello all. So, python 2.7 is in bugfix only mode. ‘trunk’ is off limit. So, where does one make improvements to the distinguished, and still very much alive, 2.x series of Python? The answer would seem to be “one doesn’t”. But must it be that way? When Morris stopped producing the Oxford III model back in ’57 in favor of new developments, it didn’t spell the end for it. The plant was sold to India and the Hindustan Ambassador continues to be developed and produced to this day. It even has fuel injection. The Morris Motor Company isn’t around anymore. So, here is my suggestion: Let’s move the current ‘trunk’ into /branches/afterlife-27. Open it for submissions from people such as myself that use 2.7 on a regular basis and are willing to give it some extra love. Host it there without the usual stringent python quality assurance, buildbot support, release management and all that rigmarole. Open-source it, if you will. Svn.python.org already plays host to some other, less official, projects such as stackless, so why not this? The python community has already decided many times over that Python2 is dead and Python3 is the future. So if you want to continue maintaining Python2, that means you need to fork it. I think you'd be best off doing so on your own infrastructure: convincing the python developers to support such a thing is quite unlikely, and furthermore, completely unnecessary. Unlike the Oxford III, you don't need to be sold python2: it's open source, you can fork it without any official approval. So, just do it. I wish you best of luck, though: most unofficial forks die a lonely death. But, if enough people feel like you do, it could become successful. But I really doubt anyone else is going to want to use it any python2 afterlife without stringent quality assurance, multi-platform support releases, and other rigamarole. You'd have to set up all that stuff for yourself if you possibly hope to attract users. I can't think of any possible use for an unreliably maintained version of python2... James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for async read/write
On Oct 19, 2010, at 1:47 PM, exar...@twistedmatrix.com wrote: Adding more platform wrappers is always nice. Keep in mind that the quality of most (all?) aio_* implementations is spotty at best, though. On Linux, they will sometimes block (for example, if you fail to align buffers properly, or open a file without O_DIRECT, or if there are too many other aio operations active on the system at the time, etc). You're thinking of the linux-specific AIO calls. Those have the properties you're describing (which makes them pretty useless for most code too), but they're completely different from the aio_* functions. The POSIX aio_* calls don't do any of that. They aren't syscalls implemented in the kernel, they're implemented in glibc. They simply create a threadpool in your process to call the standard synchronous operations, and make it difficult to reliably get completion notification (completion notification takes place via Real-Time signals (SIGEV_SIGNAL), which can be dropped if linux runs out of space in its RT-signal-queue, and when that happens you get no indication that that has occurred. You can also do completion notification via calling a function on a thread (SIGEV_THREAD), but, for that, glibc will always spawns a brand new thread for each notification, which is quite slow.) Basically: you shouldn't ever use those APIs. Especially on linux, but probably everywhere else. So, in conclusion, I disagree that adding wrappers for these would be nice. It wouldn't. It would cause some people to think they would be useful things to call, and they would always be wrong. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for async read/write
On Oct 19, 2010, at 6:44 PM, Martin v. Löwis wrote: So, in conclusion, I disagree that adding wrappers for these would be nice. It wouldn't. It would cause some people to think they would be useful things to call, and they would always be wrong. We are all consenting adults. If people want to shoot themselves in their feet, we let them. For example, we have os.open, even though there is no garbage collection for file handles, and we have os._exit, even though it doesn't call finalizers. There's a difference. os._exit is useful. os.open is useful. aio_* are *not* useful. For anything. If there's anything you think you want to use them for, you're wrong. It either won't work properly or it will worse performing than the simpler alternatives. It would absolutely be a waste of time (of both the implementor of the wrapper and the poor users who stumble across them in documentation and try to use them) to bother adding wrappers to these functions for python. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
On Oct 8, 2010, at 5:24 PM, Gisle Aas wrote: On Oct 8, 2010, at 9:22 , Jeroen Ruigrok van der Werven wrote: +1 from me. I sincerely dislike the Perl-esque -m stuff. As a Perl/Python guy I have to object to calling the -m stuff Perl-esque. This is a very Pythonish thing. In the Perl world we never treat modules as scripts; they are separate concepts written separately and installed in separate locations. There is no feature of perl similar to the Pythonish -m stuff. Yes there is. -m and -M. E.g., the widely advertised perl -MCPAN -e install. It's not identical to python's -m, to be sure, but it's *similar*. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.normcase rationale?
On Oct 3, 2010, at 9:18 AM, Dan Villiom Podlaski Christiansen wrote: A simpler alternative would probably be the F_GETPATH fcntl. An example: That requires that you have permission to open the file (and to actually do so which might have other effects), while the File Manager's FSRef method does not. If Python adds a cross-platform function to do this canonicalization, users don't have to worry about how easy it is to invoke in pure-python... James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.normcase rationale?
On Sep 26, 2010, at 7:36 AM, Paul Moore wrote: On 26 September 2010 09:01, Paul Moore p.f.mo...@gmail.com wrote: On 25 September 2010 23:57, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Paul Moore wrote: Windows has (I believe) user definable filesystems, too, but the OS has get me the real filename style calls, Does it really, though? The suggestions I've seen for doing this involve abusing the short/long filename translation machinery, and I'm not sure they're guaranteed to return the actual case rather than something that happens to work. There's another call available. I've been too lazy to go and look it up, but I'll do so sometime today. Hmm, I can't find the one I was thinking of. GetLongFileName correctly sets the case of all but the final part, and FindFile can be used to find the last part, but that's not what I recall. GetFinalPathNameByHandle works, and is documented to do so, but (a) it works on an open file handle, so you need to open the file, and (b) it's Vista and later only... Were you thinking of SHGetFileInfo? http://stackoverflow.com/questions/74451/getting-actual-file-name-with-proper-casing-on-windows James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.normcase rationale?
On Sep 24, 2010, at 10:53 AM, Paul Moore wrote: On 24 September 2010 15:29, Guido van Rossum gu...@python.org wrote: I don't think we should try to reimplement what the filesystem does. I think we should just ask the filesystem (how exactly I haven't figured out yet but I expect it will be more OS-specific than filesystem-specific). It will have to be a new API -- normcase() at least is *intended* to return a case-flattened name on OSes where case-preserving filesystems are the default, and changing it to look at the filesystem would break too much code. For a new use case we need a new API. I dug into this once, and as far as I could tell, it's possible to get the information on Windows, but there's no way on Linux to ask the filesystem. From my researches, the standard interfaces a filesystem has to implement on Linux don't offer any means of asking this question. Of course, (a) I'm no Linux expert so what do I know, and (b) it may well be possible to come up with a good enough solution by ignoring pathologically annoying theoretical cases. I'm happy to provide Windows code if someone needs it. Paul An OSX code sketch is available here (summary: call FSPathMakeRef to get an FSRef from a path string, then FSRefMakePath to make it back into a path, which will then have the correct case). And note that it only works if the file actually exists. http://stackoverflow.com/questions/370186/how-do-i-find-the-correct-case-of-a-filename It would indeed be useful to have that be available in Python. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Aug 29, 2010, at 8:16 AM, Nick Coghlan wrote: However, since even platforms other than Windows aren't immune to version upgrades of the standard C runtime Aren't they? I don't know of any other platform that lets you have two versions of libc linked into a single address space. Linux has had incompatible libc updates in the past, but it was not possible to use both in one program. I believe BSD works the same way. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'hasattr' is broken by design
On Aug 24, 2010, at 10:26 AM, Benjamin Peterson wrote: 2010/8/24 P.J. Eby p...@telecommunity.com: At 03:37 PM 8/24/2010 +0200, Hrvoje Niksic wrote: a) a business case of throwing anything other than AttributeError from __getattr__ and friends is almost certainly a bug waiting to happen, and FYI, best practice for __getattr__ is generally to bail with an AttributeError as soon as you see double underscores in the name, unless you intend to support special attributes. Unless you're in an old-style class, you shouldn't get an double underscore methods in __getattr__ (or __getattribute__). If you do, it's a bug. Uh, did you see the message that was in response to? Maybe it should be a bug report? class Foo(object): ... def __getattr__(self, name): print ATTR:,name ... def __iter__(self): yield 1 ... print list(Foo()) ATTR: __length_hint__ [1] James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FHS compliance of Python installation
On Jun 26, 2010, at 4:35 PM, Matthias Klose wrote: On 26.06.2010 22:30, C. Titus Brown wrote: On Sat, Jun 26, 2010 at 10:25:28PM +0200, Matthias Klose wrote: On 25.06.2010 02:54, Ben Finney wrote: James Y Knightf...@fuhm.net writes: Really, python should store the .py files in /usr/share/python/, the .so files in /usr/lib/x86_64- linux-gnu/python2.5-debug/, and the .pyc files in /var/lib/python2.5- debug. But python doesn't work like that. +1 So who's going to draft the ???Filesystem Hierarchy Standard compliance??? PEP? :-) This has nothing to do with the FHS. The FHS talks about data, not code. Really? It has some guidelines here for object files, etc., at least as of 2004. http://www.pathname.com/fhs/pub/fhs-2.3.html A quick scan suggests /usr/lib is the right place to look: http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA agreed for object files, but http://www.pathname.com/fhs/pub/fhs-2.3.html#USRSHAREARCHITECTUREINDEPENDENTDATA explicitely states The /usr/share hierarchy is for all read-only architecture independent *data* files. I always figured the read-only architecture independent bit was the important part there, and code is data. Emacs's el files go into / usr/share/emacs, for instance. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: On 6/24/2010 8:23 PM, James Y Knight wrote: On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: If the package has .so files that aren't compatible with other version of python, then what is the motivation for placing that in a shared location (since it can't actually be shared) Because python looks for .so files in the same place it looks for the .py files of the same package. My suggestion was that a package that contains .so files should not be shared (e.g., the entire lxml package should be placed in a version-specific path). The motivation for this PEP was to simplify the installation python packages for distros; it was not to reduce the number of .py files on the disk. Placing .so files together does not simplify that install process in any way. You will still have to handle such packages in a special way. This is a good point, but I think still falls short of a solution. For a package like lxml, indeed you are correct. Since debian needs to build it once per version, it could just put the entire package (.py files and .so files) into a different per-python-version directory. However, then you have to also consider python packages made up of multiple distro packages -- like twisted or zope. Twisted includes some C extensions in the core package. But then there are other twisted modules (installed under a twisted.foo name) which do not include C extensions. If the base twisted package is installed under a version-specific directory, then all of the submodule packages need to also be installed under the same version-specific directory (and thus built for all versions). In the past, it has proven somewhat tricky to coordinate which directory the modules for package foo should be installed in, because you need to know whether *any* of the related packages includes a native .so file, not just the current package. The converse situation, where a base package did *not* get installed into a version-specific directory because it includes no native code, but a submodule *does* include a .so file, is even trickier. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? Specifically, it's the use case where we (Debian/Ubuntu) plan on installing all Python 3.x packages into /usr/lib/python3/dist-packages. As of PEP 3147, we can do that without collisions on the pyc files, but would still have to symlink for extension module .so files, because they are always named foo.so and Python 3.2's foo.so won't (modulo PEP 384) be compatible with Python 3.3's foo.so. If the package has .so files that aren't compatible with other version of python, then what is the motivation for placing that in a shared location (since it can't actually be shared) Because python looks for .so files in the same place it looks for the .py files of the same package. E.g., given a module like lxml, it contains the following files (among others): lxml/ lxml/__init__.py lxml/__init__.pyc lxml/builder.py lxml/builder.pyc lxml/etree.so And you can only put it in one place. Really, python should store the .py files in /usr/share/python/, the .so files in /usr/lib/x86_64- linux-gnu/python2.5-debug/, and the .pyc files in /var/lib/python2.5- debug. But python doesn't work like that. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use of cgi.escape can lead to XSS vulnerabilities
On Jun 22, 2010, at 5:14 PM, Craig Younkins wrote: I suggest rewording the documentation for the method making it more clear what it should and should not be used for. I would like to see the method changed to properly escape single-quotes, but if it is not changed, the documentation should explicitly say this method does not make input safe for inclusion in HTML. Well, it *does* make the input safe for inclusion in HTML...in a double-quoted attribute. The docs could make it clearer that you should always use double- quotes around your attribute values when using it, though, I agree. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes / unicode
On Jun 22, 2010, at 1:03 PM, Ian Bicking wrote: Similarly I'd expect (from experience) that a programmer using Python to want to take the same approach, sticking with unencoded data in nearly all situations. Yeah. This is a real issue I have with the direction Python3 went: it pushes you into decoding everything to unicode early, even when you don't care -- all you really wanted to do is pass it from one API to another, with some well-defined transformations, which don't actually depend on it having being decoded properly. (For example, extracting the path from the URL and attempting to open it as a file on the filesystem.) This means that Python3 programs can become *more* fragile in the face of random data you encounter out in the real world, rather than less fragile, which was the goal of the whole exercise. The surrogateescape method is a nice workaround for this, but I can't help thinking that it might've been better to just treat stuff as possibly-invalid-but-probably-utf8 byte-strings from input, through processing, to output. It seems kinda too late for that, though: next time someone designs a language, they can try that. :) James___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] email package status in 3.X
On Jun 21, 2010, at 4:29 PM, M.-A. Lemburg wrote: Here's a little known fact: by changing the Python2 default encoding to 'undefined' (yes, that's a real codec !), you can disable all automatic string coercion in Python2. I tried that once: half the stdlib stops working if you do (for example, the re module), so it's not particularly useful for checking if your own code is unicode-safe. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HEADS UP: Compilation risk with new GCC 4.5.0
On May 12, 2010, at 9:13 AM, Jesus Cea wrote: Short history: new GCC 4.5.0 (released a month ago), when compiling with - -O3, is adding MMX/SSE instructions that requires stack aligned to 16 byte. This is wrong, since x86 ABI only requires stack aligned to 4 bytes. If you compile EVERYTHING with GCC 4.5.0, you are safe (I guess!), but if your environment has mixed compiled code (for instance, the OS libraries), you can possibly core dump. If you have an old compiled Python and you update libs compiled with GCC 4.5.0, you can crash in the process. Psyco is showing the issue, but it is not the culprit. It only leaves - -correctly- the stack in not 16-byte alignment. But there are plenty of examples of crashes not related to python+psyco. Proposal: add -fno-tree-vectorize to compilation options for 2.7/3.2. Warm 2.3/2.4/2.5/2.6/3.0/3.1 users. Or warm users compiling with GCC 4.5.0. While assuming the stack is 16byte aligned is undeniably an ABI- violation in GCC, at this point, it's surely simpler to just go along: the new unofficial ABI for x86 is that the stack must always be left in 16-byte alignment... So, just change psyco to always use 16-byte-aligned stackframes. GCC has used 16byte-aligned stackframes for a heck of a long time now (so if the stack starts 16byte aligned on entry to a function it will stay that way on calls). So usually the only way people run into unaligned stacks is via hand-written assembly code or JIT compilers. I think you'll be a lot happier just modifying Psyco than making everyone else in the world change their compiler flags. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HEADS UP: Compilation risk with new GCC 4.5.0
On May 12, 2010, at 10:01 AM, Jesus Cea wrote: On 12/05/10 15:39, James Y Knight wrote: While assuming the stack is 16byte aligned is undeniably an ABI-violation in GCC, at this point, it's surely simpler to just go along: the new unofficial ABI for x86 is that the stack must always be left in 16-byte alignment... You can not rule out other software embedding python inside, or callbacks from foreign code. For instance, Berkeley DB library can do callbacks to Python code. So? When calling callback functions, the Berkeley DB library won't un-16byte-align the stack, will it? (Assuming it's been compiled with gcc in the last 10 years) Not all the universe is GCC based. For instance, Solaris system libraries are not compiled using GCC. The world is bigger that Linux/ GCC. If the Solaris compilers don't use 16byte-aligned stackframes, and GCC on Solaris/x86 also assumes 16byte-aligned stacks, I guess GCC on Solaris/x86 is pretty broken indeed. But for Linux/x86, stacks have been de-facto 16byte aligned for so long, you can *almost* excuse the ABI violation as unimportant. But anyways, psyco should keep the stackframes 16byte aligned regardless, for performance reasons: even when accessing datatypes for which unaligned access doesn't crash, it's faster when it's aligned. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: broken mailing list links in PEP(s?)
On May 5, 2010, at 8:22 AM, Barry Warsaw wrote: On May 5, 2010, at 7:09 AM, Oleg Broytman wrote: On Wed, May 05, 2010 at 11:43:45AM +0100, Michael Foord wrote: http://mail.python.org/pipermail/python-list/2000-July/108893.html which are broken Pipermail's links aren't stable AFAIU. The numbering is changing over time. They're only unstable if you regenerate the archives and the mbox file is old enough to have been a victim of a long-fixed delimiter bug. Which is true for python-dev. And of course if you're paying attention, you can fix the mbox file (quoting From etc) such that it generates the same numbers as it did the first time. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mailing List archive corruption?
On Jan 19, 2010, at 11:07 AM, Barry Warsaw wrote: On Jan 19, 2010, at 03:50 PM, Vinay Sajip wrote: When I look at the mailing list archive for python-dev, I see some odd stuff at the bottom of the page: http://mail.python.org/pipermail/python-dev/2010-January/thread.html#95232 Anyone know what's happened? WTF? I think the archives were recently regenerated, so there's probably a fubar there. CC'ing the postmasters. That happens if messages had unescaped From lines in the middle of them. No doubt, you've now broken every link anyone had ever made into the python-dev archives, because now all the article numbers are different. BTDT...unfortunately... Pipermail really is quite crappy, sigh. Anyhow, when I did that, I went back to a backup to get the original article numbers, and edited the mbox file escaping From lines or adding additional empty messages until the newly regenerated article numbers matched the originals. I'd highly recommend going through that painful process, since I suspect a *lot* of people have links to the python-dev archive. Hope you have a backup (or can find caches on google or archive.org or something). James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Improve open() to support reading file starting with an unicode BOM
On Jan 8, 2010, at 4:14 PM, Tres Seaver wrote: I understood this proposal as a general processing guideline, not something the io library should do (but, say, a text editor). FWIW, I'm personally in favor of using the UTF-8 signature. If people consider them crazy talk, that may be because UTF-8 can't possibly have a byte order - hence I call it a signature, not the BOM. As a signature, I don't consider it crazy at all. There is a long tradition of having magic bytes in files (executable files, Postscript, PDF, ... - see /etc/magic). Having a magic byte sequence for plain text to denote the encoding is useful and helps reducing moji-bake. This is the reason it's used on Windows: notepad would normally assume that text is in the ANSI code page, and for compatibility, it can't stop doing that. So the UTF-8 signature gives them an exit strategy. Agreed. Having that marker at the start of the file makes interop with other tools *much* easier. Putting the BOM at the beginning of UTF-8 text files is not a good idea, it makes interop much *worse* on a unix system, not better. Without the BOM, most commands do the right thing with UTF-8 text. E.g. to concatenate two files: $ cat file-1 file-2 file-3 With a BOM at the beginning of the file, it won't work right. Of course, you could modify cat (and every other stream processing command) to know how to consume and emit BOMs, and omit the extra one that would show up in the middle of the stream...but even that can't work; what about: $ (cat file-1; cat file-2) file-3. Should the shell now know that when you run multiple commands, it should eat the BOM emitted from the second command? Basically, using a BOM in a utf-8 file is just not a good idea: it completely ruins interop with every standard unix tool. This is not to say that Python shouldn't have a way to read a file with a UTF-8 BOM: it just shouldn't encourage you to *write* such files. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GIL required for _all_ Python calls?
On Jan 7, 2010, at 3:27 PM, Martin v. Löwis wrote: I've been wondering whether it's possible to release the GIL in the regex engine during matching. I don't think that's possible. The regex engine can also operate on objects whose representation may move in memory when you don't hold the GIL (e.g. buffers that get mutated). Even if they stay in place - if their contents changes, regex results may be confusing. It seems probably worthwhile to optimize for the common case of using the regexp engine on an immutable object of type str or bytes, and allow releasing the GIL in *that* case, even if you have to keep it for the general case. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Drop support for ones' complement machines?
On Dec 1, 2009, at 11:08 AM, Martin v. Löwis wrote: I'd rather prefer to explicitly list what CPython assumes about the outcome of specific operations. If this is just about , |, ^, and ~, then its fine with me. I'm not even interested in going this far: I still am: with your list of assumptions, it is unclear (to me, at least) what the consequences are. So I'd rather see an explicit list of consequences, instead of buying a pig in a poke. I think all that needs to be defined is that conversion from unsigned to signed, and (negative) signed to unsigned integers have 2's complement wrapping semantics, and does not affect the bit pattern in memory. Stating it that way makes it clearer that all you're assuming is the operation of the cast operators, and it seems to me that it implies the other requirements. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPI comments and ratings, *really*?
On Nov 12, 2009, at 4:11 PM, Ben Finney wrote: I think Jesse's point (or, if he's not willing to claim it, my point) is that, compared to the mandatory comment system, it makes much *more* sense to have a mandatory field for “URL to the BTS for this project”. One might look at the competition for inspiration. Looking at CPAN. There's no comments feature, but there is a CPAN RT bug-tracker which appears to be a way for users to submit comments/problems about packages in a way common to all packages in CPAN, but distinct from upstream's bug trackers/lists/etc. I'd assume that gets emailed to the listed maintainer of the package as well as being accessible to other users, although I don't really have any idea. e.g. http://search.cpan.org/~capttofu/DBD-mysql/lib/DBD/mysql.pm There might be something to be said for providing users a way to provide feedback that doesn't require making a accounts in a bazillion separate bugtrackers. *shrug* James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPI comments and ratings, *really*?
On Nov 12, 2009, at 5:23 PM, Masklinn wrote: On 12 Nov 2009, at 22:53 , James Y Knight wrote: On Nov 12, 2009, at 4:11 PM, Ben Finney wrote: I think Jesse's point (or, if he's not willing to claim it, my point) is that, compared to the mandatory comment system, it makes much *more* sense to have a mandatory field for “URL to the BTS for this project”. One might look at the competition for inspiration. Looking at CPAN. There's no comments feature There is, on search.cpan.org. See http://search.cpan.org/~petdance/ack/ for instance, the link leads to http://cpanratings.perl.org/ (a pretty interesting example of the distributed nature of cpan in fact). Ah, I see. I totally managed to miss that...I guess that's an interesting example of a bad web ui. :) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Nov 5, 2009, at 6:04 PM, geremy condra wrote: Perhaps my test is flawed in some way? Yes: you're testing the speed of something that makes absolutely no sense to do in a tight loop, so *who the heck cares how fast any way of doing it is*! Is this thread over yet? James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
On Nov 3, 2009, at 8:55 AM, sstein...@gmail.com wrote: And, as you point out, if 3.x doesn't start getting the crap beat out of it in the real world sooner rather than later, we may find ourselves, collectively with a stale 2.x, an under battle-tested 3.x, and nowhere to go. If that happens, it's not true that there's *nowhere* to go. A solution would be to discard 3.x as a failed experiment, take everything that is useful from it and port it to 2.x, and simply continue development from the last 2.x release. And from there, features can be deprecated and then removed a few releases later, as is the usual policy. Been there, done that, on a couple other projects. It's unfortunate when you have to throw out work you've done because it failed to gain traction over the thing you tried to replace, but sometimes that's life. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
On Nov 2, 2009, at 6:24 PM, sstein...@gmail.com wrote: +1 on 2.7 being the last of the 2.x series. Enough already! -1. (not that it matters) I, personally, haven't even written my first line of 3.x code, nor have I had any good reason to. Me neither. If I saw the actual end of the line at 2.7, I would actually start looking for 3.x versions of my favorite tools and would be much more inclined to help push them along ASAP. I'd probably keep using 2.7 to be able to keep using those tools, instead. Right now, so much that I use on a daily basis doesn't even have a 3.x roadmap, much less any sort of working implementation, that I don't see switching to 3.x ever unless the 2.x line ends, and soon! I don't see switching to 3.x anytime soon either. But what's the rush? 2.x seems to be a fine edition of Python, why not let it keep going to 2.8 and beyond? Then you wouldn't have to switch to 3.x at all, and that'd save you a ton of work. (and save all the people you will have to convince to make a 3.x roadmap and do the port a ton of work too!) It really sounds like you're saying that switching to 3.x isn't worth the cost to you, but you want to force people (including yourself) to do so anyways, because ...? James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
On Nov 3, 2009, at 12:06 AM, Guido van Rossum wrote: Though I imagine what it really needs is a quirks mode parser that is compatible with the HTML dialect accepted by, say, IE6. Maybe a summer of code project? Already exists: html5lib. http://code.google.com/p/html5lib/ Or if you want a faster (yet I think less exact) HTML parser, libxml2's HTML parser, via lxml: http://codespeak.net/lxml/parsing.html#parsing-html James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a set without removing it
On Oct 25, 2009, at 2:50 AM, Terry Reedy wrote: Alex Martelli wrote: Next(s) would seem good... That does not work. It has to be next(iter(s)), and that has been tried and eliminated because it is significantly slower. But who cares about the speed of getting an arbitrary element from a set? How can it *possibly* be a problem in a real program? If you want to optimize python, this operation is certainly not the right place to start... James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug 7183 and Python 2.6.4
On Oct 22, 2009, at 11:04 AM, Barry Warsaw wrote: On Oct 22, 2009, at 10:47 AM, Benjamin Peterson wrote: 2009/10/22 Barry Warsaw ba...@python.org: So does anybody else think bug 7183 should be a release blocker for 2.6.4 final, or is even a legitimate but that we need to fix? I think it cannot hold up a release with out a reproducible code snippet. It may not be reproducible in standard Python, see David's follow up to the issue. If that holds true and we can't reproduce it, I agree we should not hold up the release for this. class Foo(property): ... __slots__=[] ... x=Foo() x.__doc__ = asdf Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'Foo' object attribute '__doc__' is read-only You can't add arbitrary attributes to instances, since some instances don't have the slot to put them in. Is that an equivalent demonstration to that which boost runs into? (except, it's using a C type not a python type). James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug 7183 and Python 2.6.4
On Oct 22, 2009, at 3:53 PM, Robert Collins wrote: On Thu, 2009-10-22 at 13:16 -0400, Tres Seaver wrote: ... That being said, I can't this bug as a release blocker: people can either upgrade to super-current Boost, or stick with 2.6.2 until they can. Thats the challenge Ubuntu faces: https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688 We've just announced our Karmic RC, boost 1.40 isn't released, and python 2.6.3 doesn't work with a released boost :( If I were running a Linux distro, I'd revert the patch in 2.6.3. And if I were running a Python release process, I'd revert that patch for python 2.6.4, and reopen the bug that it fixed, so a less-breaky patch can be made. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Package install failures in 2.6.3
On Oct 5, 2009, at 2:21 PM, Brett Cannon wrote: I should also mention this bug was not unknown. I discovered it after Distribute 0.6 was released as I always run cutting edge interpreters. Never bothered to report it until Distribute 0.6.1 was released which Tarek fixed in less than a week. I never bothered to report it for setuptools as I know it isn't maintained. It's probably in our best interest to just get people over to Distribute, let it continue to hijack setuptools, and slowly let that name fade out if it is going to continue to be unmaintained. I have to admit I find it really disheartening that we are letting an unmaintained project dictate how we fix a bug. I really hope this is a one-time deal and from this point forward we all move the community towards Distribute so we never feel pressured like this again. Even though the bug was noticed, nobody thought that, just perhaps, breaking other software in a minor point release might be a bad idea, no matter whether it was updated in less-than-a-week, or mostly- unmaintained? Once you have an API that you encourage people to subclass, *of course* it dictates how you can fix a bug. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On Oct 2, 2009, at 2:56 PM, Raymond Hettinger wrote:
Do the users get any say in this?
I imagine that some people are heavily invested in %-formatting.
Because there has been limited uptake on {}-formatting (afaict),
we still have limited experience with knowing that it is actually
better, less error-prone, easier to learn/rember, etc. Outside
a handful of people on this list, I have yet to see anyone adopt
it as the preferred syntax.
Well, I actually think it was a pretty bad idea to introduce {}
formatting, because %-formatting is well-known in many other
languages, and $-formatting is used by basically all the rest. So the
introduction of {}-formatting has always seemed silly to me, and I
wish it had not happened.
HOWEVER, much worse than having a new, different, and strange
formatting convention is having *multiple* formatting conventions
arbitrarily used in different places within the language, with no
rhyme or reason.
So, given that brace-formatting was added, and that it's been declared
the way forward, I'd *greatly* prefer it taking over everywhere in
python, instead of having to use a mixture.
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On Oct 1, 2009, at 9:11 AM, Paul Moore wrote:
This seems to me to be almost the same as the previous suggestion of
having a string subclass:
class BraceFormatter(str):
def __mod__(self, other):
# Needs more magic here to cope with dict argument
return self.format(*other)
__ = BraceFormatter
logger.debug(__(The {0} is {1}), answer, 42)
I'd rather make that:
class BraceFormatter:
def __init__(self, s):
self.s = s
def __mod__(self, other):
# Needs more magic here to cope with dict argument
return s.format(*other)
__ = BraceFormatter
That is, *not* a string subclass. Then if someone attempts to mangle
it, or use it for anything but %, it fails loudly.
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On Sep 30, 2009, at 1:01 PM, Antoine Pitrou wrote:
Why not allow logging.Formatter to take a callable, which would in
turn call the
callable with keyword arguments?
Therefore, you could write:
logging.Formatter({asctime} - {name} - {level} - {msg}.format)
and then:
logging.critical(name=Python, msg=Buildbots are down)
All this without having to learn about a separate compatibility
wrapper object.
It's a nice idea -- but I think it's better for the wrapper (whatever
form it takes) to support __mod__ so that logging.Formatter (and
everything else) doesn't need to be modified to be able to know about
how to use both callables and %ables.
Is it possible for a C function like str.format to have other methods
defined on its function type?
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On Oct 1, 2009, at 5:54 PM, Nick Coghlan wrote: I believe classes like fmt_braces/fmt_dollar/fmt_percent will be part of a solution, but they aren't a complete solution on their own. (Naming the three major string formatting techniques by the key symbols involved is a really good idea though) 1. It's easy to inadvertently convert them back to normal strings. If a formatting API even calls str on the format string then we end up with a problem (and switching to containment instead of inheritance doesn't really help, since all objects implement __str__). Using containment instead of inheritance makes sure none of the *other* operations people do on strings will appear to work, at least (substring, contains, etc). I bet explicitly calling str() on a format string is even more rare than attempting to do those things. 2. They don't help with APIs that expect a percent-formatted string and do more with it than just pass it to str.__mod__ (e.g. inspecting it for particular values such as '%(asctime)s') True, but I don't think there's many such cases in the first place, and such places can be fixed to not do that as they're found. Until they are fixed, fmt_braces will loudly fail when used with that API (assuming fmt_braces is not a subclass of str). James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On Oct 1, 2009, at 6:19 PM, Steven Bethard wrote:
I see how this could allow a user to supply a {}-format string to an
API that accepts only %-format strings. But I still don't see the
transition strategy for the API itself. That is, how does the %-format
API use this to eventually switch to {}-format strings? Could someone
please lay it out for me, step by step, showing what happens in each
version?
Here's what I said in my first message, suggesting this change.
Copypasted below:
I wrote:
1) introduce the above feature, and recommend in docs that people
only ever use new-style format strings, wrapping the string in
newstyle_formatstr() when necessary for passing to an API which uses
% internally.
2) A long time later...deprecate str.__mod__; don't deprecate
newstyle_formatstr.__mod__.
3) A while after that (maybe), remove str.__mod__ and replace all
calls in Python to % (used as a formatting operator) with .format()
so that the default is to use newstyle format strings for all APIs
from then on.
So do (1) in 3.2. Then do (2) in 3.4, and (3) in 3.6. I skipped two
versions each time because of how widely this API is used, and the
likely pain that doing the transition quickly would cause. But I guess
you *could* do it in one version each step.
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
On Sep 30, 2009, at 10:34 AM, Steven D'Aprano wrote:
E.g.
x = newstyle_formatstr({} {} {})
x % (1,2,3) == x.format(1,2,3) == 1 2 3
Moving along, let's suppose the newstyle_formatstr is introduced.
What's
the intention then? Do we go through the std lib and replace every
call
to (say)
somestring % args
with
newstyle_formatstr(somestring) % args
instead? That seems terribly pointless to me
Indeed, that *would* be terribly pointless! Actually, more than
pointless, it would be broken, as you've changed the API from taking
oldstyle format strings to newstyle format strings.
That is not the suggestion. The intention is to change /nearly
nothing/ in the std lib, and yet allow users to use newstyle string
substitution with every API.
Many Python APIs (e.g. logging) currently take a %-type formatting
string. It cannot simply be changed to take a {}-type format string,
because of backwards compatibility concerns. Either a new API can be
added to every one of those functions/classes, or, a single API can be
added to inform those places to use newstyle format strings.
This could allow for a controlled switch towards the new format
string format, with a long deprecation period for users to migrate:
1) introduce the above feature, and recommend in docs that people
only ever use new-style format strings, wrapping the string in
newstyle_formatstr() when necessary for passing to an API which uses
% internally.
And how are people supposed to know what the API uses internally?
It's documented, (as it already must be, today!).
Personally, I think your chances of getting people to write:
logging.Formatter(newstyle_formatstr(%(asctime)s - %(name)s - %
(level)s - %(msg)s))
instead of
logging.Formatter(%(asctime)s - %(name)s - %(level)s - %(msg)s)
That's not my proposal.
The user could write either:
logging.Formatter(%(asctime)s - %(name)s - %(level)s - %(msg)s)
(as always -- that can't be changed without a long deprecation
period), or:
logging.Formatter(newstyle_formatstr({asctime} - {name} - {level} -
{msg})
This despite the fact that logging has not been changed to use {}-
style formatting internally. It should continue to call self._fmt %
record.__dict__ for backward compatibility.
That's not to say that this proposal would allow no work to be done to
check the stdlib for issues. The Logging module presents one: it
checks if the format string contains %{asctime} to see if it should
bother to calculate the time. That of course would need to be changed.
Best would be to stick an instance which lazily generates its string
representation into the dict. The other APIs mentioned on this thread
(BaseHTTPServer, email.generator) will work immediately without
changes, however.
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transitioning from % to {} formatting
I'm resending a message I sent in June, since it seems the same thread
has come up again, and I don't believe anybody actually responded
(positively or negatively) to the suggestion back then.
http://mail.python.org/pipermail/python-dev/2009-June/090176.html
On Jun 21, 2009, at 5:40 PM, Eric Smith wrote:
I've basically come to accept that %-formatting can never go away,
unfortunately. There are too many places where %-formatting is used,
for example in logging Formatters. %-formatting either has to exist
or it has to be emulated.
It'd possibly be helpful if there were builtin objects which forced
the format style to be either newstyle or oldstyle, independent of
whether % or format was called on it.
E.g.
x = newstyle_formatstr({} {} {})
x % (1,2,3) == x.format(1,2,3) == 1 2 3
and perhaps, for symmetry:
y = oldstyle_formatstr(%s %s %s)
y.format(1,2,3) == x % (1,2,3) == 1 2 3
This allows the format string style decision is to be made external
to the API actually calling the formatting function. Thus, it need not
matter as much whether the logging API uses % or .format() internally
-- that only affects the *default* behavior when a bare string is
passed in.
This could allow for a controlled switch towards the new format string
format, with a long deprecation period for users to migrate:
1) introduce the above feature, and recommend in docs that people only
ever use new-style format strings, wrapping the string in
newstyle_formatstr() when necessary for passing to an API which uses %
internally.
2) A long time later...deprecate str.__mod__; don't deprecate
newstyle_formatstr.__mod__.
3) A while after that (maybe), remove str.__mod__ and replace all
calls in Python to % (used as a formatting operator) with .format() so
that the default is to use newstyle format strings for all APIs from
then on.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] please consider changing --enable-unicode default to ucs4
On Sep 28, 2009, at 4:25 AM, M.-A. Lemburg wrote: Distributions should really not be put in charge of upstream coding design decisions. I don't think you can blame distros for this one From PEP 0261: It is also proposed that one day --enable-unicode will just default to the width of your platforms wchar_t. On linux, wchar_t is 4 bytes. If there's a consensus amongst python upstream that all the distros should be shipping Python with UCS2 unicode strings, you should reach out to them and say this, in a rather more clear fashion. Currently, most signs point towards UCS4 builds as being the better option. Or, one might reasonably wonder why UCS-4 is an option at all, if nobody should enable it. People building their own Python version will usually also build their own extensions, so I don't really believe that the above scenario is very common. I'd just like to note that I've run into this trap multiple times. I built a custom python, and expected it to work with all the existing, installed, extensions (same major version as the system install, just patched). And then had to build it again with UCS4, for it to actually work. Of course building twice isn't the end of the world, and I'm certainly used to having to twiddle build options on software to get it working, but, this *does* happen, and *is* a tiny bit irritating. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] IO module precisions and exception hierarchy
On Sep 27, 2009, at 4:20 AM, Pascal Chambon wrote: Thus, at the moment IOErrors rather have the semantic of particular case of OSError, and it's kind of confusing to have them remain in their own separate tree... Furthermore, OSErrors are often used where IOErrors would perfectly fit, eg. in low level I/O functions of the OS module. Since OSErrors and IOErrors are slightly mixed up when we deal with IO operations, maybe the easiest way to make it clearer would be to push to their limits already existing designs. How about just making IOError = OSError, and introducing your proposed subclasses? Does the usage of IOError vs OSError have *any* useful semantics? James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3144 review.
On Sep 27, 2009, at 3:18 PM, Peter Moody wrote:
administrators) would use it, but it's doable. what you're claiming is
that my use case is invalid.
that's what I claim is broken.
He's claiming your solution to address your use case is confusing, not
that the use case is invalid.
I'm not going to make ipaddr
less useful (strictly removing functionality), more bulky and
confusing (adding more confusingly named classes and methods) or
otherwise break the library in a vain attempt to have it included in
the stdlib.
If I understand correctly, the proposal for addressing the issue is to
make two rather simple changes:
1) if strict=False, mask off the bits described by the netmask when
creating an IPNetwork, such that the host bits are always 0.
2) add a single new function:
def parse_net_and_addr(s):
return (IPNetwork(s), IPAddress(s.split('/')[0]))
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fuzziness in io module specs
On Sep 18, 2009, at 3:55 PM, MRAB wrote: I think that this should be an invariant: 0 = file pointer = file size so the file pointer might sometimes have to be moved. As for the question of whether 'truncate' should be able to lengthen a file, the method name suggests no; if the method name were 'resize', for example, then maybe yes, zeroing the new bytes for security. Why are you just making things up? There is a *vast* amount of precedent for how file operations should work. Python should follow that precedent and do like POSIX unless there's a compelling reason not to. Quoting: If fildes refers to a regular file, the ftruncate() function shall cause the size of the file to be truncated to length. If the size of the file previously exceeded length, the extra data shall no longer be available to reads on the file. If the file previously was smaller than this size, ftruncate() shall either increase the size of the file or fail. XSI-conformant systems shall increase the size of the file. If the file size is increased, the extended area shall appear as if it were zero-filled. The value of the seek pointer shall not be modified by a call to ftruncate(). James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] POSIX [Fuzziness in io module specs]
On Sep 18, 2009, at 8:58 PM, Antoine Pitrou wrote: I'm not sure that's true. Various Unix/Linux man pages are readily available on the Internet, but they regard specific implementations, which often depart from the spec in one way or another. POSIX specs themselves don't seem to be easily reachable; you might even have to pay for them. The POSIX specs are quite easily accessible, without payment. I got my quote by doing: man 3p ftruncate I had previously done: apt-get install manpages-posix-dev to install the posix manpages. That package contains the POSIX standard as of 2003. Which is good enough for most uses. It seems to be available here, if you don't have a debian system: http://www.kernel.org/pub/linux/docs/man-pages/man-pages-posix/ There's also a webpage, containing the official POSIX 2008 standard: http://www.opengroup.org/onlinepubs/9699919799/ And to navigate to ftruncate from there, click System Interfaces in the left pane, System Interfaces in the bottom pane, and then ftruncate in the bottom pane. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decorator syntax
On Sep 2, 2009, at 6:15 AM, Rob Cliffe wrote: So - the syntax restriction seems not only inconsistent, but pointless; it doesn't forbid anything, but merely means we have to do it in a slightly convoluted (unPythonesque) way. So please, Guido, will you reconsider? Indeed, it's a silly inconsistent restriction. When it was first added I too suggested that any expression be allowed after the @, rather than having a uniquely special restricted syntax. I argued from consistency of grammar standpoint. But Guido was not persuaded. Good luck to you. :) Here's some of the more relevant messages from the thread back when the @decorator feature was first introduced: http://mail.python.org/pipermail/python-dev/2004-August/046654.html http://mail.python.org/pipermail/python-dev/2004-August/046659.html http://mail.python.org/pipermail/python-dev/2004-August/046675.html http://mail.python.org/pipermail/python-dev/2004-August/046711.html http://mail.python.org/pipermail/python-dev/2004-August/046741.html http://mail.python.org/pipermail/python-dev/2004-August/046753.html http://mail.python.org/pipermail/python-dev/2004-August/046818.html James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] command line attachable debugger
On Jul 24, 2009, at 1:31 AM, Edward Peschko wrote: all, I'I was wondering if there was a command line python debugger that was able to attach to an existing process. I'd very much like to be able to debug over a ssh session using screen. Ed (ps - and yes, I know about winpdb, etc... that is not exactly what I'm looking for..) Winpdb is *exactly* what you asked for, so if it's not what you're looking for you'll need to be more specific about what you want that it doesn't do... James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
