Re: [Python-Dev] requirements for moving __import__ over to importlib?
On Sat, Feb 11, 2012 at 11:23 AM, PJ Eby p...@telecommunity.com wrote: What's the downside in that case? You're trying to import something that just changed in the last fraction of a second... why? I don't know if it's normal in the Python world, but these sorts of race conditions occur most annoyingly when a single process changes a file, then attempts to import it. If you open a file, write to it, explicitly close it, and then load it, you would expect to read back what you wrote, not the version that was there previously. Chris Angelico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.2): Issue #14123: Explicitly mention that old style % string formatting has caveats
On Mon, Feb 27, 2012 at 5:54 AM, Antoine Pitrou solip...@pitrou.net wrote: It would be nice to call it something else than printf-style formatting. While it is certainly modelled on printf(), knowledge of C or printf is not required to understand %-style formatting, nor even to appreciate it. -1. Calling it printf-style ties it in with its origins just as the term regex does for the 're' module. There are printf-derived features in quite a few high level languages; they may differ somewhat (Pike's sprintf() can do columnar displays; PHP's vsprintf takes an array, not some weird and mythical varargs token), but in their basics they will be similar. The name is worth keeping. Chris Angelico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
On Thu, Mar 1, 2012 at 8:08 AM, Paul Moore p.f.mo...@gmail.com wrote: It would (apparently) help Victor to fix issues in his pysandbox project. I don't know if a secure Python sandbox is an important enough concept to warrant core changes to make it possible. If a secure Python sandbox had been available last year, we would probably be still using Python at work for end-user scripting, instead of having had to switch to Javascript. At least, that would be the case if this sandbox is what I think it is (we embed a scripting language in our C++ main engine, and allow end users to customize and partly drive our code). But features enabling that needn't be core; I wouldn't object to having to get some third-party add-ons to make it all work. Chris Angelico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Playing with a new theme for the docs
On Fri, Mar 23, 2012 at 6:50 AM, Glenn Linderman v+pyt...@g.nevcal.com wrote: 3. Make the sidebar separately scrollable, so that it stays visible when scrolling down in the text. This would make it much easier to jump from section to section, if the TOC didn't get lost in the process. -1. The downside of separate scrolling is that you lose the ability to scroll-wheel the main docs if your mouse is over the TOC. I'd rather it stay as a single page, so it doesn't matter where my pointer is when I scroll. But I realise I'm bikeshedding this a bit. Bottom line: I'll use Python, and docs.python.org, regardless of the design and layout... so I'll let the expert(s) decide this one. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] .{git,bzr}ignore in cpython HG repo
On Mon, Apr 2, 2012 at 9:50 PM, Antoine Pitrou solip...@pitrou.net wrote: That said, these files will always be outdated, so we might as well remove them so that at least git / bzr users don't get confused. Apologies for what may be a stupid suggestion, but is it possible to write a script that generates .gitignore and .bzrignore from .hgignore? That ought to solve the problem - take the former two out of the repository, and everyone who wants to use git or bzr can simply generate them on requirement. Chris Angelico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative to embedding policy in PEP 418
On Wed, Apr 4, 2012 at 9:04 PM, Victor Stinner victor.stin...@gmail.com wrote: 2012/4/4 Antoine Pitrou solip...@pitrou.net: On Wed, 4 Apr 2012 02:02:12 +0200 Victor Stinner victor.stin...@gmail.com wrote: Lennart Regebro wrote: Well, get_clock(monotonic=True, highres=True) would be a vast improvement over get_clock(MONOTONIC|HIRES). I don't like this keyword API because you have to use a magically marker (True). Why True? What happens if I call get_clock(monotonic=False) or get_clock(monotonic=yes)? Since when are booleans magical? Has this thread gone totally insane? It depends if the option supports other values. But as I understood, the keyword value must always be True. If I were looking at that in documentation, my automatic guess would be that the only thing that matters is whether the argument compares-as-true or not. So get_clock(monotonic=yes) would be the same as =True, and =False wouldn't be. And get_clock(monotonic=No, you idiot, I want one that ISN'T) would... be stupid. But it'd still function :) Chris Angelico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418 glossary
On Wed, Apr 11, 2012 at 4:49 PM, Jim Jewett jimjjew...@gmail.com wrote: Clock: An instrument for measuring time. Different clocks have different characteristics; for example, a clock with nanonsecond precision Small typo. Otherwise, excellent reference document - thank you! Well worth gathering all those terms. ChrisA There's a glory for you! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Suggested addition to PEP 8 for context managers
On Mon, Apr 16, 2012 at 3:13 AM, Raymond Hettinger raymond.hettin...@gmail.com wrote: Instead, the context manager implements a different behavior. It would have been better if that behavior had been given a name: db = sqlite3.connect(filename) with auto_commit_or_rollback(db): # do a transaction I agree that it wants a name. If explicitness is the goal, would this be more suitable? db = sqlite3.connect(filename) with db.begin_transaction() as trans: # do a transaction This way, if a database engine supports multiple simultaneous transactions, the same syntax can be used. Chris Angelico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 418: Add monotonic time, performance counter and process time functions
On Tue, Apr 17, 2012 at 2:48 PM, Cameron Simpson c...@zip.com.au wrote: On 16Apr2012 01:25, Victor Stinner victor.stin...@gmail.com wrote: | I suppose that most people don't care that resolution and | precision are different things. If we're using the same definitions we discussed offline, where - resolution is the units the clock call (underneath) works in (for example, nanoseconds) - precision is the effective precision of the results, for example milliseconds I'd say people would care if they knew, and mostly care about precision. Meaning that resolution is a matter of format and API, not of clock. If you take a C clock API that returns a value in nanoseconds and return it as a Python float, you've changed the resolution. I don't think resolution matters much, beyond that (for example) nanosecond resolution allows a clock to be subsequently upgraded as far as nanosecond precision without breaking existing code, even if currently it's only providing microsecond precision. But it makes just as much sense for your resolution to be 2**64ths-of-a-second or quarters-of-the-internal-CPU-clock-speed as it does for nanoseconds. As long as it's some fraction of the SI second, every different resolution is just a fixed ratio away from every other one. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Suggested addition to PEP 8 for context managers
On Thu, Apr 19, 2012 at 12:47 AM, Guido van Rossum gu...@python.org wrote: I don't believe PEP 8 requires whitespace around all binary operators. Where do you read that? Quoting from http://www.python.org/dev/peps/pep-0008/#other-recommendations (with elision): Use spaces around arithmetic operators: No: i=i+1 submitted +=1 x = x*2 - 1 hypot2 = x*x + y*y c = (a+b) * (a-b) End quote. In my code, whether Python or any other language, I tend to follow the principle that whitespace is completely optional in these expressions, but if spaces surround any operator, they should (generally) also surround all operators of lower precedence in the same expression. So I would quite happily accept all of the expressions above (except 'submitted', which is inconsistent), but would prefer not to see something like: c=(a + b)*(a - b) which is also forbidden by PEP 8. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Suggested addition to PEP 8 for context managers
On Thu, Apr 19, 2012 at 1:06 PM, Eli Bendersky eli...@gmail.com wrote: (quoting GvR) Let's change this to something more reasonable, e.g. If operators with different priorities are used, consider adding whitespace around the operators with the lowest priority(ies). This is very much to taste, however, never use more than one space, and always have the same amount of whitespace on both sides of a binary operator. +1, a very welcome change to a piece of PEP8 I've always felt uncomfortable with. Tiny nitpick: I'd just replace the comma following however with a period or semicolon. Following or preceding? Either works, but there's a slight shift of meaning depending on which punctuation gets the upgrade. What was the original intent of the paragraph? Chris Angelico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Suggested addition to PEP 8 for context managers
On Fri, Apr 20, 2012 at 1:00 AM, Éric Araujo mer...@netwok.org wrote: Hi, +- If operators with different priorities are used, consider adding + whitespace around the operators with the lowest priority(ies). This + is very much to taste; however, never use more than one space, and + always have the same amount of whitespace on both sides of a binary + operator. Does “this is very much to taste” means that it’s a style judgment where each team or individual may make different choices? I’m not a native speaker and I’m not sure about the intended meaning. Yes. It's like writing instructions for how to make a cup of tea. You might want to put in one spoon of sugar, someone else might prefer two. On the instructions, you simply write Add sugar to taste, and that's what this is drawing analogy with. The PEP, as then written, would happily accept all of these: x = y*2+z*3 x = y*2 + z*3 x = y * 2 + z * 3 but would advise against: x =y*2 + z* 3 ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] netiquette on py-dev
On Wed, Apr 25, 2012 at 1:44 PM, Stephen J. Turnbull step...@xemacs.org wrote: I don't know of any webmail implementations that provide reply-to-list, so a lot of us end up using reply-to-all. Cleaning up the headers requires at least deleting the To (which is where the author ends up), and perhaps moving the list from Cc to To (to make it pretty, I don't think a nonempty To is actually required by the RFC). Especially on a mobile device this is a PITA. I go the other way: hit Reply, and then replace the author's address with the list's. I'd much rather have a Reply List though. Unfortunately no decent webmail seems to have it, and I'm still looking for a decent non-web-mail client too. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CRLF line endings
On Thu, May 3, 2012 at 10:50 PM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Although these are text files, the CRLF line endings are needed because otherwise, the files won't be presented correctly on Windows, e.g. in Notepad. Not all Windows editors choke on \n line endings; when I'm on Windows and run into one, I open it in Wordpad (or, if I have one, a dedicated programming editor like SciTE or the Open Watcom editor). AFAIK only Notepad (of standard Windows utilities) has trouble. Not sure if that makes a difference or not. Chris Angelico ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CRLF line endings
On Fri, May 4, 2012 at 1:28 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote: It's only really an issue for new / inexperienced users, I agree. Since these files are installed only on Windows systems, there's no reason for them not to have the native line endings. Then sure, doesn't make a lot of difference that it's only Notepad. Somebody needs to rewrite that ancient editor and give Windows a better default... ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Debian wheezy, amd64: make not finding files for bz2 and other packages
On Sat, May 5, 2012 at 4:07 AM, Edward C. Jones edcjo...@comcast.net wrote: /usr/include/bzlib.h /usr/lib/x86_64-linux-gnu/libbz2.a /usr/lib/x86_64-linux-gnu/libbz2.so /lib/x86_64-linux-gnu/libbz2.so.1 /lib/x86_64-linux-gnu/libbz2.so.1.0 /lib/x86_64-linux-gnu/libbz2.so.1.0.4 I have an Ubuntu Maverick 64-bit system, not identical but hopefully similar to your Debian. I have /usr/include/bzlib.h, but the others are all one directory level higher - /usr/lib/libbz2.a, /lib/libbz2.so.1.0.4, etc. Does your /etc/ld.so.conf.d mention the appropriate directories? ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 405 (pyvenv) and system Python upgrades
On Sat, May 5, 2012 at 6:49 AM, Carl Meyer c...@oddbird.net wrote: 2) In addition to the above, introduce a versioning marker in the standard library (is there one already?) and have some code somewhere (insert hand-waving here) check sys.version_info against the stdlib version, and fail fast with an unambiguous error if there is a mismatch. This makes the failure more explicit, but at the significant cost of making it more common: at every mismatch, not just in the apparently-rare case of a breaking change. Variant: Could the versioning marker give a minimum and/or maximum? It'd then only cause the explicit failure in the actual case of a breaking change, and the rest of the time it could happily use any X.Y.* release. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] docs.python.org pointing to Python 3 by default?
On Sun, May 20, 2012 at 4:43 AM, Glyph gl...@twistedmatrix.com wrote: Right now, e.g. http://docs.python.org/tutorial/index.html directly renders a page. I suggest that this be changed to a redirect to http://docs.python.org/release/2.7/tutorial/index.html. The fact that people can bookmark the default version of a document is kind of a bug. I'm -1 on that; unless there's a strong reason to avoid it, bookmarking the default version seems like the right thing to me. (One example of a strong reason would be if all Python modules were numbered sequentially in alphabetical order, meaning that adding a new module changes the URLs of existing modules' pages.) Compare the PostgreSQL documentation: if you do a web search for 'postgres nextval', you'll find the documentation for Postgres's sequence functions (which is correct), but chances are it'll be the old docs - version 8.1 most likely. If there's no weighting toward one in particular, I'd say that returning information for the latest version is the most logical default. Obviously there's more docs difference between Python 2 and Python 3 than between Postgres 8.1 and Postgres 9.1, but the most accessible version of a page should not IMHO distinguish between Python minor versions. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On Sun, Jun 24, 2012 at 9:40 AM, Christian Heimes li...@cheimes.de wrote: +1 for the general idea and for using Py_LIMITED_API. I still like my idea of a simple macro based on Include/patchlevel.h, for example: #define Py_API_VERSION(major, minor, micro) \ (((major) 24) | ((minor) 16) | ((micro) 8)) #if Py_LIMITED_API+0 = Py_API_VERSION(3, 3, 0) #endif This strikes me as in opposition to the Python-level policy of duck typing. Would it be more appropriate to, instead of asking if it's Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or would that result in an unnecessary proliferation of flag macros? ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Restricted API versioning
On Sun, Jun 24, 2012 at 5:08 PM, Nick Coghlan ncogh...@gmail.com wrote: On Sun, Jun 24, 2012 at 5:00 PM, Martin v. Löwis mar...@v.loewis.de wrote: This strikes me as in opposition to the Python-level policy of duck typing. Would it be more appropriate to, instead of asking if it's Python 3.3.0, ask if it's a Python that supports PY_FEATURE_FOOBAR? Or would that result in an unnecessary proliferation of flag macros? It would, hence I'm -1. I believe it is the motivation for the gcc assertion preprocessor feature, which never caught on. Right, if someone wants to check for a specific feature rather than just figuring out once the minimum version of the stable ABI that they need, then they can write an autotools macro (or equivalent in other build systems). Fair enough. I assume these sorts of things are only ever going to be added once, and not backported to old versions, so a single version number is guaranteed to suffice (it's not like available in 4.5.6 and 4.6.2 and 4.7.4). Go with the easy option! ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] stat module in C -- what to do with stat.py?
On Fri, Jun 21, 2013 at 11:06 AM, Nick Coghlan ncogh...@gmail.com wrote: On 21 June 2013 01:04, Thomas Wouters tho...@python.org wrote: If the .py file is going to be wrong or incomplete, why would we want to keep it -- or use it as fallback -- at all? If we're dead set on having a .py file instead of requiring it to be part of the interpreter (whichever that is, however it was built), it should be generated as part of the build process. Personally, I don't see the value in it; other implementations will need to do *something* special to use it anyway. Because practicality beats purity. This wrong Python code has been good enough for all Python version up until 3.4, it makes sense to keep it as a fallback instead of throwing it away. How would you know if the Python you're running on has an incorrect bitflag? If the wrong code is simply incomplete (it has the standard flags but none of the contended ones), that would at least be safe - you'll never get a false flag, but you might be unable to recognize the platform-specific ones. And then the platform-specific modules would always be adding, never overwriting, bitflags. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 as a Default in Linux Distros
On Wed, Jul 24, 2013 at 7:12 PM, Bohuslav Kabrda bkab...@redhat.com wrote:
- What should user get after using yum install python?
There are basically few ways of coping with this:
1) Just keep doing what we do, eventually far in the future drop python
package and never provide it again (= go on only with python3/python4/...
while having yum install python do nothing).
2) Do what is in 1), but when python is dropped, use virtual provide (*)
python for python3 package, so that yum install python installs python3.
3), 4) Rename python to python2 and {don't add, add} virtual provide python
in the same way that is in 1), 2)
5) Rename python to python2 and python3 to python at one point. This makes
sense to me from the traditional one version in distro + possibly compat
package shipping the old approach in Linux, but some say that Python 2 and
Python 3 are just different languages [3] and this should never be done.
If a user's (rather than dev's) perspective is of value, I'm a Debian
and Ubuntu user who wields Python as a multi-context weapon. Typing
packagemanager install python (for me that's apt-get, for you yum,
same diff) should get the one against which most other packages are
going to depend. Since your policy is to move people forward, that's
most likely going to be Python 3, unless you have a lot of packages
that simply haven't yet moved.
Your fifth option seems messy. I'd rather see python2 and python3 as
viable packages, and then use the virtual provide to symlink 'python'
to one of them, probably py3. So I'd vote for option 4. Python 2 and 3
aren't so different as to be called separate languages, there just
happen to be some backward-incompatible changes.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3 as a Default in Linux Distros
On Thu, Jul 25, 2013 at 2:21 AM, Laurent Gautier lgaut...@gmail.com wrote: - errors that are typical of Python 2 script running with Python 3-specific are probably limited (e.g., use of unicode, use of xrange, etc...) The most common, in interactive scripts at least, is likely to be: print Hello, world! SyntaxError: invalid syntax How helpful it's possible to make that one, I don't know. Is it safe to presume that it's more likely a syntax error will come from an interpreter version mismatch than a code bug? ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Official github mirror for CPython?
On Fri, Jul 26, 2013 at 12:29 AM, Eli Bendersky eli...@gmail.com wrote: Hi all, I've been looking for a Github mirror for Python, and found two: * https://github.com/python-git/python has a lot of forks/watches/starts but seems to be very out of date (last updated 4 years ago) * https://github.com/python-mirror/python doesn't appear to be very popular but is updated daily Also https://github.com/akheron/cpython seems to be the same again. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lambda [was Re: PEP 8 modernisation]
On Thu, Aug 1, 2013 at 5:58 PM, Alexander Shorin kxe...@gmail.com wrote: fun = lambda i: i[1] for key, items in groupby(sorted(items, key=fun), key=fun): print(key, ':', list(items)) I'd do a direct translation to def here: def fun(i): return i[1] for key, items in groupby(sorted(items, key=fun), key=fun): print(key, ':', list(items)) ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecating the formatter module
On Wed, Aug 14, 2013 at 6:42 PM, Nick Coghlan ncogh...@gmail.com wrote: That would be PEP 4 :) What's the normal way to update a PEP? ... proposals for deprecating modules MUST be made by providing a change to the text of this PEP, which SHOULD be a patch posted to SourceForge... Would that now be more accurately done as a Mercurial patch, or a tracker issue? ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] When to remove deprecated stuff
On Thu, Aug 22, 2013 at 11:45 PM, Michael Foord fuzzy...@voidspace.org.uk wrote: On 22 Aug 2013, at 14:00, Petri Lehtinen pe...@digip.org wrote: Django's deprecation policy works like this: They deprecate something in version A.B. It still works normally in A.B+1, generates a (silenced) DeprecationWarning in A.B+2, and is finally removed in A.B+3. So if I haven't read through the full release notes of each release nor enabled DeprecationWarnings, I end up having something broken each time I upgrade Django. So you're still using features deprecated three releases ago, you haven't checked for DeprecationWarnings and it's Django making your life difficult? Why not check for the deprecation warnings? Sounds like the DeprecationWarnings give you just one version of advance notice. You would have to be (a) upgrading every version as it comes out, and (b) checking your log of warnings prior to every upgrade. Neither A.B nor A.B+1 will help you even if you check the warnings. So it would still require checking the full release notes every time, if you want to know about what's being deprecated. Seems a lot of annoying breakage to me. Python is frequently not upgraded release-by-release. I've had servers jump several versions at a time; my oldest server now is running 3.1.1 (and 2.6.4), so when it eventually gets upgraded, it'll probably jump to 3.3 or 3.4. Unless something's producing visible warnings all the way back to 3.1, removal in 3.4 has the potential to be surprising. ChrisA ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] summing integer and class
On Thu, Oct 3, 2013 at 11:09 PM, Игорь Васильев vasilyev_i...@inbox.ru wrote: When we adding class to integer we have both slotv and slotw. x = slotv(v, w); - returns Py_NotImplemented. But in this case we should execute x = slotw(v, w); and function should be completed in the same way as when we adding integer to class. Can someone advise please where I mistake. No need to dig into the CPython source for this, the answer's pretty simple: 1+a is handled by __radd__ not __add__. class A(): def __add__(self, var): print(I'm in A class) return 5 def __radd__(self, var): print(I'm in A class, too) return 6 a=A() a+1 I'm in A class 5 1+a I'm in A class, too 6 You could ask this sort of thing on python-l...@python.org rather than python-dev. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Which direction is UnTransform? / Unicode is different
On Wed, Nov 20, 2013 at 11:03 PM, Steven D'Aprano st...@pearwood.info wrote: I *will* get confused over which direction is encoding and which is decoding. (Removing .decode() from the (unicode) str type in 3 does help a lot, if I have a Python 3 interpreter running to check against.) It took me a long time to learn that text encodes to bytes, and bytes decode back to text. Using Python 3 really helped with that. Rule of thumb: Stuff gets encoded for transmission/storage and decoded for usage. That covers encryption (you transmit the coded form and read the decoded), compression (you store the tighter form and use the expanded), Unicode (you store bytes, you work with characters), and quite a few others. I don't know that it's an iron-clad rule (though I can't off-hand think of a contrary example), but it's certainly an easy way to remember a lot of the encode/decode pairs. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 0404 and VS 2010
On Fri, Nov 22, 2013 at 10:32 PM, Nick Coghlan ncogh...@gmail.com wrote: A few folks overreacted in their concern about the community confusion such a move would inevitably create - *anything* called Python 2.8 is going to give the impression that we've changed our mind about 2.7 being the last feature release in the 2.x series, even if it has the word Stackless in front of it. From what I understand of the discussion, the problem is only because it's the specific value 2.8, which could plausibly be a Python version number. Calling it Stackless Python 10 wouldn't cause confusion (don't remember who suggested that), as it's clearly not part of the regular Python versioning system - you could say Stackless Python 10 is code compatible with Python 2.7 and there'd be no confusion. But if, say, Red Hat release something called Python 2.4.6-rhel, I would expect it to be entirely compatible with a python.org Python 2.4 release, possibly with some additional bugfixes; it says 2.4 in it, so I'm not going to go dig through its docs to figure out what my code will run on. I certainly wouldn't expect it to be 2.2 built with a new compiler, or 2.7 with additional libraries added, or anything. It's all about how it reads at first glance. In my personal opinion, I would very much like to see the 2.x line *not* be supported indefinitely. Why are so many people still using it? Because the impetus to shift is weaker than the inertia to stay. How will that be changed? By strengthening the need to move on. If anyone pledges to support a 2.x variant for the next couple of decades, then python-list will have to deal with people not knowing (and not saying) whether they're on 2.x or 3.x for probably another 30 years or more. I'd rather be able to stop warning people about input() and non-Unicode strings and from __future__ import division. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP process entry point and ill fated initiatives
On Fri, Nov 29, 2013 at 10:08 PM, anatoly techtonik techto...@gmail.com wrote: About ill fated initiatives. I don't like when people prematurely close tickets without waiting for the mutual agreement that the problem is solved. Perhaps trackers should have personal agree/disagree/meh flags to help other people state their opinions... Perhaps this offence is that makes them closing tickets without waiting for reply. Reply is not be required to close ticket for valid reason, but the point of conflict is that I don't like when tickets are prematurely closed under the reason of you. do it right instead of let's make it better... I am disrespectful to policies, that's right. I believe that policies need to be revised every couple of years, but nobody do this. Rules that work for folks that were present at the time the consensus is reached need to be explained to those who came later, and still you can't expect them to comply. Why do you feel that Python is (or ought to be) democratic? It isn't, and it should not be. Your opinion is, I am sure, given weight when decisions are made, but ultimately those decisions are not a matter of consensus. Why should the policies be revised? Because some people are coming in who don't like them? Poor reason. Being impolite will tend to reduce the weight your words are given. Demanding change is usually unsuccessful unless you can demonstrate a good reason for that change to happen, and I don't just mean I hate your policies and I think it's time you reviewed them. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Running the unit test as root/administrator
On Fri, Dec 6, 2013 at 2:29 AM, Martin v. Löwis mar...@v.loewis.de wrote: Am 05.12.13 16:21, schrieb Vajrasky Kok: On Thu, Dec 5, 2013 at 11:06 PM, Martin v. Löwis mar...@v.loewis.de wrote: Can you please phrase your question more explicit? What is it that you want to be done before writing unit tests for the spwd module? I am asking buildbot of Linux/Unix/BSD with root account. Do we have it now? Ah. I don't think we have one. If somebody would want to donate one, I suggest to run it in a VM, to reduce the (valid) security concerns that Guido has voiced. If a snapshot of the VM is made, it would be easy to restore in case a commit performs serious damage. Are we worried about malicious or accidental damage, here? I might be able to offer a buildbot VM, though I don't know what the requirements are. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Buildbot running Debian amd64 as root
In another thread it was suggested that a new buildbot running as root would be of value. I've spun up a virtual machine running Debian Wheezy amd64, have installed buildbot from the repository (version 0.8.6p1), and am ready to have it join the farm. How do I go about doing this? I've followed the instructions in https://wiki.python.org/moin/BuildBot as far as creating the slave. There's a user named buildbot but I'll be running the buildslave command as root - that would do it, right? Which means I just need to set up a name and password. The VM has 1GB RAM and is permitted up to two cores of my i5 CPU, and has 120GB disk space available. It's running behind NAT, which should be safe against accidental problems, but not malicious compromise. I'll look into firewalling it off to keep things safeish but mainly I'm trusting that Python core devs aren't going to be attacking my network :) ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backward-incompatible change to random.randrange in 2.7.6
On Wed, Dec 18, 2013 at 8:43 PM, Serhiy Storchaka storch...@gmail.com wrote: 18.12.13 04:40, Benjamin Peterson написав(ла): Mostly yes, but at least you could tell people to upgrade straight to 2.7.7 and skip 2.7.6. It'll make the people to postpone the upgrade to 2.7.6 (which fixes many security bugs) until 2.7.7 release, instead of correcting their morally-broken programs. If this is considered enough breakage to be a problem, would it be possible to issue a 2.7.6.1 or 2.7.6+fixed release that's identical to 2.7.6 but with this change reverted? It'd be a minor mess, but then people would still get those security fixes, and it means not breaking stuff in a point release. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] thread issues when embedding Python
On Wed, Dec 18, 2013 at 9:26 PM, Daniel Pocock dan...@pocock.com.au wrote: b) when each worker thread starts, call PyThreadState_New(mInterpreterState) and save the result in a thread local mPyThreadState c) use the mPyThreadState with PyEval_RestoreThread and PyEval_SaveThread before and after calling Python methods Is this a bug in PyGILState_Ensure or is it a deficiency in the documentation? It doesn't surprise me that you would need to do step b - I do seem to recall the need to call that for each new thread. Not so sure about c. Once you fixed the unrelated bug, do you still need that step? (Been a while since I last embedded Python though, and I might well be wrong.) ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot running Debian amd64 as root
On Sun, Dec 8, 2013 at 12:35 AM, Chris Angelico ros...@gmail.com wrote: In another thread it was suggested that a new buildbot running as root would be of value. I've spun up a virtual machine running Debian Wheezy amd64, have installed buildbot from the repository (version 0.8.6p1), and am ready to have it join the farm. How do I go about doing this? Is there any interest in this build slave? I haven't heard back from anyone as to where to go from here. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot running Debian amd64 as root
On Tue, Dec 31, 2013 at 12:05 AM, Chris Angelico ros...@gmail.com wrote: On Sun, Dec 8, 2013 at 12:35 AM, Chris Angelico ros...@gmail.com wrote: In another thread it was suggested that a new buildbot running as root would be of value. I've spun up a virtual machine running Debian Wheezy amd64, have installed buildbot from the repository (version 0.8.6p1), and am ready to have it join the farm. How do I go about doing this? Is there any interest in this build slave? I haven't heard back from anyone as to where to go from here. Antoine's contacted me off-list and sorted out the details. Bruce the Aussie Buildbot is now in operation. (And yes, the hostname is 'bruce'.) Thanks Antoine! ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot running Debian amd64 as root
On Tue, Dec 31, 2013 at 5:18 AM, Christian Heimes christ...@python.org wrote: The buildbot is missing some vital header files. Please run: # apt-get build-dep python3.3 to install all required dependencies. Debian Wheezy doesn't package 3.3 but only 3.2, so I grabbed 3.2's build-deps. They're now installed, so the next build should have everything for that. Does anyone happen to know what (if anything) 3.3 needs that 3.2 doesn't? I hit Force Build on the 3.3 one, so hopefully that'll show up. It's busily checking out now. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Buildbot - slave lost
Does Buildbot retain a constant TCP socket to its server? I'm seeing this: http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.3/builds/0 Results: Retry exception slave lost I have two internet connections; one is faster, but tends to drop socket connections after a few hours, and the other is a bit slower but more reliable (as long as the connection doesn't go idle for too long - neither of my upstreams is particularly MUD-friendly). Would it help to set a routing rule to send traffic to python.org:9020 via the connection that won't drop out? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot - slave lost
On Tue, Dec 31, 2013 at 11:42 AM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 31 Dec 2013 11:24:26 +1100 Chris Angelico ros...@gmail.com wrote: Does Buildbot retain a constant TCP socket to its server? I'm seeing this: http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.3/builds/0 I'm quite sure it does. It's necessary to get a real-time view of the test log. Would it help to set a routing rule to send traffic to python.org:9020 via the connection that won't drop out? The only way to know is to try it out :) If that's what it does, then it almost certainly will help. I put all my MUD connections through the other connection. The change is made, I'll trigger another build and see if it works. Thanks for the help! ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot running Debian amd64 as root
On Tue, Dec 31, 2013 at 12:35 PM, Zach Ware zachary.ware+py...@gmail.com wrote: Debian Wheezy doesn't package 3.3 but only 3.2, so I grabbed 3.2's build-deps. They're now installed, so the next build should have everything for that. Does anyone happen to know what (if anything) 3.3 needs that 3.2 doesn't? You'll need sources for lzma, but I think that should be it. Done, lzma-dev installed. Thanks. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot - slave lost
On Thu, Jan 2, 2014 at 9:17 AM, Martin v. Löwis mar...@v.loewis.de wrote: So it should be able to handle a failover from one link to the other, but it's certainly better to bind it to the more reliable transport. I believe you can somehow configure the frequency of ping messages so that you network doesn't believe the connection goes idle, plus it will attempt a reconnect if the network did indeed cancel the connection. If you wanted to study this further, you could look into the slave's twisted log file. I should have mentioned that I'd already peeked into the log and that's why I posted. There was something in there about it sending application-level keepalives (every 600 seconds, IIRC), which looked good; but it wasn't sufficient, as seen. After I switched it to the other connection, I didn't see any more problems (and it's completed a few builds, though 2.7 seems to be having trouble in the tests - not peculiar to this bot though). ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.x vs 3.x survey results
On Fri, Jan 3, 2014 at 1:18 PM, Dan Stromberg drsali...@gmail.com wrote: I don't know much (if anything ^_^) about survey methodology. I just created a 9 question survey and tossed it at a few places that Pythonistas hang out. Specifically, your methodology was to post the link to python-list and python-dev (and whatever else I didn't see). Apart from hacker news - not sure if that's a specific site or you're talking generically - your current description sounds right. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug? http.client assumes iso-8859-1 encoding of HTTP headers
On Sun, Jan 5, 2014 at 2:36 AM, Hugo G. Fierro h...@gfierro.com wrote:
I am trying to download an HTML document. I get an HTTP 301 (Moved
Permanently) with a UTF-8 encoded Location header and http.client decodes it
as iso-8859-1. When there's a non-ASCII character in the redirect URL then I
can't download the document.
In client.py def parse_headers() I see the call to decode('iso-8859-1'). My
personal hack is to use whatever charset is defined in the Content-Type
HTTP header (utf8) or fall back into iso-8859-1.
At this point I am not sure where/how a fix should occur so I thought I'd
run it by you in case I should file a bug. Note that I don't use http.client
directly, but through the python-requests library.
I'm not 100% sure, but I believe non-ASCII characters are outright
forbidden in a Location: header. It's possible that an RFC2047 tag
might be used, but my reading of RFC2616 is that that's only for text
fields, not for Location. These non-ASCII characters ought to be
percent-encoded, and anything doing otherwise is buggy.
Confirming what you're seeing with a plain socket:
s=socket.socket()
s.connect((www.starbucks.com,80))
s.send(b'GET
/store/158/AT/Karntnerstrasse/K%c3%a4rntnerstrasse-49-Vienna-9-1010
HTTP/1.1\r\nHost: www.starbucks.com\r\nAccept-Encoding: identity\r\n\r\n')
136
s.recv(1024)
b'HTTP/1.1 301 Moved Permanently\r\nContent-Type: text/html;
charset=UTF-8\r\nLocation:
http://www.starbucks.com/store/158/at/karntnerstrasse/k\xc3\xa4rntnerstrasse-49-vienna-9-1010\r\n
'
I'm pretty sure that server is in violation of the spec, so all bets
are off as to what any other server will do. If you know you're
dealing with this one server, you can probably hack around this, but I
don't think it belongs in core code. Unless, of course, I'm completely
wrong about the spec, or if there's a de facto spec that lots of
servers follow, in which case maybe it would be worth doing.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python 2.7 root buildbot showing errors
The first build my new root buildbot did showed errors in the 2.7 test suite, but I thought little of it as quite a few other 2.7 buildbots are showing red, too. But it seems they're showing different errors, so there might be something wrong with the setup. http://buildbot.python.org/all/builders/AMD64%20Debian%20root%202.7/builds/3/steps/test/logs/stdio First off, it's complaining about being unable to build _curses (lacking curses.h). Is that a mandatory prereq that I should install, or should Python be compatible with not having it? Then further down, several SSL tests attempt: s.connect_ex((svn.python.org, 444))) and get back EAGAIN when they're expecting ECONNREFUSED. Possibly my firewall's delaying things somewhat and it's timing out with a signal; when I try manually, the connection times out. Are these failures a problem? Should they be fixed? The 3.x builds are all coming up green. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7 root buildbot showing errors
On Mon, Jan 6, 2014 at 7:11 PM, Christian Heimes christ...@python.org wrote: On 06.01.2014 08:09, Chris Angelico wrote: Then further down, several SSL tests attempt: s.connect_ex((svn.python.org, 444))) and get back EAGAIN when they're expecting ECONNREFUSED. Possibly my firewall's delaying things somewhat and it's timing out with a signal; when I try manually, the connection times out. Are you running the VM on Windows? I've seen similar issues on Windows and Windows as host platform for VMs: http://bugs.python.org/issue19919 No, it's Debian Wheezy inside Debian Wheezy; though the outer system is a somewhat messy one (I installed Wheezy before it was stable, and compiled my own ALSA drivers and a few other things). But it could well be that same issue, as it seems to involve NAT. What's the policy on backporting patches to tests onto 2.7? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7 root buildbot showing errors
On Mon, Jan 6, 2014 at 7:58 PM, Christian Heimes christ...@python.org wrote: Interesting, maybe it's a general NAT issue? So far I have seen the issue on Windows only. What kind of VM are you using? I'm using virtualbox for my Windows VMs. It's Oracle VirtualBox v4.2.20 r90963. Just backport the test fixes. Test fixes and new tests are not new feature so you are always allowed to add new tests or fix existing tests. Okay. I don't have write access, myself. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 460: Add bytes % args and bytes.format(args) to Python 3.5
On Tue, Jan 7, 2014 at 12:44 AM, Antoine Pitrou solip...@pitrou.net wrote: BTW, there's a subtlety here: ``%s`` currently means insert the result of calling __str__, but bytes formatting should *not* call __str__. Since it derives from the C printf notation, it means insert string here. The fact that __str__ will be called is secondary to that. I would say it's not a problem for bytes formatting to call __bytes__, or in some other way convert to bytes without calling __str__. Will it be confusing to have bytes and str supporting distinctly different format operations? Might it be better to instead create a separate and very different method on a bytes, just to emphasize the difference? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 460: Add bytes % args and bytes.format(args) to Python 3.5
On Wed, Jan 8, 2014 at 9:12 PM, Victor Stinner victor.stin...@gmail.com wrote: IMO some formatting commands must not be implemented. For example, alignment is used to display something on screen, not in network protocols or binary file formats. Must not, or need not? I can understand that those sorts of features would be less valuable, but they do make sense. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 complexity
On Thu, Jan 9, 2014 at 11:21 AM, MRAB pyt...@mrabarnett.plus.com wrote:
On the other hand:
I need a new battery.
What kind of battery?
I don't care!
Or, bringing it back to Python: How do you write a set out to a file?
foo = {1, 2, 4, 8, 16, 32}
open(foo.txt,w).write(foo) # Uh... nope!
I don't want to have to worry about how it's formatted! I just want to
write that set out and have someone read it in later!
A text string is just as abstract as any other complex type. For some
reason, we've grown up thinking that ABCD == \x61\x62\x63\x64 ==
ABCD, even though it's just as logical for those bytes to represent
12.1414 or 1094861636 or 1145258561. There's no difference between
encoding one thing to bytes and encoding another thing to bytes, and
it's critical to get those encodes/decodes right.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 complexity
On Thu, Jan 9, 2014 at 5:50 PM, Lennart Regebro rege...@gmail.com wrote:
To be honest, you can define text as A stream of bytes that are split
up in lines separated by a linefeed, and do some basic text
processing like that. Just very *basic*, but still. Replacing
characters. Extracting certain lines etc.
You would have to define it as A stream of bytes encoded in
{ASCII|Latin-1|CP-1252|UTF-8} that etc etc. Otherwise, those bytes
might be EBCDIC, UTF-16, or anything else, and your code will fail.
And once you've demanded that, well, you're right back here with
clarifying encodings, so you may as well just pass encoding=ascii
and do it honestly.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 complexity
On Fri, Jan 10, 2014 at 11:53 AM, anatoly techtonik techto...@gmail.com wrote: 2. introduce autodetect mode to open functions 1. read and transform on the fly, maintaining a buffer that stores original bytes and their mapping to letters. The mapping is updated as bytes frequency changes. When the buffer is full, you have the best candidate. Bad idea. Bad, bad idea! No biscuit. Sit! This sort of magic is what brings the bush hid the facts bug in Windows Notepad. If byte value distribution is used to guess encoding, there's no end to the craziness that can result. How do you know that the byte values 0x41 0x42 0x43 0x44 are supposed to mean upper-case ASCII letters and not a 32-bit integer or floating-point value, or some accented lower-case letter A's in EBCDIC, or anything else? Maybe if you have a whole document, AND you know for sure that it's linguistic text, then maybe - MAYBE - you could guess with reasonable reliability. But even then, how can you be sure? Remember, too, you might have to deal with something that's actually mis-encoded. If you're told this is UTF-8 and you find the byte sequence ED B3 BF, do you decide that it can't possibly be UTF-8 and pick a different encoding to decode with? That would produce no end of trouble, where the actual result you want is (most likely) to throw an error. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 complexity
On Fri, Jan 10, 2014 at 1:39 PM, Steven D'Aprano st...@pearwood.info wrote: On Fri, Jan 10, 2014 at 12:22:02PM +1100, Chris Angelico wrote: On Fri, Jan 10, 2014 at 11:53 AM, anatoly techtonik techto...@gmail.com wrote: 2. introduce autodetect mode to open functions 1. read and transform on the fly, maintaining a buffer that stores original bytes and their mapping to letters. The mapping is updated as bytes frequency changes. When the buffer is full, you have the best candidate. Bad idea. Bad, bad idea! No biscuit. Sit! This sort of magic is what brings the bush hid the facts bug in Windows Notepad. If byte value distribution is used to guess encoding, there's no end to the craziness that can result. I think that heuristics to guess the encoding have their role to play, if the caller understands the risks. For example, an application might give the user the choice of specifying the codec, or having the app guess it. (I dislike the term Auto detect, since that implies a level of certainty which often doesn't apply to real files.) There is already a third-party library, chardet, which does this. Perhaps the std lib should include this? Perhaps chardet should be considered best-of-breed atomic reactor, but the std lib could include a battery to do something similar. I don't think we ought to dismiss this idea out of hand. I don't deny that chardet has its place, but would you use it like this (I'm assuming it works with Py3, the docs seem to imply Py2): text = with open(blah, rb) as f: while True: data = f.read(256) if not data: break text += data.decode(chardet.detect(data)['encoding']) Certainly not. But that's how the file-open-mode of auto detect sounds. At very least, it has to do something like this _until_ it has confidence; maybe it can retain the chardet state after the first read, but it's still going to have to decode as little as you first read. How can it handle this case? first_char = open(blah, encoding=auto).read(1) Somehow it needs to know how many bytes to read (and not read too many more, preferably - buffering a line-ish is reasonable, buffering a megabyte not so much) and figure out what's one character. I see this as similar to the Python 2 input() function. It's not the file-open builtin's job to do something advanced and foot-shooting as automatic charset detection. If you want that, you should be prepared for its failures and the messes of partial reads, and call on chardet yourself, same as you should use eval(input()) explicitly in Py3 (and, in my opinion, eval(raw_input()) equally explicitly in Py2). I'm not saying that chardet is bad, but I *am* saying, and I stand by this, that an auto-detect option on file open is a bad idea. Unix comes with a 'file' command which will tell you even more about what something is. (For what it thinks are text files, I believe it uses heuristics similar to chardet to guess an encoding.) Would you want a parameter to the open() builtin that tries to read the file as an image, or an audio file, or a document, or an executable, and automatically decodes it to a PIL.Image, an mm.wave, etc, or execute the code and return its stdout, all entirely automatically? I don't think so. Not open()'s job. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] RFC: PEP 460: Add bytes % args and bytes.format(args) to Python 3.5
On Mon, Jan 13, 2014 at 4:57 AM, Juraj Sukop juraj.su...@gmail.com wrote: On Sun, Jan 12, 2014 at 6:22 PM, Steven D'Aprano st...@pearwood.info wrote: First, utf16_string confuses me. What is it? If it is a Unicode string, i.e.: It is a Unicode string which happens to contain code points outside U+00FF (as with the TTF example above), so that it triggers the (at least) 2-bytes memory representation in CPython 3.3+. I agree, I chose the variable name poorly, my bad. When I'm talking about Unicode strings based on their maximum codepoint, I usually call them something like ASCII string, Latin-1 string, BMP string, and SMP string. Still not wholly accurate, but less confusing than naming an encoding... oh wait, two of those _are_ encodings :| But you could use narrow string for the first two. Or string(0..127) for ASCII, string(0..255) for Latin-1, and then for consistency string(0..65535) and string(0..1114111) for the others, except that I doubt that'd be helpful :) At any rate, BMP as a term for includes characters outside of Latin-1 but all on the Basic Multilingual Plane would probably be close enough to get away with. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Automatic encoding detection [was: Re: Python3 complexity - 2 use cases]
On Tue, Jan 14, 2014 at 10:48 AM, Jim J. Jewett jimjjew...@gmail.com wrote: The barrier for entry to the standard library is higher than mere usefulness. Agreed. But most programs will need it, and people will either include (the same) 3rd-party library themselves, or write their own workaround, or have buggy code *is* sufficient. Well, no, that's not sufficient on its own either. But yes, it's a stronger argument. But having a batch process crash one run in ten (where it didn't crash at all under Python 2) is a bad thing. There are environments where (once I knew about it) I would add chardet (if I could get approval for the 3rd-party component). Having it *do the wrong thing* one run in ten is even worse. If you need chardet, then get approval for the third-party component. That's a political issue, not a technical one. This needs to be in the stdlib because I'm not allowed to install anything else? I hope not. Also, a PyPI package is free to update independently of the Python version schedule. The stdlib is bound. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Test failures when running as root
And now for something completely different. My root buildbot is finally now able to telnet out and get Connection refused errors. (For the curious, the VirtualBox NAT mode doesn't work properly, but the new NAT Network mode does. Why? I have no idea. But if anyone else is having the same problem, upgrade to the latest VirtualBox and set up a NAT Network. All I care is, it now works.) The test suite is now failing at another point, and this applies to 2.7, 3.3, and 3.x. == ERROR: test_initgroups (test.test_posix.PosixGroupsTester) -- Traceback (most recent call last): File /root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_posix.py, line 1143, in test_initgroups g = max(self.saved_groups) + 1 ValueError: max() arg is an empty sequence -- The saved_groups value comes from posix.getgroups(), and it's being used to try to get a group that this user doesn't have (I think). When I run Python as root, posix.getgroups() returns [0], but apparently it's not returning any groups when the test runs. So, two questions. Firstly, is this a problem that needs to be fixed in Python, or is it a configuration change that I made? It began failing recently, so possibly when I rebooted the VM as part of VirtualBox changes I mucked something up. And secondly, how can I run the tests manually? I can't find a binary inside the buildarea tree. Does it get deleted afterward? Apologies if these are dumb questions, hopefully they're a small distraction from PEP 460 arguments! ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures when running as root
On Tue, Jan 14, 2014 at 2:14 PM, Zachary Ware zachary.ware+py...@gmail.com wrote: On Mon, Jan 13, 2014 at 6:48 PM, Chris Angelico ros...@gmail.com wrote: And secondly, how can I run the tests manually? I can't find a binary inside the buildarea tree. Does it get deleted afterward? Yes, that's the 'clean' step of the buildbot build process. I'd suggest making another clone elsewhere (you can clone from the buildarea just to make the clone faster, but I'd leave the buildarea alone otherwise), then building and testing should be as simple as `./configure --with-pydebug make ./python -m test.test_posix`. Doh. Yeah, I can see the 'clean' step in the build process, I should have known. Of course. Thanks, that's what I'll do then. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test failures when running as root
On Tue, Jan 14, 2014 at 2:03 PM, Terry Reedy tjre...@udel.edu wrote: On 1/13/2014 7:48 PM, Chris Angelico wrote: ValueError: max() arg is an empty sequence try: g = max(self.saved_groups) + 1 except ValueError: g = 1 Unless someone says that it is a bug for posix.getgroups to return an empty list, I would say that the test should be fixed by trying the code above. I can't see anything in the getgroups man page [1] to suggest that it's a bug to return an empty list. But I can't replicate the behaviour either - not on the system Python, at least (2.7.3), hence the query about rerunning tests. Will raise an issue on the tracker. [1] eg http://linux.die.net/man/2/getgroups ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Common subset of python 2 and python 3
On Wed, Jan 15, 2014 at 11:22 PM, Martin v. Löwis mar...@v.loewis.de wrote: I don't think that it is possible to write an interpreter that is fully compatible for all it accepts. Would you think that the program print(repr(2**80).endswith(L)) is in the subset that should be supported by both Python 2 and Python 3? Easiest fix for that would be to have long.__repr__ omit the L tag. Then it'll do the same as it would in Py3. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Common subset of python 2 and python 3
On Thu, Jan 16, 2014 at 1:25 AM, Eric V. Smith e...@trueblade.com wrote: Easiest fix for that would be to have long.__repr__ omit the L tag. Then it'll do the same as it would in Py3. I think Martin's point is not this specific thing, but that such a subset would be useless. Would you drop dict.items() because it returns different things in both languages? Drop range() because it's different? There are many, many such differences. The common subset is not useful. Fair enough. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461 Final?
On Sat, Jan 18, 2014 at 12:51 PM, Ethan Furman et...@stoneleaf.us wrote: It seems there are two possibilities with %a: 1) have it be ascii(repr(obj)) Wouldn't that be redundant? ascii() is already repr()-like. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] .clinic.c vs .c.clinic
On Sat, Jan 18, 2014 at 8:02 PM, Serhiy Storchaka storch...@gmail.com wrote: 2. I'm not use any IDE, but if you use, it can be important for you. If IDE shows sources tree, unlikely you want to see generated *.clinic.c files in them. This will increase the list of sources almost twice. A point for the contrary side: In any editor or IDE with syntax highlighting, a .clinic.c file will be highlighted as C code, but it would take extra configuration to handle a .clinic file that way. But that's a relatively minor consideration (AIUI most people won't be looking at the .clinic files much, and for those who do, configure the editor appropriately). ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Enable Hostname and Certificate Chain Validation
On Wed, Jan 22, 2014 at 11:15 PM, Donald Stufft don...@stufft.io wrote: Do you really think those people would be making the same complaints if they could restore the previous behavior with a simple boolean flag delivered either via environment variable or in their own code? You assume that it's easy to tweak the code. From personal experience just today I can say that this isn't always the case. I was asked a question about an internal program that had been in use since the late 1990s, and which had originally been written to work with Netscape Navigator and had been updated to work with Firefox, but not Chrome. The original author is still around, but it's too much hassle to get that code dug into, so it's far easier just to accept a small issue with Chrome (since the program's not used very often anyway). But if Chrome had completely broken that program, the solution would simply be keep using Firefox, not fix the program - it's not considered a bug. Now, maybe it wouldn't be a problem if the fix is an environment variable, but imagine a thousand-computer deployment and you have to tweak the environment on all of them. Feel like doing that just because the newest Python needs it? Not so much. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Enable Hostname and Certificate Chain Validation
On Thu, Jan 23, 2014 at 1:08 AM, Jesse Noller jnol...@gmail.com wrote: Now, maybe it wouldn't be a problem if the fix is an environment variable, but imagine a thousand-computer deployment and you have to tweak the environment on all of them. Feel like doing that just because the newest Python needs it? Not so much. What's the bet that that application will be ported to python 3.4/3.5 if this is the case? I'd say approaching 0, which is ok. Define ported to. (This particular application isn't Python, so the specifics don't apply, but in general.) Usually that means simply run on. Something that was written for Python 3.2 will probably run on 3.3, and on 3.4, and on 3.5 as well. You certainly wouldn't expect one small corner of it to suddenly start doing different stuff, and if you do, you'll blame Python... which would mean that you're right, that program wouldn't be run on 3.4. Is that a good thing? I don't know, but I think not. In a big company with lots of seats, every option is looking like a sysadmin's nightmare. That said, though, I agree *in principle* that secure-by-default is the way to go. It's just the backward-incompatibility of *changing* it. I like how requests is going. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str.rreplace
On Sat, Jan 25, 2014 at 3:38 AM, Antoine Pitrou solip...@pitrou.net wrote: On Fri, 24 Jan 2014 18:32:17 +0200 Ram Rachum r...@rachum.com wrote: Question: Why is there no str.rreplace in Python? What would it do? (also, I think such questions are better asked on python-ideas) Or python-list. Chances are there's a way to do it already, which would be of interest to other people who might be looking. But I've no idea what semantics are expected. :) ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str.rreplace
On Sat, Jan 25, 2014 at 4:19 AM, Ram Rachum r...@rachum.com wrote: I now looked at the 17 most recent python-list threads. Out of them: - 58% are about third-party packages. - 17% are off-topic (not even programming related) - 11% are 2-vs-3 discussions - 5% are job offers. - 5% (which is just one thread out of 17) is about Python the language. My analysis here is based on what I see arriving in Gmail, so some of them may have been dropped into spam. But these are the threads with the most recent posts: The potential for a Python 2.8 - discussing the language, though the last few posts drifted off into numeric jokes (also fun). Class and instance related questions - short thread but completely on topic (so far) Python declarative - not all the code shown has been Python, and a lot of the discussion centers around alternatives like XML and JSON, but it's definitely focused on Python datetime as subclass of date - on topic Can post a code but afraid of plagiarism - haven't been following it, but last I saw it was on topic Elementree and insert new element if it is not present - might count as discussion of a separate module, I guess generate De Bruijn sequence memory and string vs lists - all about how to do it in Python, looks on topic to me Need Help with Programming Science Project - the OP never said that the program was to be in Python, but if we assume that, it's completely on topic Further down than that we have a few about SQLite, which Python comes with, and an announcement of a new version of Dipy. Far as I can see, that's only two threads that are truly about third-party modules (that and lxml). Yes, there's some noise on the list, but it's not as bad as 16/17ths of the threads. Maybe you're reading it in some way other than the mailing list, and it accrues more noise? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Quick poll: should help() show bound arguments?
On Sat, Jan 25, 2014 at 3:07 PM, Larry Hastings la...@hastings.org wrote: What should it be? A) pydoc and help() should not show bound parameters in the signature, like inspect.signature. B) pydoc and help() should show bound parameters in the signature, like inspect.getfullargspec. Vote for A. As far as I'm concerned, all these foo are equally callable and equally take one parameter named a: def foo1(a): pass class C: def foo(self, a): pass foo2=C().foo class C: def __call__(self, a): pass foo3=C() def two_arg(b, a): pass foo4=lambda a: two_arg(0, a) If I call them as fooN(), fooN(1), and fooN(1,2), the middle one works and the other two throw exceptions, ergo they are one-argument functions. The fact that two of them happen to be bound methods is an implementation detail; it's just a form of currying, which foo4 happens also to be (in that C.foo takes two args, C().foo takes one). ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPI offline, status is ok
On Mon, Feb 10, 2014 at 5:23 PM, anatoly techtonik techto...@gmail.com wrote: http://status.python.org/ shows all green https://pypi.python.org/pypi/gazest shows Error 503 backend read error backend read error Guru Meditation: XID: 2792709923 Working for me. But then, your email only just came through, and it's dated four hours ago, so maybe python-dev was also having trouble. Or maybe your ISP was having trouble. Hard to say now. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 7:04 PM, Chris Withers ch...@simplistix.co.uk wrote: To implement __lt__ in Python 2, I could do: def __lt__(self, other): if not isinstance(other, Range): return True return ((self._lower, self._upper, self._bounds) (other._lower, other._upper, other._bounds)) Because None 1 raises a TypeError, in Python 3 I have to do: def __lt__(self, other): if not isinstance(other, Range): return NotImplemented for attr in '_lower', '_upper', '_bounds': self_value = getattr(self, attr) other_value = getattr(other, attr) if self_value == other_value: pass elif self_value is None: return True elif other_value is None: return False else: return self_value other_value return False Am I missing something? How can I get this method down to a sane size? Can you be certain that all your values are either None or positive integers? If so, try this: def __lt__(self, other): if not isinstance(other, Range): return True # or NotImplemented, not sure why this change return ((self._lower or 0, self._upper or 0, self._bounds or 0) (other._lower or 0, other._upper or 0, other._bounds or 0)) That'll treat all Nones as 0, and compare them accordingly. If you can't depend on them being positive (eg if you need None to be less than 0 rather than equal - even more so if you need it to be less than negative numbers), you'll need to more explicitly check. But this is nice and tidy, if it works. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 9:20 PM, Antoine Pitrou solip...@pitrou.net wrote: Hmm, it seems you're right, but I'm quite sure some DBMSes have a consistent way of ordering NULLs when using ORDER BY on a nullable column. Yes, and I believe it's part of the SQL-92 spec. Certainly here's PostgreSQL's take on the matter: http://www.postgresql.org/docs/9.3/static/sql-select.html#SQL-ORDERBY In short, NULL is by default considered greater than anything else (last in an ascending sort, first in a descending sort), but this can be inverted. Oddly enough, the syntax is NULLS FIRST vs NULLS LAST, not NULLS LOW vs NULLS HIGH. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Fri, Feb 14, 2014 at 9:42 PM, Nick Coghlan ncogh...@gmail.com wrote: IIRC, MySQL and PostgreSQL sort them in the opposite order from each other Ouch! You're right: http://dev.mysql.com/doc/refman/5.7/en/working-with-null.html When doing an ORDER BY, NULL values are presented first if you do ORDER BY ... ASC and last if you do ORDER BY ... DESC. Not configurable in MySQL. It's apparently not part of the standard, according to Wikipedia: http://en.wikipedia.org/wiki/Order_by So definitely SQL's handling of NULL should not be any sort of guide as regards Python's treatment of None. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Sat, Feb 15, 2014 at 4:14 AM, Chris Barker chris.bar...@noaa.gov wrote:
(though it could get a bit tricky -- what would AlwaysGreater float('inf)
evaluate to?
It'd be true. AlwaysGreater is greater than infinity. It is greater
than float(nan). It is greater than the entire concept of floating
point. It is greater than everything imaginable... except for a nice
MLT - a mutton, lettuce, and tomato sandwich... oh.
Where things get tricky is when you compare two AlwaysGreater objects.
Or even compare one against itself. It has to be greater.
Great.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Tue, Feb 18, 2014 at 1:30 AM, M.-A. Lemburg m...@egenix.com wrote: Also think of the implications of changing None at this point. It would allow us to write programs that work Python = 3.5 and Python = 2.7, but fail mysteriously in all other versions in between. What a mess that would be... Yes, that's unfortunately true. I don't know that that's in itself that much of a problem. (BTW, I wouldn't call it = 2.7; it'd be all 2.x, it's not like some of the 2.7.y versions aren't included.) There are already barriers to supporting 2.7 and 3.1/3.2, like the uasdf notation for Unicode literals. Making it easier to support 2.x and 3.x from the same codebase is an improvement that has been done and will be done more. It's not such a mysterious failure; it's just that you support Python 2.5+ and 3.4+ (or whatever the specific versions are). ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 3 niggle: None 1 raises TypeError
On Wed, Feb 19, 2014 at 3:10 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: Sorry if this has already been suggested, but why not introduce a new singleton to make the database people happier if not happy? To avoid confusion call it dbnull? A reasonable compromise or complete cobblers? :) That would be a major change to the DB API. Possibly the best solution, though. Start off by having the default be to return None (as now) and have a flag that can be set please return sys.dbnull instead (does dbnull belong in sys?), and let that settle for a few years. Recommend that all applications explicitly set the flag, either to True or to False. Then eventually, with the full deprecation warnings, change the default to True. (Or maybe make it an error to not set it.) Then, after another long round of deprecations, drop the None behaviour from the spec altogether. But even in its early steps, that could solve the problem. Anyone who wants to sort a list of tuples that came from the database can simply set the flag (assuming their back-end supports that flag) and happily use dbnull. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEPs don't redirect on python.org
Apologies if this is misdirected! I notice the switch to the new python.org web site has happened; but now PEPs are simply 404: http://www.python.org/dev/peps/pep-0008/ However, trimming the URL offers a redirect: http://www.python.org/dev/peps/ - http://legacy.python.org/dev/peps/ from which the content can be found. Can a blanket redirect rule be put in that makes all the PEPs at least go to /dev/peps/ ? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEPs don't redirect on python.org
On Thu, Feb 20, 2014 at 10:25 AM, Chris Angelico ros...@gmail.com wrote: I notice the switch to the new python.org web site has happened; but now PEPs are simply 404: http://www.python.org/dev/peps/pep-0008/ However, trimming the URL offers a redirect: http://www.python.org/dev/peps/ - http://legacy.python.org/dev/peps/ Oh! Must have been a transitional period. The PEPs now redirect too. Thanks, whoever did that! Makes the linking much easier. :) ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 463: Exception-catching expressions
PEP: 463 Title: Exception-catching expressions Version: $Revision$ Last-Modified: $Date$ Author: Chris Angelico ros...@gmail.com Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 15-Feb-2014 Python-Version: 3.5 Post-History: 16-Feb-2014, 21-Feb-2014 Abstract Just as PEP 308 introduced a means of value-based conditions in an expression, this system allows exception-based conditions to be used as part of an expression. Motivation == A number of functions and methods have parameters which will cause them to return a specified value instead of raising an exception. The current system is ad-hoc and inconsistent, and requires that each function be individually written to have this functionality; not all support this. * dict.get(key, default) - second positional argument in place of KeyError * next(iter, default) - second positional argument in place of StopIteration * list.pop() - no way to return a default * seq[index] - no way to handle a bounds error * min(sequence, default=default) - keyword argument in place of ValueError * sum(sequence, start=default) - slightly different but can do the same job * statistics.mean(data) - no way to handle an empty iterator Rationale = The current system requires that a function author predict the need for a default, and implement support for it. If this is not done, a full try/except block is needed. Since try/except is a statement, it is impossible to catch exceptions in the middle of an expression. Just as if/else does for conditionals and lambda does for function definitions, so does this allow exception catching in an expression context. This provides a clean and consistent way for a function to provide a default: it simply raises an appropriate exception, and the caller catches it. With some situations, an LBYL technique can be used (checking if some sequence has enough length before indexing into it, for instance). This is not safe in all cases, but as it is often convenient, programmers will be tempted to sacrifice the safety of EAFP in favour of the notational brevity of LBYL. Additionally, some LBYL techniques (eg involving getattr with three arguments) warp the code into looking like literal strings rather than attribute lookup, which can impact readability. A convenient EAFP notation solves all of this. There's no convenient way to write a helper function to do this; the nearest is something ugly using either lambda:: def except_(expression, exception_list, default): try: return expression() except exception_list: return default() value = except_(lambda: 1/x, ZeroDivisionError, lambda: float(nan)) which is clunky, and unable to handle multiple exception clauses; or eval:: def except_(expression, exception_list, default): try: return eval(expression, globals_of_caller(), locals_of_caller()) except exception_list as exc: l = locals_of_caller().copy() l['exc'] = exc return eval(default, globals_of_caller(), l) def globals_of_caller(): return sys._getframe(2).f_globals def locals_of_caller(): return sys._getframe(2).f_locals value = except_(1/x,ZeroDivisionError, Can't divide by zero ) which is even clunkier, and relies on implementation-dependent hacks. (Writing globals_of_caller() and locals_of_caller() for interpreters other than CPython is left as an exercise for the reader.) Raymond Hettinger `expresses`__ a desire for such a consistent API. Something similar has been `requested`__ `multiple`__ `times`__ in the past. __ https://mail.python.org/pipermail/python-ideas/2014-February/025443.html __ https://mail.python.org/pipermail/python-ideas/2013-March/019760.html __ https://mail.python.org/pipermail/python-ideas/2009-August/005441.html __ https://mail.python.org/pipermail/python-ideas/2008-August/001801.html Proposal Just as the 'or' operator and the three part 'if-else' expression give short circuiting methods of catching a falsy value and replacing it, this syntax gives a short-circuiting method of catching an exception and replacing it. This currently works:: lst = [1, 2, None, 3] value = lst[2] or No value The proposal adds this:: lst = [1, 2] value = lst[2] except IndexError: No value Specifically, the syntax proposed is:: expr except exception_list: default where expr, exception_list, and default are all expressions. First, expr is evaluated. If no exception is raised, its value is the value of the overall expression. If any exception is raised, exception_list is evaluated, and should result in either a type or a tuple, just as with the statement form of try/except. Any matching exception will result in the corresponding default expression being evaluated and becoming the value of the expression. As with the statement form of try/except, non-matching exceptions will propagate upward
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Fri, Feb 21, 2014 at 3:54 PM, Ethan Furman et...@stoneleaf.us wrote: (I'll vote on the syntax when the bikeshedding begins ;). Go get the keys to the time machine! Bikeshedding begins a week ago. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Fri, Feb 21, 2014 at 10:35 PM, Nick Coghlan ncogh...@gmail.com wrote: On 21 February 2014 13:15, Chris Angelico ros...@gmail.com wrote: PEP: 463 Title: Exception-catching expressions Great work on this Chris - this is one of the best researched and justified Python syntax proposals I've seen :) It is? Wow... I'm not sure what that says about other syntax proposals. This is one week's python-ideas discussion plus one little script doing analysis on the standard library. Hardly PhD level research :) Parentheses around the entire expression Generator expressions require parentheses, unless they would be strictly redundant. Ambiguities with except expressions could be resolved in the same way, forcing nested except-in-except trees to be correctly parenthesized and requiring that the outer expression be clearly delineated. `Steven D'Aprano elaborates on the issue.`__ __ https://mail.python.org/pipermail/python-ideas/2014-February/025647.html I'd like to make the case that the PEP should adopt this as its default position. My rationale is mainly that if we start by requiring the parentheses, it's pretty straightforward to take that requirement away in specific cases later, as well as making it easier to introduce multiple except clauses if that ever seems necessary. However, if we start without the required parentheses, that's it - we can't introduce a requirement for parentheses later if we decide the bare form is too confusing in too many contexts, and there's plenty of potential for such confusion. If once the parens are made mandatory, they'll most likely stay mandatory forever - I can't imagine there being any strong impetus to change the language to make them optional. The required parentheses also help in the cases where there is a nearby colon with a different meaning: if check() except Exception: False: ... if (check() except Exception: False): People can already write: if (x if y else z): without the parens, and it works. Readability suffers when the same keyword is used twice (here if rather than the colon, but same difference), yet the parens are considered optional. Python is a language that, by and large, lacks syntactic salt; style guides are free to stipulate more, but the language doesn't make demands. I would strongly *recommend* using parens in all the cases you've shown, especially lambda: lambda x: calculate(x) except Exception: None lambda x: (calculate(x) except Exception: None) as it would otherwise depend on operator precedence; but mandating them feels to me like demanding readability. def f(a: OS dependent = os_defaults[os.name] except KeyError: None): pass def f(a: OS dependent = (os_defaults[os.name] except KeyError: None)): pass Ehh, that one's a mess. I'd be looking at breaking out the default: default = os_defaults[os.name] except KeyError: None def f(a: OS dependent = default): pass with possibly some better name than 'default'. The one-liner is almost 80 characters long without indentation and with very short names. But if it's okay to wrap it, that would work without the parens: def f( a: OS dependent = os_defaults[os.name] except KeyError: None, another_arg ., more, args .., ): pass Clarity is maintained by judicious placement of newlines just as much as by parentheses. Rather than making people consider do I need the parentheses in this case or not?, adopting the genexp rule makes it simple: yes, you need them, because the compiler will complain if you leave them out. Yes, that is a reasonable line of argument. On the other hand, there's no demand for parens when you mix and and or: x or y and z I'd wager more than half of Python programmers would be unable to say for sure which would be evaluated first. The compiler could have been written to reject this (by placing and and or at the same precedence and having no associativity - I'm not sure if the current lexer in CPython can do that, but it's certainly not conceptually inconceivable), but a decision was made to make this legal. Retrieving a message from either a cache or the internet, with auth check:: logging.info(Message shown to user: %s,((cache[k] except LookupError: (backend.read(k) except OSError: 'Resource not available') ) if check_permission(k) else 'Access denied' ) except BaseException: This is like a bare except clause) I don't think taking it all the way to one expression shows the new construct in the best light. Keeping this as multiple statements assigning to a temporary variable improves the readability quite a bit: Yeah, good point. I tried to strike a balance between simple and complex examples, but it's hard to judge. I would also move the bare except clause equivalent out to a separate example. Remember, you're trying to convince people to *like* the PEP, not scare them away
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Fri, Feb 21, 2014 at 11:37 PM, Paul Moore p.f.mo...@gmail.com wrote: On 21 February 2014 11:35, Nick Coghlan ncogh...@gmail.com wrote: Just as PEP 308 introduced a means of value-based conditions in an expression, this system allows exception-based conditions to be used as part of an expression. Great work on this Chris - this is one of the best researched and justified Python syntax proposals I've seen :) Agreed - given the number of differing opinions on python-ideas, it's particularly impressive how well the debate was conducted too. On that subject, I'd like to thank everyone involved in the python-ideas discussion, and particularly Steven D'Aprano, Rob Cliffe, and Andrew Barnert; it was an amazingly productive thread, getting to nearly four hundred emails before seriously meandering. And even then, it mostly just started looping back on itself, which isn't surprising given that it was so long - anyone not majorly invested in the topic won't have read every single post. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 12:53 AM, Nick Coghlan ncogh...@gmail.com wrote: On 21 February 2014 22:42, Chris Angelico ros...@gmail.com wrote: People can already write: if (x if y else z): without the parens, and it works. Readability suffers when the same keyword is used twice (here if rather than the colon, but same difference), yet the parens are considered optional. Python is a language that, by and large, lacks syntactic salt; style guides are free to stipulate more, but the language doesn't make demands. I would strongly *recommend* using parens in all the cases you've shown, especially lambda: lambda x: calculate(x) except Exception: None lambda x: (calculate(x) except Exception: None) as it would otherwise depend on operator precedence; but mandating them feels to me like demanding readability. Right, that's why my main motivation for this suggestion is the one relating to keeping future options open. If the parentheses are optional, than adding multiple except clauses latter isn't possible, since this would already be valid, but mean something different: expr except Exception1: default1 except Exception2: default2 The deferral currently has this snippet: In order to ensure compatibility with future versions, ensure that any consecutive except operators are parenthesized to guarantee the interpretation you expect. That's not a reasonable expectation - either the parentheses have to be mandatory as part of the deferral, or else multiple except clause support needs to be listed as rejected rather than deferred. I've spent the better part of the last hour debating this in my head. It's basically a question of simplicity versus future flexibility: either keep the syntax clean and deny the multiple-except-clause option, or mandate the parens and permit it. The first option has, in my own head, the stronger case - this is designed for simplicity, and it wouldn't be that big a deal to completely reject multiple except clauses and simply require that the ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 1:59 AM, Chris Angelico ros...@gmail.com wrote: I've spent the better part of the last hour debating this in my head. It's basically a question of simplicity versus future flexibility: either keep the syntax clean and deny the multiple-except-clause option, or mandate the parens and permit it. The first option has, in my own head, the stronger case - this is designed for simplicity, and it wouldn't be that big a deal to completely reject multiple except clauses and simply require that the Oops, hit the wrong key and sent that half-written. ... and simply require that the statement form be used. But the whelming opinion of python-dev seems to be in favour of the parens anyway, and since they give us the possibility of future expansion effectively for free, I've gone that way. Parens are now required; the syntax is: value = (expr except Exception: default) and, as per genexp rules, redundant parens can be omitted: print(lst[i] except IndexError: Out of bounds) ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 1:50 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 22 Feb 2014 00:28:01 +1000 Nick Coghlan ncogh...@gmail.com wrote: Neither of these objections addresses the problems with the status quo, though: - the status quo encourages overbroad exception handling (as illustrated by examples in the PEP) I don't get this. Using the proper exception class in a try...except suite is no more bothersome than using the proper exception class in this newly-proposed construct. Overbroad exception handling comes in two ways. One is simply catching Exception or BaseException when a narrower class would be better, and that's not addressed by this PEP (except insofar as it does not have a bare except: syntax, and so it forces you to at least be explicit about catching BaseException). The other is this: try: f(x[i]) except IndexError: f(default) Translating that into this form: f(x[i] except IndexError: default) means that an IndexError thrown inside f() will not be caught. While it is, of course, possible to write that currently: try: arg = x[i] except IndexError: arg = default f(arg) it's that much less readable, and as evidenced by several examples in the standard library (some of which are in the PEP, and a few more can be found in my associated collection [1]), it's common to short-cut this. By providing a clean and convenient syntax for catching an exception in just one small expression, we encourage the narrowing of the catch front. ChrisA [1] https://github.com/Rosuav/ExceptExpr/blob/master/examples.py ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 1:03 AM, Eli Bendersky eli...@gmail.com wrote: Chris, while I also commend you for the comprehensive PEP, I'm -1 on the proposal, for two main reasons: 1. Many proposals suggest new syntax to gain some succinctness. Each has to be judged for its own merits, and in this case IMHO the cons eclipse the pros. I don't think this will save a lot of code in a typical well-structured program - maybe a few lines out of hundreds. On the other hand, it adds yet another syntax to remember and understand, which is not the Pythonic way. It doesn't need to save a huge number of lines. Just like lambda and the if/else expression, it's there as a tool - if it makes your code easier to read, it's a good too, and if it makes it harder, then don't use it. Yes, it's more to learn, but so is the proliferation of ad-hoc alternatives, several of which are listed in the 'Motivation' section at the top. 2. Worse, this idea subverts exceptions to control flow, which is not only un-Pythonic but also against the accepted practices of programming in general. Here, the comparison to PEP 308 is misguided. PEP 308, whatever syntax it adds, still remains within the domain of normal control flow. PEP 463, OTOH, makes it deliberately easy to make exceptions part of non-exceptional code, encouraging very bad programming practices. Again, I point to the current alternatives, including: * dict[key] - dict.get(key, default) * min(sequence) - min(sequence, default=default), as of 3.4 Both offer a way to either get an exception back or use a (usually constant) default value. In each case, the author of the class/function had to cater to the fact that some callers might want it to not raise an exception. The alternative is to always use the full try/except block, which leads to questions like How can I keep going after an exception? with code like this: try: spam = d[spam] ham = d[ham] eggs = d[eggs] sausage = d[sausage] except KeyError: thing_that_failed = The dict class offers a way to avoid the exception here, by showing that it's non-exceptional: spam = d.get(spam,) ham = d.get(ham,) eggs = d.get(eggs,) sausage = d.get(sausage,) But not everything does. Writing that with just exception handling looks like this: try: spam = d[spam] except KeyError: span = try: ham = d[ham] except KeyError: ham = try: eggs = d[eggs] except KeyError: eggs = try: sausage = d[sausage] except KeyError: sausage = with extreme likelihood of an error - do you see what I got wrong there? With inline exception handling, d could be a custom class that simply defines [...] to raise KeyError on unknowns, and the code can be written thus: spam = d[spam] except KeyError: ham = d[ham] except KeyError: eggs = d[eggs] except KeyError: sausage = d[sausage] except KeyError: It's still a bit repetitive, but that's hard to avoid. And it puts the exception handling at the exact point where it stops being exceptional and starts being normal - exactly as the try/except statement should. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 1:22 AM, Rob Cliffe rob.cli...@btinternet.com wrote: Small point: in one of your examples you give a plug for the PEP note the DRY improvement. I would suggest that similarly perhaps in your Lib/tarfile.py:2198 example you point out the increase in readability due to the 2 lines lining up in your Lib/ipaddress.py:343 example you point out that the new form is probably an improvement (i.e. probably closer to the author's intention) as it will NOT catch an AttributeError evaluating ips.append (this would matter e.g. if append were mis-spelt). YOU are clearly aware of this but it would often escape the casual reader. Sure. Added a paragraph down the bottom of the section explaining the benefit of the narrowed scope. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 1:34 AM, Brett Cannon br...@python.org wrote: While I like the general concept, I agree that it looks too much like a crunched statement; the use of the colon is a non-starter for me. I'm sure I'm not the only one whose brain has been trained to view a colon in Python to mean statement, period. This goes against that syntactic practice and just doesn't work for me. I'm -1 with the current syntax, but it can go into the + range if a better syntax can be chosen. We bikeshedded that extensively on -ideas. The four best options are: value = (expr except Exception: default) value = (expr except Exception - default) value = (expr except Exception pass default) value = (expr except Exception then default) Note that the last option involves the creation of a new keyword. Would any of the others feel better to you? ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 2:19 AM, Brett Cannon br...@python.org wrote:
I understand you are arguing that a try expression will lead to people just
doing `something() except Exception: None` or whatever and that people will
simply get lazy and not think about what they are doing with their
exceptions. Unfortunately they already are; that shipped sailed when we
didn't eliminate bare except clauses in Python 3 (hopefully we can change
that in Python 4).
That's already come up a few times. I wrote a script to try to find
potential conversion targets, and technically it's correct to show
this up:
Doc/tools/rstlint.py:73:
try:
compile(code, fn, 'exec')
except SyntaxError as err:
yield err.lineno, 'not compilable: %s' % err
Yes, technically you could treat that as an expression. Well, okay.
Current version of the proposal doesn't include 'as', so you can't do
that. Let's grab the next one.
Lib/asyncio/events.py:40:
try:
self._callback(*self._args)
except Exception:
logger.exception('Exception in callback %s %r',
self._callback, self._args)
That could, per PEP 463, become:
(self._callback(*self._args) except Exception:
logger.exception('Exception in callback %s %r',
self._callback, self._args))
But that would be abuse of the syntax. It's just like converting this:
for s in list_of_strings:
print(s)
into an expression:
[print(s) for s in list_of_strings]
list(map(print,list_of_strings))
Neither of them is a good way to write code, yet that's not the fault
of either map() or the list comp.
I'm totally fine restricting this proposal to not having any concept of
exception catching or finally clause: it just replaces the simplest
try/except clause possible (while requiring an exception be specified). That
takes care of the common control flow use case of exceptions while requiring
more thought for more complex cases.
The finally clause was never in the proposal. I added it as an open
issue early on, just to see if it gathered any interest, but the
weight of opinion agreed with my initial feeling: it's utterly useless
to the exception syntax. Same goes for an else clause. Currently, the
two complexities not possible in the expression form are capturing the
exception with 'as' (for technical reasons) and stringing multiple
except clauses on (for complexity reasons). In the standard library,
counting up all the cases that could plausibly be converted to
expression form, very very few use either feature. So this is a case
of do just the really simple option and catch 98.5% of the actual
usage.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 3:46 AM, Eric Snow ericsnowcurren...@gmail.com wrote: Be sure to capture in the PEP (within reason) a summary of concerns and rebuttals/acquiescence. Eli's, Brett's, and Antoine's concerns likely reflect what others are thinking as well. The PEP and its result will be better for recording such matters. In fact, that's a key goal of PEPs. :) I now have a paragraph summarizing the narrowing of scope benefit, though not worded as a rebuttal of a concern. Eli's concern is mainly that this can be abused, which everything can; should I include something about that? Brett's issue is with the colon. I'll write up a paragraph or two of reasoning as to why that was selected, possibly tomorrow, but the main reason is simply that nobody came up with a compelling argument in favour of any other. Hardly the most epic tale :) ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 4:32 AM, Antoine Pitrou solip...@pitrou.net wrote: value = (expr except Exception then default) +0.5 But I'm aware it requires reserving then as a keyword, which might need a prior SyntaxWarning. There are no instances of then used as a name in the Python stdlib, I already checked. If anyone has a good-sized codebase to check, you're welcome to use the same analysis script - it's an AST parser/walker, so it finds actual use as a name, not in a comment or a string or anything. https://github.com/Rosuav/ExceptExpr/blob/master/find_except_expr.py ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
of where did _attrs come from, and got bogged down. Does quacks like a dict have to include a .get() method? Translate numbers to names, falling back on the numbers:: g = grp.getgrnam(tarinfo.gname)[2] except KeyError: tarinfo.gid u = pwd.getpwnam(tarinfo.uname)[2] except KeyError: tarinfo.uid # Lib/tarfile.py:2198: try: g = grp.getgrnam(tarinfo.gname)[2] except KeyError: g = tarinfo.gid try: u = pwd.getpwnam(tarinfo.uname)[2] except KeyError: u = tarinfo.uid This one is a valid example, but totally unparseable by humans. Moreover, it promotes a bad pattern, as you mask KeyErrors in 'grp.getgrnam(tarinfo.gname)' call. My translation masks nothing that the original didn't mask. The KeyError will come from the function call; it would be IndexError if the function returns a too-short tuple, and that one's allowed to bubble up. import pwd pwd.getpwnam(rosuav) pwd.struct_passwd(pw_name='rosuav', pw_passwd='x', pw_uid=1000, pw_gid=1000, pw_gecos='Chris Angelico,,,', pw_dir='/home/rosuav', pw_shell='/bin/bash') pwd.getpwnam(spam) Traceback (most recent call last): File stdin, line 1, in module KeyError: 'getpwnam(): name not found: spam' (Note that it's possible for 'import pwd' to fail, in which case pwd is set to None early in the script. But this particular bit of code checks if pwd before continuing anyway, so we don't expect AttributeError here.) Calculate the mean of a series of numbers, falling back on zero:: value = statistics.mean(lst) except statistics.StatisticsError: 0 try: value = statistics.mean(lst) except statistics.StatisticsError: value = 0 I think all of the above more readable with try statement. Readability is a matter of personal preference, to some extent. I find it clearer to use the shorter form, for the same reason as I'd use this: def query(prompt, default): return input(%s [%s]: %(prompt, default)) or default I wouldn't use long-hand: def query(prompt, default): s = input(%s [%s]: %(prompt, default)) if not s: s = default return s It's easier to see that it's calling something, and defaulting to something else. Retrieving a message from either a cache or the internet, with auth check:: logging.info(Message shown to user: %s,((cache[k] except LookupError: (backend.read(k) except OSError: 'Resource not available') ) if check_permission(k) else 'Access denied' ) except BaseException: This is like a bare except clause) try: if check_permission(k): try: _ = cache[k] except LookupError: try: _ = backend.read(k) except OSError: _ = 'Resource not available' else: _ = 'Access denied' except BaseException: _ = This is like a bare except clause logging.info(Message shown to user: %s, _) If you replace '_' with a 'msg' (why did you use '_'??) then try statements are *much* more readable. I've removed that example. The reason for using _ was because I wanted it to have the feel of still being an expression, where nothing's named. But it's not a very helpful example anyway; part of the confusion comes from the if/else in the middle, which completely wrecks evaluation order expectations. Lib/ipaddress.py:343:: try: ips.append(ip.ip) except AttributeError: ips.append(ip.network_address) Becomes:: ips.append(ip.ip except AttributeError: ip.network_address) or it may become: ips.append(getattr(ip, 'ip', ip.network_address)) or address = getattr(ip, 'ip', ip.network_address) ips.append(address) There's a subtle difference here that makes that not equivalent. With the original try/except statement, evaluation proceeds thus: 1) Attempt to look up ip.ip. If that succeeds, call ips.append(). 2) If AttributeError is not thrown in #1, done. Otherwise, proceed. 3) Attempt to look up ip.network_address. If that succeeds, call ips.append. 4) Any exception raised will propagate. This means that, if neither ip nor network_address is defined, an AttributeError will come up, but that if ip is, network_address won't even be looked at. My version narrows the scope slightly, but is functionally similar. 1) Attempt to look up ip.ip. 2) If AttributeError is thrown in #1, attempt to look up ip.network_address. 3) If either #1 or #2 succeeds, call ips.append. 4) Any exception raised anywhere else will propagate. Your version, however, inverts the evaluation order: 1) Attempt to look up ip.network_address 2) If AttributeError is thrown in #1, propagate it up and stop evaluating. 3) Retrieve ip.ip, defaulting to the looked-up network
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 9:06 AM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Nick Coghlan wrote: On 21 February 2014 13:15, Chris Angelico ros...@gmail.com wrote: Generator expressions require parentheses, unless they would be strictly redundant. Ambiguities with except expressions could be resolved in the same way, forcing nested except-in-except trees to be correctly parenthesized I'd like to make the case that the PEP should adopt this as its default position. I generally agree, but I'd like to point out that this doesn't necessarily mean making the parenthesizing rules as strict as they are for generator expressions. The starting point for genexps is that the parens are part of the syntax, the same way that square brackets are part of the syntax of a list comprehension; we only allow them to be omitted in very special circumstances. On the other hand, I don't think there's any harm in allowing an except expression to stand on its own when there is no risk of ambiguity, e.g. foo = things[i] except IndexError: None should be allowed, just as we allow x = a if b else c and don't require x = (a if b else c) I'm inclined to agree with you. I fought against the mandated parens for a long time. Would be happy to un-mandate them, as long as there's no way for there to be ambiguity. Is CPython able to make an operator non-associative? That is, to allow these: value = expr except Exception: default value = (expr except Exception: default) except Exception: default value = expr except Exception: (default except Exception: default) but not this: value = expr except Exception: default except Exception: default ? By effectively demanding parens any time two except-expressions meet, we could leave the option open to read multiple clauses off a single base exception. If that can be done, and if people's need for clarity can be satisfied, and if the rules aren't too complicated, I'd be happy to go back to parens are optional. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 10:29 AM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Antoine Pitrou wrote: lst = [1, 2] value = lst[2] except IndexError: No value the gain in concision is counterbalanced by a loss in readability, This version might be more readable: value = lst[2] except No value if IndexError since it puts the normal and exceptional values next to each other, and relegates the exception type (which is of much less interest) to a trailing aside. I'll expand on this in the PEP shortly, but there are two downsides to this. 1) Everywhere else in Python, if is followed by a boolean, and except is followed by an exception list. As a boolean, IndexError is always going to be true, which will confuse a human; and No value doesn't look like a modern exception at all. 2) Order of evaluation puts the main expression first, then the exception list, and lastly the default. Putting them in another order in the code is confusing. With ternary if/else, this is justified, but it's usually something to avoid. 3) Erm, among the downsides are such diverse elements... ahem. The part after the except clause is a full expression, and could plausibly have the if/else ternary operator. Starting with some expression, then if, then another expression, means you have to look for an else before you can be sure you're reading it correctly. Of course, any sane programmer who actually puts an if/else inside an except/if should parenthesize, but when you read someone else's code, you have to be prepared for the more normal sort of programmer. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 12:10 PM, Glenn Linderman v+pyt...@g.nevcal.com wrote: Here's a challenge: There has been a big thread about None versus (SQL) Null. Show how an except: expression can help the DB API more easily convert from using None to using a new Null singleton, and you'll have a winner :) Heh! I'm not entirely sure how that would work, as I've only skimmed the DB API thread, but I understand part of it is to do with sorting. I'm not sure how you could embed an except-expression into that without either changing the sort function or potentially doing part of the sort twice: lst = sorted(stuff) except TypeError: sorted(stuff, key=keyfunc) at which point you may as well go straight to the key= form from the start. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 12:55 PM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Steven D'Aprano wrote: result = computation( int(arg) except ValueError: abort(Invalid int) ) Actually, not quite so nice as I first thought, since you're relying on the side-effects of abort() rather than returning a value. Yeah, while I was writing that I wondered whether you should be allowed to write int(arg) except ValueError: raise UserError(Invalid int) That looks heretical, because 'raise' can't in any way be interpreted as a value-returning expression. But you can achieve the same result using a function that always raises and exception, so forbidding it on those grounds would be pointless. def throw(exc): raise exc int(arg) except ValueError: throw(UserError(Invalid int)) Tiny helper function and then it doesn't need any special syntax. It's not so much that PEP 463 forbids this, as that it doesn't explicitly permit it. The 'default' part of the syntax is an expression, raise is not an expression, ergo it's not permitted. But if you think this is sufficiently common (translate an exception on the way through), show some code and justify its addition - or just write up a separate proposal for raise X to become an expression, same as yield X is. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 1:07 PM, Victor Stinner victor.stin...@gmail.com wrote: At the first read, I'm unable to understand this long expression. At the second read, I'm still unable to see which instruction will be executed first: lvl1[key] or lvl2[key]? The advantage of the current syntax is that the control flow is obvious, from the top to the bottom: # start try: x = lvl1[key] # first instruction except KeyError: try: x = lvl2[key] except KeyError: x = f(key) # latest instruction # end That's why I'm strongly in favour of syntax variants that have evaluation order be equally obvious: left to right. Its notation may be uglier, but C's ternary operator does get this right, where Python's executes from the inside out. It's not a big deal when most of it is constants, but it can help a lot when the expressions nest. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 3:04 PM, Chris Angelico ros...@gmail.com wrote: On Sat, Feb 22, 2014 at 1:07 PM, Victor Stinner victor.stin...@gmail.com wrote: At the first read, I'm unable to understand this long expression. At the second read, I'm still unable to see which instruction will be executed first: lvl1[key] or lvl2[key]? The advantage of the current syntax is that the control flow is obvious, from the top to the bottom: # start try: x = lvl1[key] # first instruction except KeyError: try: x = lvl2[key] except KeyError: x = f(key) # latest instruction # end That's why I'm strongly in favour of syntax variants that have evaluation order be equally obvious: left to right. Its notation may be uglier, but C's ternary operator does get this right, where Python's executes from the inside out. It's not a big deal when most of it is constants, but it can help a lot when the expressions nest. I've added a couple of paragraphs to my draft PEP: https://raw.github.com/Rosuav/ExceptExpr/master/pep-0463.txt If someone could please commit that version to the official repo? Or I can submit a diff against the peps repo if that would be easier. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 4:01 PM, Steven D'Aprano st...@pearwood.info wrote: (Chris, I think that ought to go in the motivation section of the PEP.) Added to my draft, and here's the peps diff: diff -r c52a2ae3d98e pep-0463.txt --- a/pep-0463.txt Fri Feb 21 23:27:51 2014 -0500 +++ b/pep-0463.txt Sat Feb 22 16:33:37 2014 +1100 @@ -43,6 +43,34 @@ * statistics.mean(data) - no way to handle an empty iterator +Had this facility existed early in Python's history, there would have been +no need to create dict.get() and related methods; the one obvious way to +handle an absent key would be to respond to the exception. One method is +written which signal the absence in one way, and one consistent technique +is used to respond to the absence. Instead, we have dict.get(), and as of +Python 3.4, we also have min(... default=default), and myriad others. We +have a LBYL syntax for testing inside an expression, but there is currently +no EAFP notation; compare the following:: + +# LBYL: +if key in dic: +process(dic[key]) +else: +process(None) +# As an expression: +process(dic[key] if key in dic else None) + +# EAFP: +try: +process(dic[key]) +except KeyError: +process(None) +# As an expression: +process(dic[key] except KeyError: None) + +Python generally recommends the EAFP policy, but must then proliferate +utility functions like dic.get(key,None) to enable this. + Rationale = ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 8:20 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 22 Feb 2014 16:12:27 +0900 Stephen J. Turnbull step...@xemacs.org wrote: Note in support: I originally thought that get methods would be more efficient, but since Nick pointed out that haveattr is implemented by catching the exception (Yikes! LBYL implemented by using EAFP!), I assume that get methods also are (explicitly or implicitly) implemented that way. Well, the only way to know that a key (or attribute) exists is to do the lookup. What else would you suggest? And, yes, EAFP can avoid race conditions and the like (besides being more efficient with non-trivial keys). Which means that, fundamentally, EAFP is the way to do it. So if PEP 463 expressions had existed from the beginning, hasattr() probably wouldn't have been written - people would just use an except-expression instead. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tangent on class level scoping rules (was Re: PEP 463: Exception-catching expressions)
On Sat, Feb 22, 2014 at 10:57 AM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Nick Coghlan wrote: As Chris later noted, you likely *could* still implement expression local name binding for an except expression without a full closure, it would just be rather difficult. I'm still not convinced it would be all *that* difficult. Seems to me it would be semantically equivalent to renaming the inner variable and adding a finally clause to unbind it. Is there something I'm missing? An inner scope should shadow rather than unbinding. Ideally: spam = Initial spam try: 1/0 except Exception as spam: assert isinstance(spam, Exception) assert [spam for spam in [List Comp]][0] == List Comp with open(test,w) as spam: assert hasattr(spam,write) assert (lambda spam: spam)(Function) == Function assert spam == Initial spam Currently, the list comp and lambda work that way. The exception will unbind the name, not changing the scope at all but preventing refloop after exit. The 'with... as' works in the same scope and leaves the name bound, which is a bit surprising in some cases (you usually end up with a closed file object, or whatever it be, lurking around). A clean inner-scope concept would solve all of this. The lambda would still be a closure, because you can pass that around. All the others would be inner scopes - shadowing cleanly and then disappearing. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Sat, Feb 22, 2014 at 8:58 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 22 Feb 2014 20:29:27 +1100 Chris Angelico ros...@gmail.com wrote: Which means that, fundamentally, EAFP is the way to do it. So if PEP 463 expressions had existed from the beginning, hasattr() probably wouldn't have been written - people would just use an except-expression instead. Really? hasattr() is much easier to write than the corresponding except-expression. But would it be sufficiently easier to justify the creation of a built-in? Imagine this were the other way around: we have except-expressions, but we don't have hasattr. Now someone comes onto python-ideas and says, Wouldn't it be nice if we had a hasattr() function to tell us whether something has an attribute or not. Considering that hasattr can be easily implemented using an except-expression, it would be unlikely to be considered worthy of a built-in. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
