Re: [Python-Dev] Add transform() and untranform() methods

On 15.11.2013 08:13, Nick Coghlan wrote: > On 15 November 2013 11:10, Terry Reedy wrote: >> On 11/14/2013 5:32 PM, Victor Stinner wrote: >> >>> I don't like the functions codecs.encode() and codecs.decode() because >>> the type of the result depends on the encoding (second parameter). We >>> try t

Re: [Python-Dev] Add transform() and untranform() methods

On Fri, 15 Nov 2013 09:03:37 +1000 Nick Coghlan wrote: > > > And add transform() and untransform() methods to bytes and str types. > > In practice, it might be same codecs registry for all codecs just with > > a new attribute. > > This is completely the wrong approach. There's zero justification

[Python-Dev] Finding overlapping matches with re assertions: bug or feature?

I was surprised to find that "this works": if you want to find all _overlapping_ matches for a regexp R, wrap it in (?=(R)) and feed it to (say) finditer. Here's a very simple example, finding all overlapping occurrences of "xx": pat = re.compile("(?=(xx))") for it in pat.finditer

Re: [Python-Dev] Finding overlapping matches with re assertions: bug or feature?

On 15 November 2013 06:48, Tim Peters wrote: > Is that a feature? Or an accident? It's very surprising to find a > non-empty match inside an empty match (the outermost lookahead > assertion). Personally, I would read (?=(R))" as finding an empty match at a point where R starts. There's no impli

Re: [Python-Dev] Assign(expr* targets, expr value) - why targetS?

On Tue, Nov 12, 2013 at 5:08 PM, Benjamin Peterson wrote: > 2013/11/12 anatoly techtonik : >> On Sun, Nov 10, 2013 at 8:34 AM, Benjamin Peterson >> wrote: >>> 2013/11/10 anatoly techtonik : http://hg.python.org/cpython/file/1ee45eb6aab9/Parser/Python.asdl In Assign(expr* targets,

Re: [Python-Dev] Assign(expr* targets, expr value) - why targetS?

On Fri, Nov 15, 2013 at 12:54 PM, anatoly techtonik wrote: > On Tue, Nov 12, 2013 at 5:08 PM, Benjamin Peterson > wrote: >> 2013/11/12 anatoly techtonik : >>> On Sun, Nov 10, 2013 at 8:34 AM, Benjamin Peterson >>> wrote: 2013/11/10 anatoly techtonik : > http://hg.python.org/cpython/fi

Re: [Python-Dev] Add transform() and untranform() methods

On Fri, Nov 15, 2013 at 05:13:34PM +1000, Nick Coghlan wrote: > A few things I noticed while implementing the recent updates: > > - as you noted in your other email, while MAL is on record as saying > the codecs module is intended for arbitrary codecs, not just Unicode > encodings, readers of the

Re: [Python-Dev] Add transform() and untranform() methods

15.11.13 12:02, Steven D'Aprano написав(ла): It would be really good to be able to query the available codecs. For example, many applications offer an "Encoding" menu, where you can specify the codec used for text. That's hard in Python, since you can't retrieve a list of known codecs. And you

Re: [Python-Dev] Add transform() and untranform() methods

On Fri, Nov 15, 2013 at 10:22:28AM +0100, Antoine Pitrou wrote: > On Fri, 15 Nov 2013 09:03:37 +1000 Nick Coghlan wrote: > > > > > And add transform() and untransform() methods to bytes and str types. > > > In practice, it might be same codecs registry for all codecs just with > > > a new attribu

Re: [Python-Dev] Add transform() and untranform() methods

On Fri, 15 Nov 2013 21:28:35 +1100 Steven D'Aprano wrote: > > One benefit is: > > import codecs > codec = get_name_of_compression_codec() > result = codecs.encode(data, codec) That's a good point. > If encoding/decoding is intended to be completely generic (even if 99% > of the uses will be w

Re: [Python-Dev] Add transform() and untranform() methods

15.11.13 12:28, Steven D'Aprano написав(ла): One benefit is: import codecs codec = get_name_of_compression_codec() result = codecs.encode(data, codec) And this is a hole in a security if you don't check codec name before calling a codec. See topic about utilizing zip-bombs via codecs machiner

Re: [Python-Dev] unicode Exception messages in py2.7

Hi, FWIW, the pure Python traceback.py module has a slightly different (and saner) behavior: >>> e = Exception(u"xx\u1234yy") >>> traceback.print_exception(Exception, e, None) Exception: xx\u1234yy I'd suggest that the behavior of the two should be unified anyway. The traceback module uses value

Re: [Python-Dev] [Python-checkins] cpython: Close #17828: better handling of codec errors

On 15 November 2013 17:22, Stefan Behnel wrote: > > I can't see any bit of information being added by chaining the exceptions > in this specific case. > > Remember that each change to exception messages and/or exception chaining > will break someone's doctests somewhere, and it's really ugly to wo

Re: [Python-Dev] Add transform() and untranform() methods

On 15 November 2013 20:33, Antoine Pitrou wrote: > On Fri, 15 Nov 2013 21:28:35 +1100 > Steven D'Aprano wrote: >> >> One benefit is: >> >> import codecs >> codec = get_name_of_compression_codec() >> result = codecs.encode(data, codec) > > That's a good point. > >> If encoding/decoding is intended

Re: [Python-Dev] Add transform() and untranform() methods

On Fri, 15 Nov 2013 21:45:31 +1000 Nick Coghlan wrote: > > The reason I'm now putting some effort into better documenting the > status quo for codec handling in Python 3 and filing off some of the > rough edges (rather than proposing adding any new APIs to Python 3.x) > is because the users I car

Re: [Python-Dev] Add transform() and untranform() methods

2013/11/15 Nick Coghlan : > The reason I'm now putting some effort into better documenting the > status quo for codec handling in Python 3 and filing off some of the > rough edges (rather than proposing adding any new APIs to Python 3.x) > is because the users I care about in this matter are web de

Re: [Python-Dev] Add transform() and untranform() methods

On 15 November 2013 12:07, Victor Stinner wrote: >> A new API for binary transforms is potentially an academically >> interesting concept, but it solves zero current real world problems. > > I would like to reply the same for these codecs: they are not solving > any real world problem :-) As Nick

Re: [Python-Dev] Add transform() and untranform() methods

On 15.11.2013 12:45, Nick Coghlan wrote: > On 15 November 2013 20:33, Antoine Pitrou wrote: >> On Fri, 15 Nov 2013 21:28:35 +1100 >> Steven D'Aprano wrote: >>> >>> One benefit is: >>> >>> import codecs >>> codec = get_name_of_compression_codec() >>> result = codecs.encode(data, codec) >> >> That'

Re: [Python-Dev] Add transform() and untranform() methods

On Thu, Nov 14, 2013 at 7:32 PM, Victor Stinner wrote: > I would prefer to split the registry of codecs to have 3 registries: > > - "encoding" (a better name can found): encode str=>bytes, decode bytes=>str > - bytes: encode bytes=>bytes, decode bytes=>bytes > - str: encode str=>str, decode str=

Re: [Python-Dev] unicode Exception messages in py2.7

Am 15.11.13 00:57, schrieb Chris Barker: > Maybe so -- but we are either maintaining 2.7 or not -- it WIL be > around for along time yet... Procedurally, it's really easy. Ultimately it's up to the release manager to decide which changes go into a release and which don't, and Benjamin has already

Re: [Python-Dev] Add transform() and untranform() methods

On 15 November 2013 22:24, Paul Moore wrote: > On 15 November 2013 12:07, Victor Stinner wrote: >>> A new API for binary transforms is potentially an academically >>> interesting concept, but it solves zero current real world problems. >> >> I would like to reply the same for these codecs: they a

Re: [Python-Dev] Add transform() and untranform() methods

On Fri, 15 Nov 2013 23:50:23 +1000 Nick Coghlan wrote: > > My perspective is that, in current Python, that *is* the right thing > for people to do, and any hypothetical new API proposed for Python 3.5 > would do nothing to change what's right for Python 3.4 code (or Python > 2/3 compatible code).

Re: [Python-Dev] Assign(expr* targets, expr value) - why targetS?

2013/11/15 anatoly techtonik : > On Tue, Nov 12, 2013 at 5:08 PM, Benjamin Peterson > wrote: >> 2013/11/12 anatoly techtonik : >>> On Sun, Nov 10, 2013 at 8:34 AM, Benjamin Peterson >>> wrote: 2013/11/10 anatoly techtonik : > http://hg.python.org/cpython/file/1ee45eb6aab9/Parser/Python

Re: [Python-Dev] Add transform() and untranform() methods

On 16 November 2013 00:04, Antoine Pitrou wrote: >> Rather than the more useful: >> >> >>> b"abcdef".decode("hex") >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: 'hex' decoder returned 'bytes' instead of 'str'; use >> codecs.decode() to decode to arbitrary types > >

Re: [Python-Dev] Add transform() and untranform() methods

Walter Dörwald writes: > Am 15.11.2013 um 00:42 schrieb Serhiy Storchaka : > > > > 15.11.13 00:32, Victor Stinner написав(ла): > >> And add transform() and untransform() methods to bytes and str types. > >> In practice, it might be same codecs registry for all codecs just with > >> a new att

Re: [Python-Dev] Add transform() and untranform() methods

On 11/14/2013 11:13 PM, Nick Coghlan wrote: The proposal I posted to issue 7475 back in April (and, in the absence of any objections to the proposal, finally implemented over the past few weeks) was to take advantage of the fact that the codecs.encode and codecs.decode convenience functions exis

Re: [Python-Dev] Add transform() and untranform() methods

On Sat, 16 Nov 2013 00:46:15 +1000 Nick Coghlan wrote: > On 16 November 2013 00:04, Antoine Pitrou wrote: > >> Rather than the more useful: > >> > >> >>> b"abcdef".decode("hex") > >> Traceback (most recent call last): > >> File "", line 1, in > >> TypeError: 'hex' decoder returned 'bytes' inst

[Python-Dev] Summary of Python tracker Issues

ACTIVITY SUMMARY (2013-11-08 - 2013-11-15) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open4265 (+38) closed 27119 (+45) total 31384 (+83) Open issues wit

Re: [Python-Dev] The pysandbox project is broken

On Tue, Nov 12, 2013 at 01:16:55PM -0800, Victor Stinner wrote: > pysandbox cannot be used in practice > > > To protect the untrusted namespace, pysandbox installs a lot of > different protections. Because of all these protections, it becomes > hard to write Py

Re: [Python-Dev] unicode Exception messages in py2.7

Hi again, I figured that even using the traceback.py module and getting "Exception: \u1234\u1235\u5321" is rather useless if you tried to raise an exception with a message in Thai. I believe this to also be a bug, so I opened https://bugs.pypy.org/issue1634 . According to this thread, however, p

Re: [Python-Dev] The pysandbox project is broken

2013/11/15 Trent Nelson : > This sounds a lot like the work I initially did with PyParallel to > try and intercept/prevent parallel threads mutating main-thread > objects. > > I ended up arriving at a much better solution by just relying on > memory protection; main thread pages

Re: [Python-Dev] unicode Exception messages in py2.7

On Fri, Nov 15, 2013 at 9:21 AM, Armin Rigo wrote: > I figured that even using the traceback.py module and getting > "Exception: \u1234\u1235\u5321" is rather useless if you tried to > raise an exception with a message in Thai. yup. > I believe this to also be > a bug, so I opened https://bugs.

Re: [Python-Dev] The pysandbox project is broken

On Nov 15, 2013, at 12:34 PM, Victor Stinner wrote: > 2013/11/15 Trent Nelson : >>This sounds a lot like the work I initially did with PyParallel to >>try and intercept/prevent parallel threads mutating main-thread >>objects. >> >>I ended up arriving at a much better solution by

Re: [Python-Dev] unicode Exception messages in py2.7

On Fri, Nov 15, 2013 at 12:41 PM, Chris Barker wrote: > On Fri, Nov 15, 2013 at 9:21 AM, Armin Rigo wrote: > > > I figured that even using the traceback.py module and getting > > "Exception: \u1234\u1235\u5321" is rather useless if you tried to > > raise an exception with a message in Thai. > > y

Re: [Python-Dev] unicode Exception messages in py2.7

On Fri, Nov 15, 2013 at 5:24 AM, "Martin v. Löwis" wrote: > Procedurally, it's really easy. Ultimately it's up to the release > manager to decide which changes go into a release and which don't, and > Benjamin has already voiced an opinion. Very early in the conversation, though honestly, probab

Re: [Python-Dev] Add transform() and untranform() methods

Am 15.11.2013 um 16:57 schrieb "Stephen J. Turnbull" : > > Walter Dörwald writes: >>> Am 15.11.2013 um 00:42 schrieb Serhiy Storchaka : >>> >>> 15.11.13 00:32, Victor Stinner написав(ла): And add transform() and untransform() methods to bytes and str types. In practice, it might be same

Re: [Python-Dev] Finding overlapping matches with re assertions: bug or feature?

[Tim] >> Is that a feature? Or an accident? It's very surprising to find a >> non-empty match inside an empty match (the outermost lookahead >> assertion). [Paul Moore] > Personally, I would read (?=(R))" as finding an empty match at a point > where R starts. There's no implication that R is in

Re: [Python-Dev] cpython: Issue #19544 and Issue #6516: Restore support for --user and --group parameters

Am 15.11.2013 19:07, schrieb jason.coombs: > http://hg.python.org/cpython/rev/b9c9c4b2effe > changeset: 87119:b9c9c4b2effe > user:Andrew Kuchling > date:Fri Nov 15 13:01:52 2013 -0500 > summary: > Issue #19544 and Issue #6516: Restore support for --user and --group > parameter

Re: [Python-Dev] The pysandbox project is broken

On 13/11/13 00:49, Josiah Carlson wrote: Python-dev is for the development of the Python core language, the CPython runtime, and libraries. Your sandbox, despite using and requiring deep knowledge of the runtime, is not developing those things. If you had a series of requests for the language

Re: [Python-Dev] unicode Exception messages in py2.7

Armin Rigo wrote: I figured that even using the traceback.py module and getting "Exception: \u1234\u1235\u5321" is rather useless if you tried to raise an exception with a message in Thai. But at least it tells you that *something* went wrong, and points to the place in the code where it happen

Re: [Python-Dev] Add transform() and untranform() methods

On 16 Nov 2013 02:36, "Antoine Pitrou" wrote: > > On Sat, 16 Nov 2013 00:46:15 +1000 > Nick Coghlan wrote: > > On 16 November 2013 00:04, Antoine Pitrou wrote: > > >> Rather than the more useful: > > >> > > >> >>> b"abcdef".decode("hex") > > >> Traceback (most recent call last): > > >> File ""

Re: [Python-Dev] The pysandbox project is broken

On 16 Nov 2013 08:25, "Christian Tismer" wrote: > > On 13/11/13 00:49, Josiah Carlson wrote: >> >> >> Python-dev is for the development of the Python core language, the CPython runtime, and libraries. Your sandbox, despite using and requiring deep knowledge of the runtime, is not developing those

Re: [Python-Dev] The pysandbox project is broken

On Fri, Nov 15, 2013 at 4:31 PM, Nick Coghlan wrote: > "Use an OS level sandbox" *is* better from a security point of view. It's > just not portable :P > Honestly, I don't believe in portable security. :-) BTW, in case it wasn't clear, I think it was a courageous step by Victor to declare defea

Re: [Python-Dev] Add transform() and untranform() methods

2013/11/16 Nick Coghlan : > To address Serhiy's security concerns with the compression codecs (which are > technically independent of the question of restoring the aliases), I also > plan to document how to systematically blacklist particular codecs in an > application by setting attributes on the

Re: [Python-Dev] The pysandbox project is broken

On 16.11.13 01:35, Guido van Rossum wrote: On Fri, Nov 15, 2013 at 4:31 PM, Nick Coghlan > wrote: "Use an OS level sandbox" *is* better from a security point of view. It's just not portable :P Honestly, I don't believe in portable security. :-) BTW, in case

[Python-Dev] (#19562) Asserts in Python stdlib code (datetime.py)

http://bugs.python.org/issue19562 propose to change the first assert in Lib/datetime.py assert 1 <= month <= 12, month to assert 1 <= month <= 12,'month must be in 1..12' to match the next two asserts out of the *53* in the file. I think that is the wrong direction of change, but that is not

Re: [Python-Dev] (#19562) Asserts in Python stdlib code (datetime.py)

[Terry Reedy] > Should stdlib code use assert at all? Of course, and for exactly the same reasons we use `assert()` in Python's C code: to verify preconditions, postconditions, and invariants that should never fail. Assertions should never be used to, e.g., verify user-supplied input (or anythin

Re: [Python-Dev] (#19562) Asserts in Python stdlib code (datetime.py)

> Should stdlib code use assert at all? > > If user input can trigger an assert, then the code should raise a normal > exception that will not disappear with -OO. > > If the assert is testing program logic, then it seems that the test belongs > in the test file, in this case, test/test_datetime.py.

Re: [Python-Dev] The pysandbox project is broken

On 11/15/2013 02:24 PM, Christian Tismer wrote: I appreciate very much that Victor tried his best to fill that old gap. And after that breakage happened again, I think it is urgent to have an > in-depth discussion how that situation should be treated in the > future. +1 -- ~Ethan~ __