Re: [Python-Dev] What does a double coding cookie mean?

2016-03-20 Thread Guido van Rossum
On Thu, Mar 17, 2016 at 9:50 AM, Serhiy Storchaka wrote: > On 17.03.16 16:55, Guido van Rossum wrote: >> >> On Thu, Mar 17, 2016 at 5:04 AM, Serhiy Storchaka >> wrote: Should we recommend that everyone use tokenize.detect_encoding()? >>> >>>

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-20 Thread Ethan Furman
On 03/17/2016 04:54 PM, Glenn Linderman wrote: On 3/16/2016 12:59 AM, Serhiy Storchaka wrote: Actually "must match the regular expression" is not correct, because re.match() implies anchoring at the start. I have proposed more correct regular expression in other branch of this thread.

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-20 Thread Nick Coghlan
On 20 March 2016 at 07:46, Glenn Linderman wrote: > Diagnosing ambiguous conditions, even including my example above, might be > useful... for a few files... is it worth the effort? What % of .py sources > have coding specifications? What % of those have two? And there's a

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-20 Thread M.-A. Lemburg
On 17.03.2016 15:02, Serhiy Storchaka wrote: > On 17.03.16 15:14, M.-A. Lemburg wrote: >> On 17.03.2016 01:29, Guido van Rossum wrote: >>> Should we recommend that everyone use tokenize.detect_encoding()? >> >> I'd prefer a separate utility for this somewhere, since >> tokenize.detect_encoding()

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Serhiy Storchaka
On 17.03.16 15:14, M.-A. Lemburg wrote: On 17.03.2016 01:29, Guido van Rossum wrote: Should we recommend that everyone use tokenize.detect_encoding()? I'd prefer a separate utility for this somewhere, since tokenize.detect_encoding() is not available in Python 2. I've attached an example

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Glenn Linderman
On 3/16/2016 12:59 AM, Serhiy Storchaka wrote: On 16.03.16 09:46, Glenn Linderman wrote: On 3/16/2016 12:09 AM, Serhiy Storchaka wrote: On 16.03.16 08:34, Glenn Linderman wrote: From the PEP 263: More precisely, the first or second line must match the regular expression

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Glenn Linderman
On 3/19/2016 2:37 PM, Serhiy Storchaka wrote: On 19.03.16 19:36, Glenn Linderman wrote: On 3/19/2016 8:19 AM, Serhiy Storchaka wrote: On 16.03.16 08:03, Serhiy Storchaka wrote: I just tested with Emacs, and it looks that when specify different codings on two different lines, the first coding

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Serhiy Storchaka
On 19.03.16 19:36, Glenn Linderman wrote: On 3/19/2016 8:19 AM, Serhiy Storchaka wrote: On 16.03.16 08:03, Serhiy Storchaka wrote: I just tested with Emacs, and it looks that when specify different codings on two different lines, the first coding wins, but when specify different codings on the

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread M.-A. Lemburg
On 17.03.2016 18:53, Serhiy Storchaka wrote: > On 17.03.16 19:23, M.-A. Lemburg wrote: >> On 17.03.2016 15:02, Serhiy Storchaka wrote: >>> On 17.03.16 15:14, M.-A. Lemburg wrote: On 17.03.2016 01:29, Guido van Rossum wrote: > Should we recommend that everyone use

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Stephen J. Turnbull
Glenn Linderman writes: > On 3/19/2016 8:19 AM, Serhiy Storchaka wrote: > > Therefore current CPython behavior can be correct, and the regular > > expression in PEP 263 should be changed to use greedy repetition. > > Just because emacs works that way (and even though I'm an emacs user),

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Glenn Linderman
On 3/19/2016 8:19 AM, Serhiy Storchaka wrote: On 16.03.16 08:03, Serhiy Storchaka wrote: On 15.03.16 22:30, Guido van Rossum wrote: I came across a file that had two different coding cookies -- one on the first line and one on the second. CPython uses the first, but mypy happens to use the

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread M.-A. Lemburg
On 17.03.2016 01:29, Guido van Rossum wrote: > I've updated the PEP. Please review. I decided not to update the > Unicode howto (the thing is too obscure). Serhiy, you're probably in a > better position to fix the code looking for cookies to pick the first > one if there are two on the same line

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Serhiy Storchaka
On 16.03.16 08:03, Serhiy Storchaka wrote: On 15.03.16 22:30, Guido van Rossum wrote: I came across a file that had two different coding cookies -- one on the first line and one on the second. CPython uses the first, but mypy happens to use the second. I couldn't find anything in the spec or

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread M.-A. Lemburg
On 17.03.2016 15:55, Guido van Rossum wrote: > On Thu, Mar 17, 2016 at 5:04 AM, Serhiy Storchaka wrote: >>> Should we recommend that everyone use tokenize.detect_encoding()? >> >> Likely. However the interface of tokenize.detect_encoding() is not very >> simple. > > I just

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Serhiy Storchaka
On 17.03.16 21:11, Guido van Rossum wrote: I tried this and it was too painful, so now I've just changed the regex that mypy uses to use non-eager matching (https://github.com/python/mypy/commit/b291998a46d580df412ed28af1ba1658446b9fe5). \s* matches newlines. {0,1}? is the same as ??.

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Terry Reedy
On 3/16/2016 3:14 AM, Serhiy Storchaka wrote: On 16.03.16 02:28, Guido van Rossum wrote: I agree that the spirit of the PEP is to stop at the first coding cookie found. Would it be okay if I updated the PEP to clarify this? I'll definitely also update the docs. Could you please also update

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Guido van Rossum
On Thu, Mar 17, 2016 at 5:04 AM, Serhiy Storchaka wrote: >> Should we recommend that everyone use tokenize.detect_encoding()? > > Likely. However the interface of tokenize.detect_encoding() is not very > simple. I just found that out yesterday. You have to give it a

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Brett Cannon
On Thu, 17 Mar 2016 at 07:56 Guido van Rossum wrote: > On Thu, Mar 17, 2016 at 5:04 AM, Serhiy Storchaka > wrote: > >> Should we recommend that everyone use tokenize.detect_encoding()? > > > > Likely. However the interface of tokenize.detect_encoding() is

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Serhiy Storchaka
On 17.03.16 21:11, Guido van Rossum wrote: This will raise SyntaxError if the encoding is unknown. That needs to be caught in mypy's case and then it needs to get the line number from the exception. Good point. "lineno" and "offset" attributes of SyntaxError is set to None by

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Glenn Linderman
On 3/16/2016 5:29 PM, Guido van Rossum wrote: I've updated the PEP. Please review. I decided not to update the Unicode howto (the thing is too obscure). Serhiy, you're probably in a better position to fix the code looking for cookies to pick the first one if there are two on the same line (or do

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Guido van Rossum
I've updated the PEP. Please review. I decided not to update the Unicode howto (the thing is too obscure). Serhiy, you're probably in a better position to fix the code looking for cookies to pick the first one if there are two on the same line (or do whatever you think should be done there).

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Stephen J. Turnbull
Guido van Rossum writes: > > Should we recommend that everyone use tokenize.detect_encoding()? +1 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Serhiy Storchaka
On 17.03.16 02:29, Guido van Rossum wrote: I've updated the PEP. Please review. I decided not to update the Unicode howto (the thing is too obscure). Serhiy, you're probably in a better position to fix the code looking for cookies to pick the first one if there are two on the same line (or do

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-19 Thread Serhiy Storchaka
On 17.03.16 19:23, M.-A. Lemburg wrote: On 17.03.2016 15:02, Serhiy Storchaka wrote: On 17.03.16 15:14, M.-A. Lemburg wrote: On 17.03.2016 01:29, Guido van Rossum wrote: Should we recommend that everyone use tokenize.detect_encoding()? I'd prefer a separate utility for this somewhere, since

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-18 Thread Serhiy Storchaka
On 17.03.16 16:55, Guido van Rossum wrote: On Thu, Mar 17, 2016 at 5:04 AM, Serhiy Storchaka wrote: Should we recommend that everyone use tokenize.detect_encoding()? Likely. However the interface of tokenize.detect_encoding() is not very simple. I just found that out

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-18 Thread Guido van Rossum
On Wed, Mar 16, 2016 at 12:59 AM, M.-A. Lemburg wrote: > The only reason to read up to two lines was to address the use of > the shebang on Unix, not to be able to define two competing > source code encodings :-) I know. I was just surprised that the PEP was sufficiently vague

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread Serhiy Storchaka
On 16.03.16 09:46, Glenn Linderman wrote: On 3/16/2016 12:09 AM, Serhiy Storchaka wrote: On 16.03.16 08:34, Glenn Linderman wrote: From the PEP 263: More precisely, the first or second line must match the regular expression "coding[:=]\s*([-\w.]+)". The first group of this

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread M.-A. Lemburg
On 16.03.2016 01:28, Guido van Rossum wrote: > I agree that the spirit of the PEP is to stop at the first coding > cookie found. Would it be okay if I updated the PEP to clarify this? > I'll definitely also update the docs. +1 The only reason to read up to two lines was to address the use of the

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread Glenn Linderman
On 3/16/2016 12:09 AM, Serhiy Storchaka wrote: On 16.03.16 08:34, Glenn Linderman wrote: From the PEP 263: More precisely, the first or second line must match the regular expression "coding[:=]\s*([-\w.]+)". The first group of this expression is then interpreted as encoding name. If

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread Serhiy Storchaka
On 16.03.16 02:28, Guido van Rossum wrote: I agree that the spirit of the PEP is to stop at the first coding cookie found. Would it be okay if I updated the PEP to clarify this? I'll definitely also update the docs. Could you please also update the regular expression in PEP 263 to "^[

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread Serhiy Storchaka
On 16.03.16 08:34, Glenn Linderman wrote: From the PEP 263: More precisely, the first or second line must match the regular expression "coding[:=]\s*([-\w.]+)". The first group of this expression is then interpreted as encoding name. If the encoding is unknown to Python, an error

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread Glenn Linderman
On 3/15/2016 11:07 PM, Chris Angelico wrote: On Wed, Mar 16, 2016 at 5:03 PM, Serhiy Storchaka wrote: On 15.03.16 22:30, Guido van Rossum wrote: I came across a file that had two different coding cookies -- one on the first line and one on the second. CPython uses the

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread Jonathan Goble
On Wed, Mar 16, 2016 at 2:07 AM, Chris Angelico wrote: > Why would you ever have two coding cookies in a file? Surely this > should be either an error, or ill-defined (ie parsers are allowed to > pick whichever they like, including raising)? > > ChrisA +1. If multiple coding

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread Chris Angelico
On Wed, Mar 16, 2016 at 5:03 PM, Serhiy Storchaka wrote: > On 15.03.16 22:30, Guido van Rossum wrote: >> >> I came across a file that had two different coding cookies -- one on >> the first line and one on the second. CPython uses the first, but mypy >> happens to use the

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-16 Thread Serhiy Storchaka
On 15.03.16 22:30, Guido van Rossum wrote: I came across a file that had two different coding cookies -- one on the first line and one on the second. CPython uses the first, but mypy happens to use the second. I couldn't find anything in the spec or docs ruling out the second interpretation.

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-15 Thread Ben Finney
Guido van Rossum writes: > I agree that the spirit of the PEP is to stop at the first coding > cookie found. Would it be okay if I updated the PEP to clarify this? > I'll definitely also update the docs. +1, it never occurred to me that the specification could mean otherwise.

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-15 Thread Guido van Rossum
I agree that the spirit of the PEP is to stop at the first coding cookie found. Would it be okay if I updated the PEP to clarify this? I'll definitely also update the docs. On Tue, Mar 15, 2016 at 2:04 PM, Brett Cannon wrote: > > > On Tue, 15 Mar 2016 at 13:31 Guido van Rossum

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-15 Thread MRAB
On 2016-03-15 20:53, MRAB wrote: On 2016-03-15 20:30, Guido van Rossum wrote: I came across a file that had two different coding cookies -- one on the first line and one on the second. CPython uses the first, but mypy happens to use the second. I couldn't find anything in the spec or docs

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-15 Thread Jon Ribbens
On Tue, Mar 15, 2016 at 01:30:08PM -0700, Guido van Rossum wrote: > I came across a file that had two different coding cookies -- one on > the first line and one on the second. CPython uses the first, but mypy > happens to use the second. I couldn't find anything in the spec or > docs ruling out

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-15 Thread Brett Cannon
On Tue, 15 Mar 2016 at 13:31 Guido van Rossum wrote: > I came across a file that had two different coding cookies -- one on > the first line and one on the second. CPython uses the first, but mypy > happens to use the second. I couldn't find anything in the spec or > docs

Re: [Python-Dev] What does a double coding cookie mean?

2016-03-15 Thread MRAB
On 2016-03-15 20:30, Guido van Rossum wrote: I came across a file that had two different coding cookies -- one on the first line and one on the second. CPython uses the first, but mypy happens to use the second. I couldn't find anything in the spec or docs ruling out the second interpretation.