Re: [Python-Dev] Issue?

[Tim Peters]

I wrote Python's sort, so I may know something about it ;-)  To my
eyes, no, there's not "an issue" here, but a full explanation would be
very involved.

For some sorting algorithms, it's possible to guarantee a redundant
comparison is never made.  For example, a pure insertion sort.

But Python's sort is quite complex, using a number of distinct
strategies dynamically adapting to structure found in the data as it
goes along.  There's a possibility for a small bit of redundant work
whenever switching from one strategy to another.  That's typical of
"hybrid" strategies.

Your case involves a switch between Python's sort's two simplest
strategies.  Let's just look at what matters:

[22, 33, 11, 11]

The first step is searching for "a natural run", an initial
sub-sequence that's already sorted.  22 < 33 says the first two
elements are already in order.  Then 11 < 33 says that's the end of
the run.

That part is over now.

The next step is using a binary insertion sort to move 11 into the
already-sorted [22, 33] prefix.  A binary search starts looking "in
the middle" first, but a 2-element run is so short that 33 "is the
middle", so it first compares 11 to 33 _again_.  So it goes.  Code to
detect that special case could be added, but it would probably cost
more than it saves (it always costs time to test for it, but that time
is pure waste unless the tests succeed).

This is specific to an ascending natural run of length 2, so is quite
a special case.  For example, in

[22, 33, 44, 11, 11]

11 < 44 ends the natural run, and then the binary search first
compares 11 to 33 ("the middle" of the 3-element natural run), and no
redundant comparison gets made.  But in

[22, 33, 44. 43, 11]

43 gets compared to 44 again on the _second_ iteration of the binary
search loop.

In general, this particular strategy switch ends up doing at most one
redundant comparison only if the first element after an ascending
natural run belongs immediately before the last element of the run.
For technical reasons, this can happen at most

min(len(the_list) / 32, number_of_natural_runs_in_the_list)

times, so it's generally trivial in an average-case O(N log N)
process.  It's so rare it would be minor in an O(N) process too,
unless N is very small.

A principled way to avoid this would be to skip the "find a run" step
if N is very small, leaving the whole thing to binary insertion sort.
Then a redundant comparison would never be done for small N.  But that
would end up doing more comparisons _overall_ in common cases where a
short list starts with a relatively (compared to N) sizable ascending
or descending run.

So I'm happy enough with the tradeoffs already in place.


On Wed, Aug 22, 2018 at 10:37 AM 楼晓峰 <1520306...@qq.com> wrote:
> Why compare twice?
>
> class Student(object):
>
> def __init__(self, name, score):
> self.name = name
> self.score = score
>
> def __str__(self):
> return '(%s: %s)' % (self.name, self.score)
>
> __repr__ = __str__
>
> def __lt__(self, s):
> #print(self, '--', s)
> if(self.score print(self, '<', s)
> return True
> if(self.score>s.score):
> print(self, '>', s)
> return False
> if(self.score==s.score):
> if(self.name>s.name):
> print(self, '>', s)
> return False
> if(self.name print(self, '<', s)
> return True
> if(self.name==s.name):
> print(self, '==', s)
> return False
>
> def __eq__(self, s):
> return (self.score == s.score) and (self.name == s.name)
> def __gt__(self, s):
> return not ((self == s) or (self < s))
> def __le__(self, s):
> return ((self == s) or (self < s))
> def __ge__(self, s):
> return ((self == s) or (self > s))
> def __nq__(self, s):
> return not (self == s)
>
> L = [Student('Tim', 22), Student('Bob', 33), Student('Kevin', 11), 
> Student('Alice', 11)]
> print(sorted(L))
>
> Output:
> (Bob: 33) > (Tim: 22)
> (Kevin: 11) < (Bob: 33)
> (Kevin: 11) < (Bob: 33)
> (Kevin: 11) < (Tim: 22)
> (Alice: 11) < (Tim: 22)
> (Alice: 11) < (Kevin: 11)
> [(Alice: 11), (Kevin: 11), (Tim: 22), (Bob: 33)]
>
> Best regards,
> Xiaofeng
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/tim.peters%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue?

[Chris Barker via Python-Dev]

python used the "timsort" sorting routine:

https://en.wikipedia.org/wiki/Timsort

So you can look at that and confirm that this is correct behaviour (I'm
betting it is :-)

But in general, sorting is O(n log(n)) -- there are going to be more than n
comparisons.

If comparing is slow, you want to use a key function, to reduce your
comparison to a simple and fast one:

sorted(L, key=lambda i: (i.name, i.score))

or something like that.

personally, I advocate adding a "key_fun" attribute to classes you want to
make efficiently sortable, so you'd have:

sorted(L, key=Student.key_fun)

There was a discussion on python-ideas about adding a __sort_key__ protocol
to python, but there were too many downsides.

-CHB




On Wed, Aug 22, 2018 at 3:40 AM, 楼晓峰 <1520306...@qq.com> wrote:

>
> *Why compare twice?*
>
> class Student(object):
>
> def __init__(self, name, score):
> self.name = name
> self.score = score
>
> def __str__(self):
> return '(%s: %s)' % (self.name, self.score)
>
> __repr__ = __str__
>
> def __lt__(self, s):
> #print(self, '--', s)
> if(self.score print(self, '<', s)
> return True
> if(self.score>s.score):
> print(self, '>', s)
> return False
> if(self.score==s.score):
> if(self.name>s.name):
> print(self, '>', s)
> return False
> if(self.name print(self, '<', s)
> return True
> if(self.name==s.name):
> print(self, '==', s)
> return False
>
> def __eq__(self, s):
> return (self.score == s.score) and (self.name == s.name)
> def __gt__(self, s):
> return not ((self == s) or (self < s))
> def __le__(self, s):
> return ((self == s) or (self < s))
> def __ge__(self, s):
> return ((self == s) or (self > s))
> def __nq__(self, s):
> return not (self == s)
>
> L = [Student('Tim', 22), Student('Bob', 33), Student('Kevin', 11),
> Student('Alice', 11)]
> print(sorted(L))
>
> Output:
> (Bob: 33) > (Tim: 22)
> *(Kevin: 11) < (Bob: 33)*
> *(Kevin: 11) < (Bob: 33)*
> (Kevin: 11) < (Tim: 22)
> (Alice: 11) < (Tim: 22)
> (Alice: 11) < (Kevin: 11)
> [(Alice: 11), (Kevin: 11), (Tim: 22), (Bob: 33)]
>
> *Best regards,*
> *Xiaofeng*
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> chris.barker%40noaa.gov
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue?

[Steven D'Aprano]

On Wed, Aug 22, 2018 at 06:40:42PM +0800, 楼晓峰 wrote:

> Why compare twice?

This is not a mailing list for asking for help with your own code. You 
can try this list instead:

https://mail.python.org/mailman/listinfo/python-list



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 32933

[Sanyam Khurana]

Hey Tony,

You can raise a PR and then start working on writing a test case of
it. People would then be able to see what exactly you've done and
suggest changes if need be.

Let me know if you've any more questions.

On Sat, Mar 3, 2018 at 11:24 PM, TonyFlury via Python-Dev
 wrote:
> I mentioned when I raised issue 32933 (mock_open doesn't support dunder iter
> protocol) that I have a one line fix for the issue.
>
> Who should I send this detail too ? Obviously it will need test cases too -
> I wrote a single very simple one just to prove to myself it worked - but it
> isn't complete by any stretch of the imagination.
>
> --
> Anthony Flury
> anthony.fl...@btinternet.com
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/sanyam.khurana01%40gmail.com
>



-- 
Mozilla Rep
http://www.SanyamKhurana.com
Github: CuriousLearner
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #21071: change struct.Struct.format type from bytes to str

[Victor Stinner]

2017-06-16 10:40 GMT+02:00 Nick Coghlan :
> As long as it's noted in the "Porting to Python 3.7" section of the
> 3.7 What's New guide, this seems like a sensible change to me.

Yes, the change is already documented there:
https://github.com/python/cpython/pull/845/files

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #21071: change struct.Struct.format type from bytes to str

[Nick Coghlan]

On 16 June 2017 at 14:19, Serhiy Storchaka  wrote:
> 27.03.17 15:12, Victor Stinner пише:
>>
>> I would like to change struct.Struct.format type from bytes to str. I
>> don't expect that anyone uses this attribute, and struct.Struct()
>> constructor accepts both bytes and str.
>>
>> http://bugs.python.org/issue21071
>>
>> It's just to be convenient: more functions accept str than bytes in
>> Python 3. Example: print() (python3 -bb raises an exceptions if you
>> pass bytes to print).
>>
>> Is anything opposed to breaking the backward compatibility?
>
>
> If nobody opposed to this change it will be made in short time.

As long as it's noted in the "Porting to Python 3.7" section of the
3.7 What's New guide, this seems like a sensible change to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #21071: change struct.Struct.format type from bytes to str

[Serhiy Storchaka]

27.03.17 15:12, Victor Stinner пише:

I would like to change struct.Struct.format type from bytes to str. I
don't expect that anyone uses this attribute, and struct.Struct()
constructor accepts both bytes and str.

http://bugs.python.org/issue21071

It's just to be convenient: more functions accept str than bytes in
Python 3. Example: print() (python3 -bb raises an exceptions if you
pass bytes to print).

Is anything opposed to breaking the backward compatibility?


If nobody opposed to this change it will be made in short time.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue with _thread.interrupt_main (29926)

[Martin Panter]

On 29 March 2017 at 00:40, Terry Reedy  wrote:
> [. . .] Eryk Sun suggested a patch for Windows, (and
> the possibility of using pthread_kill).  Can you possibly do one for *nix?
> This is out of my ballpark, but the bug (relative to console behavior) is a
> nuisance.

I'll try to find time, but no promises.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue with _thread.interrupt_main (29926)

[Terry Reedy]

Steven, thanks for verifying bug on *nix.

On 3/28/2017 2:00 AM, Martin Panter wrote:

On 28 March 2017 at 03:11, Steven D'Aprano  wrote:

On Mon, Mar 27, 2017 at 10:33:44PM -0400, Terry Reedy wrote:

https://bugs.python.org/issue29926 was opened as an IDLE issue, which
means that most watching the new issues list would ignore it.  But I
think it is an issue with _thread.interrupt_main (which IDLE calls in
respond to ^C) not interrupting time.sleep(n) in main thread*.  I tested
on Windows, don't know yet about OP.  Since there is no Expert's Index
listing for _thread (or threading), I am asking here for someone who
knows anything to take a look.

*

time.sleep(10)



<... remainder of 10 seconds pass>
KeyboardInterrupt



I get similar behaviour under Linux. I don't have the debug print, but
the KeyboardInterrupt doesn't interrupt the sleep until the 10 seconds
are up.


Looking at the implementation, _thread.interrupt_main just calls
PyErr_SetInterrupt. It doesn’t appear to send a signal. I played with
“strace” and couldn’t see any evidence of a signal. I guess it just
sets a flag that will be polled. To actually interrupt the “sleep”
call, you might need to use “pthread_kill” or similar (at least on
Unix).


I copied this to the issue.  Eryk Sun suggested a patch for Windows, 
(and the possibility of using pthread_kill).  Can you possibly do one 
for *nix?  This is out of my ballpark, but the bug (relative to console 
behavior) is a nuisance.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue with _thread.interrupt_main (29926)

[Martin Panter]

On 28 March 2017 at 03:11, Steven D'Aprano  wrote:
> On Mon, Mar 27, 2017 at 10:33:44PM -0400, Terry Reedy wrote:
>> https://bugs.python.org/issue29926 was opened as an IDLE issue, which
>> means that most watching the new issues list would ignore it.  But I
>> think it is an issue with _thread.interrupt_main (which IDLE calls in
>> respond to ^C) not interrupting time.sleep(n) in main thread*.  I tested
>> on Windows, don't know yet about OP.  Since there is no Expert's Index
>> listing for _thread (or threading), I am asking here for someone who
>> knows anything to take a look.
>>
>> *
>> >>> time.sleep(10)
>> 
>> 
>> <... remainder of 10 seconds pass>
>> KeyboardInterrupt
>
>
> I get similar behaviour under Linux. I don't have the debug print, but
> the KeyboardInterrupt doesn't interrupt the sleep until the 10 seconds
> are up.

Looking at the implementation, _thread.interrupt_main just calls
PyErr_SetInterrupt. It doesn’t appear to send a signal. I played with
“strace” and couldn’t see any evidence of a signal. I guess it just
sets a flag that will be polled. To actually interrupt the “sleep”
call, you might need to use “pthread_kill” or similar (at least on
Unix).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue with _thread.interrupt_main (29926)

[Steven D'Aprano]

On Mon, Mar 27, 2017 at 10:33:44PM -0400, Terry Reedy wrote:
> https://bugs.python.org/issue29926 was opened as an IDLE issue, which 
> means that most watching the new issues list would ignore it.  But I 
> think it is an issue with _thread.interrupt_main (which IDLE calls in 
> respond to ^C) not interrupting time.sleep(n) in main thread*.  I tested 
> on Windows, don't know yet about OP.  Since there is no Expert's Index 
> listing for _thread (or threading), I am asking here for someone who 
> knows anything to take a look.
> 
> *
> >>> time.sleep(10)
> 
> 
> <... remainder of 10 seconds pass>
> KeyboardInterrupt


I get similar behaviour under Linux. I don't have the debug print, but 
the KeyboardInterrupt doesn't interrupt the sleep until the 10 seconds 
are up.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Steve Dower]

There's a difference between "private", "stable for 3.x" and "stable for all 3" 
though. It's the third category that's getting too many functions added without 
due consideration.

Top-posted from my Windows Phone

-Original Message-
From: "Victor Stinner" <victor.stin...@gmail.com>
Sent: ‎12/‎21/‎2016 8:40
To: "Nathaniel Smith" <n...@pobox.com>
Cc: "Steve Dower" <steve.do...@python.org>; "Serhiy Storchaka" 
<storch...@gmail.com>; "Python Dev" <python-dev@python.org>
Subject: Re: [Python-Dev] Issue #23903 - stable API is incomplete

2016-12-21 17:21 GMT+01:00 Nathaniel Smith <n...@pobox.com>:
> It sounds like the opt-out approach isn't working very well, and maybe an
> opt-in approach should be considered instead? I recognize that the way C
> headers work makes this difficult, but it seems like something needs to
> change.

I proposed something different:
"Python 3.7: remove all private C functions from the Python C API?"
https://mail.python.org/pipermail/python-dev/2016-September/146386.html

Create subdirectories in Include/ to define private functions in
different files.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Steve Dower]

"maybe the test suite should error out if any unexpected symbols appear in the 
stable ABI?"

This or on build (personally I prefer this sort of validation at build time, 
but I know others would prefer to defer it).

We have a script now that can extract all the right functions, though I think 
it'll only work in the source tree as it relies on clinic. But that should make 
it fairly straightforward to spit out a list and compare it to a checked in 
list.

At the same time, we have a problem in the current release, which is the 
functions I listed earlier. I would really like to fix that without blocking on 
getting the right long-term fix (since the immediate fix only affects one file 
in the Windows distribution, though it has implications for supportability).

Top-posted from my Windows Phone

-Original Message-
From: "Nathaniel Smith" <n...@pobox.com>
Sent: ‎12/‎21/‎2016 8:22
To: "Steve Dower" <steve.do...@python.org>
Cc: "Serhiy Storchaka" <storch...@gmail.com>; "Victor Stinner" 
<victor.stin...@gmail.com>; "Python Dev" <python-dev@python.org>
Subject: Re: [Python-Dev] Issue #23903 - stable API is incomplete

On Dec 21, 2016 7:43 AM, "Steve Dower" <steve.do...@python.org> wrote:

"Ok, now why should _Py_PrintReferences() function be exported?"


It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so it is 
excluded from the headers (my list was automatically generated).



It sounds like the opt-out approach isn't working very well, and maybe an 
opt-in approach should be considered instead? I recognize that the way C 
headers work makes this difficult, but it seems like something needs to change. 


Or maybe the test suite should error out if any unexpected symbols appear in 
the stable ABI?


-n___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Victor Stinner]

2016-12-21 17:21 GMT+01:00 Nathaniel Smith :
> It sounds like the opt-out approach isn't working very well, and maybe an
> opt-in approach should be considered instead? I recognize that the way C
> headers work makes this difficult, but it seems like something needs to
> change.

I proposed something different:
"Python 3.7: remove all private C functions from the Python C API?"
https://mail.python.org/pipermail/python-dev/2016-September/146386.html

Create subdirectories in Include/ to define private functions in
different files.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Nathaniel Smith]

On Dec 21, 2016 7:43 AM, "Steve Dower"  wrote:

"Ok, now why should _Py_PrintReferences() function be exported?"

It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so it
is excluded from the headers (my list was automatically generated).


It sounds like the opt-out approach isn't working very well, and maybe an
opt-in approach should be considered instead? I recognize that the way C
headers work makes this difficult, but it seems like something needs to
change.

Or maybe the test suite should error out if any unexpected symbols appear
in the stable ABI?

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Serhiy Storchaka]

On 21.12.16 17:41, Steve Dower wrote:

"Ok, now why should _Py_PrintReferences() function be exported?"

It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so
it is excluded from the headers (my list was automatically generated).

And ideally, private functions that are deliberately exported would have
comments, or if they're new, a version check on Py_LIMITED_API.


Seconded.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Steve Dower]

"Ok, now why should _Py_PrintReferences() function be exported?"

It probably shouldn't, but it needs an #ifndef Py_LIMITED_API check so it is 
excluded from the headers (my list was automatically generated).

And ideally, private functions that are deliberately exported would have 
comments, or if they're new, a version check on Py_LIMITED_API.

Top-posted from my Windows Phone

-Original Message-
From: "Victor Stinner" <victor.stin...@gmail.com>
Sent: ‎12/‎21/‎2016 6:25
To: "Serhiy Storchaka" <storch...@gmail.com>
Cc: "Python Dev" <python-dev@python.org>
Subject: Re: [Python-Dev] Issue #23903 - stable API is incomplete

2016-12-21 14:06 GMT+01:00 Serhiy Storchaka <storch...@gmail.com>:
>> These functions are private. Would it be possible to not export them?
>
> Private functions used in public macros (like _Py_NewReference) should be
> exported.

Ah, _Py_NewReference is used in the PyObject_INIT(op, typeobj) *macro*, right.

IMO it's an issue with our public API: for the stable ABI, we should
replace such macro with a function which hides implementation details.

Example from pystate.h:

#ifdef Py_BUILD_CORE
PyAPI_DATA(_Py_atomic_address) _PyThreadState_Current;
#  define PyThreadState_GET() \
 ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current))
#else
#  define PyThreadState_GET() PyThreadState_Get()
#endif


Ok, now why should _Py_PrintReferences() function be exported? This
private function is only called from Py_FinalizeEx(). It is not used
in a macro.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Victor Stinner]

2016-12-21 14:06 GMT+01:00 Serhiy Storchaka :
>> These functions are private. Would it be possible to not export them?
>
> Private functions used in public macros (like _Py_NewReference) should be
> exported.

Ah, _Py_NewReference is used in the PyObject_INIT(op, typeobj) *macro*, right.

IMO it's an issue with our public API: for the stable ABI, we should
replace such macro with a function which hides implementation details.

Example from pystate.h:

#ifdef Py_BUILD_CORE
PyAPI_DATA(_Py_atomic_address) _PyThreadState_Current;
#  define PyThreadState_GET() \
 ((PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current))
#else
#  define PyThreadState_GET() PyThreadState_Get()
#endif


Ok, now why should _Py_PrintReferences() function be exported? This
private function is only called from Py_FinalizeEx(). It is not used
in a macro.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Serhiy Storchaka]

On 21.12.16 11:50, Victor Stinner wrote:

2016-12-21 2:52 GMT+01:00 Steve Dower :

_PyBytes_DecodeEscape
_PyDebug_PrintTotalRefs
_PyThreadState_Current
_PyTrash_thread_deposit_object
_PyTrash_thread_destroy_chain
_PyUnicode_DecodeUnicodeEscape
_Py_AddToAllObjects
_Py_ForgetReference
_Py_GetRefTotal
_Py_HashSecret_Initialized
_Py_NegativeRefcount
_Py_NewReference
_Py_PrintReferenceAddresses
_Py_PrintReferences
_Py_RefTotal


These functions are private. Would it be possible to not export them?


Private functions used in public macros (like _Py_NewReference) should 
be exported.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Victor Stinner]

2016-12-21 2:52 GMT+01:00 Steve Dower :
> _PyBytes_DecodeEscape
> _PyDebug_PrintTotalRefs
> _PyThreadState_Current
> _PyTrash_thread_deposit_object
> _PyTrash_thread_destroy_chain
> _PyUnicode_DecodeUnicodeEscape
> _Py_AddToAllObjects
> _Py_ForgetReference
> _Py_GetRefTotal
> _Py_HashSecret_Initialized
> _Py_NegativeRefcount
> _Py_NewReference
> _Py_PrintReferenceAddresses
> _Py_PrintReferences
> _Py_RefTotal

These functions are private. Would it be possible to not export them?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #23903 - stable API is incomplete

[Steve Dower]

I should also point out that when 3.6.0 releases, all of these will already be 
in the stable API for other platforms. I'm not going to take a stance on 
whether we can break it there between 3.6.0 and 3.6.1, but it may already be 
too late to remove any.

Top-posted from my Windows Phone

-Original Message-
From: "Steve Dower" 
Sent: ‎12/‎20/‎2016 17:56
To: "Python Dev" 
Subject: [Python-Dev] Issue #23903 - stable API is incomplete

For those who aren't aware, the stable API (PEP 384) is broken on 
Windows because the exports from python3.dll have not been kept up to date.

Over at http://bugs.python.org/issue23903 we're trying to address this 
by automatically generating the DLL based on the headers. This has shown 
that many more functions and data items are in the stable ABI than expected.

If it's left entirely to me, I'm planning to add all the public APIs 
into python3.dll, which will commit them to the stable API for good, and 
remove the _private APIs that have been added since we last updated the DLL.

However, if you've added an API recently that you didn't mean to be in 
the stable API, here is your chance to remove it. The full list of APIs 
that have never been available on Windows in the stable ABI but are in 
the headers are below. If it should not be considered long-term stable, 
then it needs "#ifndef Py_LIMITED_API" around it.

Note that anything NOT on this list has already been released, and so it 
cannot be removed from the stable API at this time. I want to resolve 
this for 3.5.3 (that is, release all of these as stable and then it 
cannot be undone), which is coming up very soon, so if any of the public 
APIs should NOT be stable, please fix them, and if any of the private 
APIs SHOULD be stable, they'll probably need version-specific guards 
(see moduleobject.h).

Cheers,
Steve

Full list of APIs to be added to python3.dll:

PyAST_FromNode
PyAST_FromNodeObject
PyAST_Validate
PyCmpWrapper_Type
PyCodec_NameReplaceErrors
PyErr_GetExcInfo
PyErr_ResourceWarning
PyErr_SetExcFromWindowsErr
PyErr_SetExcFromWindowsErrWithFilename
PyErr_SetExcFromWindowsErrWithFilenameObject
PyErr_SetExcFromWindowsErrWithFilenameObjects
PyErr_SetExcInfo
PyErr_SetExcWithArgsKwargs
PyErr_SetFromErrnoWithFilenameObjects
PyErr_SetFromWindowsErr
PyErr_SetFromWindowsErrWithFilename
PyErr_SetImportError
PyErr_SetImportErrorSubclass
PyErr_SyntaxLocationEx
PyExc_BlockingIOError
PyExc_BrokenPipeError
PyExc_ChildProcessError
PyExc_ConnectionAbortedError
PyExc_ConnectionError
PyExc_ConnectionRefusedError
PyExc_ConnectionResetError
PyExc_FileExistsError
PyExc_FileNotFoundError
PyExc_InterruptedError
PyExc_IsADirectoryError
PyExc_ModuleNotFoundError
PyExc_NotADirectoryError
PyExc_PermissionError
PyExc_ProcessLookupError
PyExc_RecursionError
PyExc_ResourceWarning
PyExc_StopAsyncIteration
PyExc_TimeoutError
PyExc_WindowsError
PyImport_AddModuleObject
PyImport_ExecCodeModuleObject
PyImport_ImportFrozenModuleObject
PyImport_ImportModuleLevelObject
PyMarshal_ReadObjectFromString
PyMarshal_WriteLongToFile
PyMarshal_WriteObjectToFile
PyMarshal_WriteObjectToString
PyMem_Calloc
PyMember_GetOne
PyMember_SetOne
PyMemoryView_FromMemory
PyModuleDef_Init
PyModuleDef_Type
PyModule_AddFunctions
PyModule_ExecDef
PyModule_FromDefAndSpec2
PyModule_GetNameObject
PyModule_NewObject
PyModule_SetDocString
PyNode_AddChild
PyNode_Free
PyNode_ListTree
PyNode_New
PyNumber_InPlaceMatrixMultiply
PyNumber_MatrixMultiply
PyOS_CheckStack
PyOS_FSPath
PyObject_Calloc
PyObject_GenericSetDict
PyParser_SimpleParseStringFlagsFilename
PySys_AddXOption
PySys_GetXOptions
PyThread_GetInfo
PyThread_ReInitTLS
PyThread_acquire_lock
PyThread_acquire_lock_timed
PyThread_allocate_lock
PyThread_create_key
PyThread_delete_key
PyThread_delete_key_value
PyThread_exit_thread
PyThread_free_lock
PyThread_get_key_value
PyThread_get_stacksize
PyThread_get_thread_ident
PyThread_init_thread
PyThread_release_lock
PyThread_set_key_value
PyThread_set_stacksize
PyThread_start_new_thread
PyUnicode_AsMBCSString
PyUnicode_AsUCS4
PyUnicode_AsUCS4Copy
PyUnicode_AsWideCharString
PyUnicode_DecodeCodePageStateful
PyUnicode_DecodeLocale
PyUnicode_DecodeLocaleAndSize
PyUnicode_DecodeMBCS
PyUnicode_DecodeMBCSStateful
PyUnicode_EncodeCodePage
PyUnicode_EncodeLocale
PyUnicode_FindChar
PyUnicode_GetLength
PyUnicode_ReadChar
PyUnicode_Substring
PyUnicode_WriteChar
Py_DecodeLocale
Py_EncodeLocale
Py_FileSystemDefaultEncodeErrors
Py_SetPath
Py_hexdigits
_PyBytes_DecodeEscape
_PyDebug_PrintTotalRefs
_PyThreadState_Current
_PyTrash_thread_deposit_object
_PyTrash_thread_destroy_chain
_PyUnicode_DecodeUnicodeEscape
_Py_AddToAllObjects
_Py_ForgetReference
_Py_GetRefTotal
_Py_HashSecret_Initialized
_Py_NegativeRefcount
_Py_NewReference
_Py_PrintReferenceAddresses
_Py_PrintReferences
_Py_RefTotal
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Joseph Martinot-Lagarde]

Victor Stinner  gmail.com> writes:

> 
> Hi,
> 
> I changed the Python compiler to ignore any kind "constant
> expressions", whereas it only ignored strings and integers before:
> http://bugs.python.org/issue26204
> 
> The compiler now also emits a SyntaxWarning on such case. IMHO the
> warning can help to detect bugs for developers who just learnt Python.
> 
> The warning is *not* emited for strings, since triple quoted strings
> are a common syntax for multiline comments.
> 
> The warning is *not* emited neither for ellispis (...) since "f():
> ..." is a legit syntax for abstract function.
> 

I frequently use 1/0 as a quick break in a script or a program (it's even
more useful with post-mortem debugging). Would it be considered as a
constant and ignored instead of raising a ZeroDivisionError ?

Joseph




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Victor Stinner]

2016-02-09 10:57 GMT+01:00 Joseph Martinot-Lagarde :
> I frequently use 1/0 as a quick break in a script or a program (it's even
> more useful with post-mortem debugging). Would it be considered as a
> constant and ignored instead of raising a ZeroDivisionError ?

"self.x - self.y" and "1/0" are not removed since they have side effects.

Right now, "(1, 2, 3)" is not removed. But later we may remove it,
since it has no side effect, it's a constant statement.

Note: We are talking about statements. 1 is not removed in "lambda: 1"
which is a valid expression ;-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Georg Brandl]

On 02/09/2016 10:57 AM, Joseph Martinot-Lagarde wrote:
> Victor Stinner  gmail.com> writes:
> 
>> 
>> Hi,
>> 
>> I changed the Python compiler to ignore any kind "constant
>> expressions", whereas it only ignored strings and integers before:
>> http://bugs.python.org/issue26204
>> 
>> The compiler now also emits a SyntaxWarning on such case. IMHO the
>> warning can help to detect bugs for developers who just learnt Python.
>> 
>> The warning is *not* emited for strings, since triple quoted strings
>> are a common syntax for multiline comments.
>> 
>> The warning is *not* emited neither for ellispis (...) since "f():
>> ..." is a legit syntax for abstract function.
>> 
> 
> I frequently use 1/0 as a quick break in a script or a program (it's even
> more useful with post-mortem debugging). Would it be considered as a
> constant and ignored instead of raising a ZeroDivisionError ?

At first, expressions involving operators are not seen as constant.  But
1/2 would be removed, since the peepholer will evaluate it to 0.5 (or 0)
and the constant-removal pass will recognize it as a constant (assuming
this ordering of the passes).

In the case of 1/0 the peepholer will try to evaluate it, but get an
exception and therefore not touch the expression further.

cheers,
Georg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Michel Desmoulin]

Hello,

Le 08/02/2016 20:13, Guido van Rossum a écrit :

On Mon, Feb 8, 2016 at 9:44 AM, Victor Stinner  wrote:

I changed the Python compiler to ignore any kind "constant
expressions", whereas it only ignored strings and integers before:
http://bugs.python.org/issue26204

The compiler now also emits a SyntaxWarning on such case. IMHO the
warning can help to detect bugs for developers who just learnt Python.

Hum. I'm not excited by this idea. It is not bad syntax. Have you
actually seen newbies who were confused by such things?




I give regular Python trainings and I see similar errors regularly such as:

- not returning something;
- using something without putting the result back in a variable.

However, these are impossible to warn about.

What's more, I have yet to see somebody creating a constant and not 
doing anything with it. I never worked with Ruby dev though.


My sample of dev is not big enough to be significant, but I haven't met 
this issue yet. I still like the idea, anything making Python easier for 
beginers is a good thing for me.


One particular argument against it is the use of linters, but you must 
realize most beginers don't use linters. Just like they don't use 
virtualenv, pip, pdb, etc. They are part of a toolkit you learn to use 
on the way, but not something you start with. Besides, many people using 
Python are not dev, and will just never take the time to use linters, 
not learn about them.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Andrew Barnert via Python-Dev]

On Tuesday, February 9, 2016 8:14 AM, Michel Desmoulin 
 wrote:

> I give regular Python trainings and I see similar errors regularly such as:
> 
> - not returning something;
> - using something without putting the result back in a variable.
> 
> However, these are impossible to warn about.
> 
> What's more, I have yet to see somebody creating a constant and not 
> doing anything with it. I never worked with Ruby dev though.
> 

> My sample of dev is not big enough to be significant, but I haven't met 
> this issue yet. I still like the idea, anything making Python easier for 
> beginers is a good thing for me.

What idea do you like? Somehow warning about the things that are impossible to 
warn about? Or warning about something different that isn't any of the things 
your novices have faced? Or...?

> One particular argument against it is the use of linters, but you must 
> realize most beginers don't use linters.

That doesn't mean the compiler should do everything linters do.

Rank beginners are generally writing very simple programs, where the whole 
thing can be visualized at once, so many warnings aren't relevant. And they 
haven't learned many important language features, so many warnings are 
relevant, but they aren't prepared to deal with them (e.g., global variables 
everywhere because they haven't learned to declare functions yet). As a 
teacher, do you want to explain all those warnings to them? Or teach them the 
bad habit of ignoring warnings? Or just not teach them to use linters (or 
static type checkers, or other such tools) until they're ready to write code 
that should pass without warnings?

Part of learning to use linters effectively is learning to configure them. 
That's almost certainly not something you want to be teaching beginners when 
they're just starting out. But if the compiler started adding a bunch of 
warnings that people had to configure, a la gcc, you'd be forced to teach them 
right off the bat.

And meanwhile, once past the initial stage, many beginners _do_ use linters, 
they just don't realize it. If you use PyCharm or Eclipse/PyDev or almost any 
IDE except IDLE, it may be linting in the background and showing you the 
results as inline code hints, or in some other user-friendly way, or at least 
catching some of the simpler things a linter would check for. Whether you want 
to use those tools in your teaching is up to you, but they exist. And if they 
need any support from the compiler to do their job better, presumably they'd 
ask for it.

> They are part of a toolkit you learn to use 

> on the way, but not something you start with. Besides, many people using 
> Python are not dev, and will just never take the time to use linters, 
> not learn about them.


If people who aren't going to go deep enough into Python to write scripts 
longer than a page don't need linters, then they certainly don't need a bunch 
of warnings from the compiler either.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Yury Selivanov]

On 2016-02-08 8:02 PM, Steven D'Aprano wrote:

On Mon, Feb 08, 2016 at 05:43:25PM -0500, Yury Selivanov wrote:


On 2016-02-08 5:19 PM, Terry Reedy wrote:

On 2/8/2016 4:51 PM, Victor Stinner wrote:

2016-02-08 22:28 GMT+01:00 Alexander Walters :

What incantation do you need to do to make that behavior apparent?

I didn't know. I just checked. It's assert used with a non-empty tuple:


assert ("tuple",)

:1: SyntaxWarning: assertion is always true, perhaps remove
parentheses?

I think this should be left to linters also.


I agree.  I'd remove that warning.


Please don't remove the warning, it is very useful.

Compare an assertion written correctly:

py> assert 1==2, "Error in arithmetic"
Traceback (most recent call last):
   File "", line 1, in 
AssertionError: Error in arithmetic


with the simple mistake of wrapping the "tuple" in parens:

py> assert (1==2, "Error in arithmetic")
:1: SyntaxWarning: assertion is always true, perhaps remove parentheses?
py>


You're right!  It's indeed a trap that we should warn about.

Thanks!

Yury

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[John Mark Vandenberg]

On Tue, Feb 9, 2016 at 7:34 AM, Guido van Rossum  wrote:
> On Mon, Feb 8, 2016 at 11:51 AM, Victor Stinner
>  wrote:
>> Le 8 févr. 2016 8:14 PM, "Guido van Rossum"  a écrit :
>>> Hum. I'm not excited by this idea. It is not bad syntax.
>>
>> Do you see an use case for "constant statements" other than strings and
>> ellipsis?
>
> The same use case as for all dead code: it could be a placeholder for
> something better in the future.

Allowing dead code is useful as it allows complex code to be left in
place.  It can be risky removing the code.

Unused literals are stupefyingly simple statements.
A line of merely a constant, e.g. 'True' or '1', does not present the
same risks or benefits.
That it is a hope for something better?
It could be easily replaced with 'pass', '...', a comment, and/or a
string literal explaining what needs improving.

> It could also be generated code where the generator expects the
> optimizer to remove it (or doesn't care).

Why shouldnt a user see that it is generating such code?
There is a decent chance that it is a bug in the generated code.

fwiw, this is a syntax warning in Ruby - "unused literal ignored",
since 2003 (5aadcd9).

--
John Vandenberg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Chris Angelico]

On Tue, Feb 9, 2016 at 12:41 PM, MRAB  wrote:
> On 2016-02-09 00:53, Guido van Rossum wrote:
>>
>> The warning for 'assert (cond, msg)' was specifically put in because
>> this is a nasty trap. It's *always* a mistaken attempt to write
>> 'assert cond, msg' -- usually in an attempt to break a long line
>> without using a backslash. I'd actually consider promoting it to a
>> syntax error rather than removing the warning.
>>
>> Compared to other "lint warnings" this one is much nastier -- it is
>> also much more certain that it is a mistake. (Much more certain than
>> e.g. an undefined variable, which could still be legitimate code due
>> to dynamic updates to globals() or builtins.)
>
> Would there be less chance of confusion if there were some kind of syntax
> such as "assert cond with msg"?

Is assert the *only* statement that has a comma separating unrelated
items? Every other statement that uses a comma is separating identical
items (eg "import os, sys" - "os" and "sys" are equivalent), and
tokens that have completely different meaning are separated by a word.
The only other exception I can think of - pun intended - is the old
"except BaseException, e:" syntax, which got dropped in Py3. Maybe
it's time to introduce a new syntax with a view to deprecating the
comma syntax ("use this old syntax if you need to support Python
2.7").

+1.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Guido van Rossum]

On Mon, Feb 8, 2016 at 5:41 PM, MRAB  wrote:
> On 2016-02-09 00:53, Guido van Rossum wrote:
>>
>> The warning for 'assert (cond, msg)' was specifically put in because
>> this is a nasty trap. It's *always* a mistaken attempt to write
>> 'assert cond, msg' -- usually in an attempt to break a long line
>> without using a backslash. I'd actually consider promoting it to a
>> syntax error rather than removing the warning.
>>
>> Compared to other "lint warnings" this one is much nastier -- it is
>> also much more certain that it is a mistake. (Much more certain than
>> e.g. an undefined variable, which could still be legitimate code due
>> to dynamic updates to globals() or builtins.)
>
> Would there be less chance of confusion if there were some kind of syntax
> such as "assert cond with msg"?

Perhaps, but as long as the "with msg" isn't mandatory and the "assert
x, y" syntax is still valid we'd still have to warn about "assert (x,
y)". Note that in general "assert constant" is not a problem (assert
True and assert False have their uses :-). It's only the literal tuple
form.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[MRAB]

On 2016-02-09 00:53, Guido van Rossum wrote:

The warning for 'assert (cond, msg)' was specifically put in because
this is a nasty trap. It's *always* a mistaken attempt to write
'assert cond, msg' -- usually in an attempt to break a long line
without using a backslash. I'd actually consider promoting it to a
syntax error rather than removing the warning.

Compared to other "lint warnings" this one is much nastier -- it is
also much more certain that it is a mistake. (Much more certain than
e.g. an undefined variable, which could still be legitimate code due
to dynamic updates to globals() or builtins.)
Would there be less chance of confusion if there were some kind of 
syntax such as "assert cond with msg"?


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Guido van Rossum]

Personally I don't think it's worth the churn.

On Mon, Feb 8, 2016 at 5:49 PM, Chris Angelico  wrote:
> On Tue, Feb 9, 2016 at 12:41 PM, MRAB  wrote:
>> On 2016-02-09 00:53, Guido van Rossum wrote:
>>>
>>> The warning for 'assert (cond, msg)' was specifically put in because
>>> this is a nasty trap. It's *always* a mistaken attempt to write
>>> 'assert cond, msg' -- usually in an attempt to break a long line
>>> without using a backslash. I'd actually consider promoting it to a
>>> syntax error rather than removing the warning.
>>>
>>> Compared to other "lint warnings" this one is much nastier -- it is
>>> also much more certain that it is a mistake. (Much more certain than
>>> e.g. an undefined variable, which could still be legitimate code due
>>> to dynamic updates to globals() or builtins.)
>>
>> Would there be less chance of confusion if there were some kind of syntax
>> such as "assert cond with msg"?
>
> Is assert the *only* statement that has a comma separating unrelated
> items? Every other statement that uses a comma is separating identical
> items (eg "import os, sys" - "os" and "sys" are equivalent), and
> tokens that have completely different meaning are separated by a word.
> The only other exception I can think of - pun intended - is the old
> "except BaseException, e:" syntax, which got dropped in Py3. Maybe
> it's time to introduce a new syntax with a view to deprecating the
> comma syntax ("use this old syntax if you need to support Python
> 2.7").
>
> +1.
>
> ChrisA
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[xiscu]

>> New behaviour:
>>
>> haypo@smithers$ ./python
>> Python 3.6.0a0 (default:759a975e1230, Feb  8 2016, 18:21:23)
> def f():
>> ...  False
>> ...

Ok, I see in your case there's no return :-)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[francismb]

Hi,

On 02/08/2016 06:44 PM, Victor Stinner wrote:
> Hi,
> 
> I changed the Python compiler to ignore any kind "constant
> expressions", whereas it only ignored strings and integers before:
> http://bugs.python.org/issue26204
> 
> The compiler now also emits a SyntaxWarning on such case. IMHO the
> warning can help to detect bugs for developers who just learnt Python.
> 
[...]
> New behaviour:
>
> haypo@smithers$ ./python
> Python 3.6.0a0 (default:759a975e1230, Feb  8 2016, 18:21:23)
 def f():
> ...  False
> ...
> :2: SyntaxWarning: ignore constant statement
>


Just for my understanding:

What would happen if someone has functions where some return
constant expressions and others not and then that functions
are used depending on some other context. E.g:


def behaviour2(ctx):
   return 1

def behaviour1(ctx):
   return some_calculation_with(ctx)


[...]

if ... :
  return behaviour1(ctx)
else :
  return behaviour2()


Is that going to raise a warning?


Thanks in advance!
francis

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Ethan Furman]

On 02/08/2016 10:00 AM, francismb wrote:
> On 02/08/2016 06:44 PM, Victor Stinner wrote:

>> I changed the Python compiler to ignore any kind "constant
>> expressions", whereas it only ignored strings and integers before:
>> http://bugs.python.org/issue26204
>>
>> The compiler now also emits a SyntaxWarning on such case. IMHO the
>> warning can help to detect bugs for developers who just learnt
>> Python.
>>
> [...]
>> New behaviour:
>>
>> haypo@smithers$ ./python
>> Python 3.6.0a0 (default:759a975e1230, Feb  8 2016, 18:21:23)
>> --> def f():
>> ...  False
>> ...
>> :2: SyntaxWarning: ignore constant statement
>
> Just for my understanding:
>
> What would happen if someone has functions where some return
> constant expressions and others not and then that functions
> are used depending on some other context. E.g:
>
> def behaviour2(ctx):
> return 1
>
> def behaviour1(ctx):
> return some_calculation_with(ctx)
>
>
> [...]
>
> if ... :
>return behaviour1(ctx)
> else :
>return behaviour2()
>
>
> Is that going to raise a warning?

No, because those constants are being used (returned).  Only constants 
that aren't used at all get omitted.


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Ethan Furman]

On 02/08/2016 09:44 AM, Victor Stinner wrote:

> Are you ok with the new warning?

+1

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Gregory P. Smith]

On Mon, Feb 8, 2016 at 9:44 AM Victor Stinner 
wrote:

> Hi,
>
> I changed the Python compiler to ignore any kind "constant
> expressions", whereas it only ignored strings and integers before:
> http://bugs.python.org/issue26204
>
> The compiler now also emits a SyntaxWarning on such case. IMHO the
> warning can help to detect bugs for developers who just learnt Python.
>
> The warning is *not* emited for strings, since triple quoted strings
> are a common syntax for multiline comments.
>
> The warning is *not* emited neither for ellispis (...) since "f():
> ..." is a legit syntax for abstract function.
>
> Are you ok with the new warning?
>

I'm +1 on this.

-gps


>
>
> New behaviour:
>
> haypo@smithers$ ./python
> Python 3.6.0a0 (default:759a975e1230, Feb  8 2016, 18:21:23)
> >>> def f():
> ...  False
> ...
> :2: SyntaxWarning: ignore constant statement
>
> >>> import dis; dis.dis(f)
>   2   0 LOAD_CONST   0 (None)
>   3 RETURN_VALUE
>
>
> Old behaviour:
>
> haypo@smithers$ python3
> Python 3.4.3 (default, Jun 29 2015, 12:16:01)
> >>> def f():
> ...  False
> ...
> >>> import dis; dis.dis(f)
>   2   0 LOAD_CONST   1 (False)
>   3 POP_TOP
>   4 LOAD_CONST   0 (None)
>   7 RETURN_VALUE
>
>
>
> Before strings and numbers were already ignored. Example:
>
> haypo@smithers$ python3
> Python 3.4.3 (default, Jun 29 2015, 12:16:01)
>
> >>> def f():
> ...  123
> ...
> >>> import dis; dis.dis(f)
>   2   0 LOAD_CONST   0 (None)
>   3 RETURN_VALUE
>
>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Guido van Rossum]

On Mon, Feb 8, 2016 at 11:51 AM, Victor Stinner
 wrote:
> Le 8 févr. 2016 8:14 PM, "Guido van Rossum"  a écrit :
>> Hum. I'm not excited by this idea. It is not bad syntax.
>
> Do you see an use case for "constant statements" other than strings and
> ellipsis?

The same use case as for all dead code: it could be a placeholder for
something better in the future.

It could also be generated code where the generator expects the
optimizer to remove it (or doesn't care).

If you want to do linter integration that should probably be
integrated with the user's editor, like it is in PyCharm, and IIUC
people can do this in e.g. Emacs, Sublime or Vim as well. Leave the
interpreter alone.

> Such statement does nothing. Previously the compiler emited
> LOAD_CONST+POP_TOP.
>
> GCC also emits a warning on such code.
>
>> Have you
>> actually seen newbies who were confused by such things?
>
> Well, not really. But I don't see any use case of such code except of
> obvious mistakes. Sometimes such code appears after multiple refactoring
> (and mistakes).
>
> Are you suggesting to remove the warning?

I haven't seen this warning yet. I take it this is new in the 3.6 branch?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[John Mark Vandenberg]

On Tue, Feb 9, 2016 at 8:20 AM, Victor Stinner  wrote:
> Le 8 févr. 2016 9:34 PM, "Guido van Rossum"  a écrit :
>> If you want to do linter integration that should probably be
>> integrated with the user's editor, like it is in PyCharm, and IIUC
>> people can do this in e.g. Emacs, Sublime or Vim as well. Leave the
>> interpreter alone.
>
> In GCC, warnings are welcome because it does one thing: compile code. GCC is
> used by developers. Users use the produced binary.
>
> In Python, it's different because it executes code and runs code. It's used
> by developers and users.
>
> It's more tricky to make choices like showing or not deprecation warnings.
>
> It looks like most Python developers prefer to use an external linter.

fwiw, pyflakes doesnt detect this.  I've created a bug for that

https://bugs.launchpad.net/pyflakes/+bug/1543246

-- 
John Vandenberg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Ethan Furman]

On 02/08/2016 10:15 AM, Ethan Furman wrote:

On 02/08/2016 09:44 AM, Victor Stinner wrote:

 > Are you ok with the new warning?

+1


Changing my vote:

-1 on the warning

+0 on simply removing the unused constant

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Victor Stinner]

Le 8 févr. 2016 9:10 PM, "Alexander Walters"  a
écrit :
>
> I am not keen on a SyntaxWarning.  Either something is python syntax, or
it is not.

Oh I forgot to mention that Python already emits SyntaxWarning, on "assert
True" for example.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Guido van Rossum]

On Mon, Feb 8, 2016 at 9:44 AM, Victor Stinner  wrote:
> I changed the Python compiler to ignore any kind "constant
> expressions", whereas it only ignored strings and integers before:
> http://bugs.python.org/issue26204
>
> The compiler now also emits a SyntaxWarning on such case. IMHO the
> warning can help to detect bugs for developers who just learnt Python.

Hum. I'm not excited by this idea. It is not bad syntax. Have you
actually seen newbies who were confused by such things?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Andrew Barnert via Python-Dev]

> On Feb 8, 2016, at 11:13, Guido van Rossum  wrote:
> 
>> On Mon, Feb 8, 2016 at 9:44 AM, Victor Stinner  
>> wrote:
>> I changed the Python compiler to ignore any kind "constant
>> expressions", whereas it only ignored strings and integers before:
>> http://bugs.python.org/issue26204
>> 
>> The compiler now also emits a SyntaxWarning on such case. IMHO the
>> warning can help to detect bugs for developers who just learnt Python.
> 
> Hum. I'm not excited by this idea. It is not bad syntax. Have you
> actually seen newbies who were confused by such things?

This does overlap to some extent with a problem that newbies *do* get confused 
by (and that transplants from Ruby don't find confusing, but do keep 
forgetting): writing an expression as the last statement in a function and then 
getting a TypeError or AttributeError about NoneType from the caller. Victor's 
example of a function that was presumably meant to return False, but instead 
just evaluates False and returns None, does happen.

But often, that last expression isn't a constant, but something like self.y - 
self.x. So I'm not sure how much this warning would help that case. In fact, it 
might add to the confusion if sometimes you get a warning and sometimes you 
don't. (And you wouldn't want a warning about any function with no return whose 
last statement is an expression, because often that's perfectly reasonable 
code, where the expression is a mutating method call, like 
self.spam.append(arg).)

Also, there are plenty of other common newbie/transplant problems that are 
similar to this one but can't be caught with a warning, like just referencing a 
function or method instead of calling it because you left the parens off. 
That's *usually* a bug, but not always--it could be a LBYL check for an 
attribute's presence, for example.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Victor Stinner]

Le 8 févr. 2016 9:34 PM, "Guido van Rossum"  a écrit :
> If you want to do linter integration that should probably be
> integrated with the user's editor, like it is in PyCharm, and IIUC
> people can do this in e.g. Emacs, Sublime or Vim as well. Leave the
> interpreter alone.

In GCC, warnings are welcome because it does one thing: compile code. GCC
is used by developers. Users use the produced binary.

In Python, it's different because it executes code and runs code. It's used
by developers and users.

It's more tricky to make choices like showing or not deprecation warnings.

It looks like most Python developers prefer to use an external linter.

I don't really care of the warning, I will remove it.

> I haven't seen this warning yet. I take it this is new in the 3.6 branch?

Yes it's a recent change in the default branch (a few hours ago).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Guido van Rossum]

On Mon, Feb 8, 2016 at 1:20 PM, Victor Stinner  wrote:
> Le 8 févr. 2016 9:34 PM, "Guido van Rossum"  a écrit :
>> If you want to do linter integration that should probably be
>> integrated with the user's editor, like it is in PyCharm, and IIUC
>> people can do this in e.g. Emacs, Sublime or Vim as well. Leave the
>> interpreter alone.
>
> In GCC, warnings are welcome because it does one thing: compile code. GCC is
> used by developers. Users use the produced binary.
>
> In Python, it's different because it executes code and runs code. It's used
> by developers and users.
>
> It's more tricky to make choices like showing or not deprecation warnings.
>
> It looks like most Python developers prefer to use an external linter.
>
> I don't really care of the warning, I will remove it.

Thanks!

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Alexander Walters]

What incantation do you need to do to make that behavior apparent?

tritium@gesa:~$ python3.5 -W all
Python 3.5.1 (default, Dec 18 2015, 02:15:10)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
Jedi is not installed, falling back to readline
>>> assert True
>>>


On 2/8/2016 16:23, Victor Stinner wrote:



Le 8 févr. 2016 9:10 PM, "Alexander Walters" > a écrit :

>
> I am not keen on a SyntaxWarning.  Either something is python 
syntax, or it is not.


Oh I forgot to mention that Python already emits SyntaxWarning, on 
"assert True" for example.


Victor



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/tritium-list%40sdamon.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[John Mark Vandenberg]

On Tue, Feb 9, 2016 at 8:41 AM, Alexander Walters
 wrote:
>
>
> On 2/8/2016 16:37, John Mark Vandenberg wrote:
>>
>> fwiw, pyflakes doesnt detect this. I've created a bug for that
>> https://bugs.launchpad.net/pyflakes/+bug/1543246
>
>
> Flake8 does, so it might be in the ... poorly named ... pep8 checker.

I believe the pep8 checker does not have a check for this.
Could you confirm; which flake8 code are you seeing?

-- 
John Vandenberg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Alexander Walters]

On 2/8/2016 16:37, John Mark Vandenberg wrote:
fwiw, pyflakes doesnt detect this. I've created a bug for that 
https://bugs.launchpad.net/pyflakes/+bug/1543246 


Flake8 does, so it might be in the ... poorly named ... pep8 checker.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Victor Stinner]

Le 8 févr. 2016 8:14 PM, "Guido van Rossum"  a écrit :
> Hum. I'm not excited by this idea. It is not bad syntax.

Do you see an use case for "constant statements" other than strings and
ellipsis?

Such statement does nothing. Previously the compiler emited
LOAD_CONST+POP_TOP.

GCC also emits a warning on such code.

> Have you
> actually seen newbies who were confused by such things?

Well, not really. But I don't see any use case of such code except of
obvious mistakes. Sometimes such code appears after multiple refactoring
(and mistakes).

Are you suggesting to remove the warning?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Alexander Walters]

I am not keen on a SyntaxWarning.  Either something is python syntax, or 
it is not.  This warning catches something linters have been catching 
for ages.  I really don't see the value in adding this, and can see it 
causing more confusion than it solves.  In the #python irc channel, we 
see quite a few newbie mistakes, but declaring a constant that isn't 
used is rarely if ever one of them.


On 2/8/2016 12:44, Victor Stinner wrote:

Hi,

I changed the Python compiler to ignore any kind "constant
expressions", whereas it only ignored strings and integers before:
http://bugs.python.org/issue26204

The compiler now also emits a SyntaxWarning on such case. IMHO the
warning can help to detect bugs for developers who just learnt Python.

The warning is *not* emited for strings, since triple quoted strings
are a common syntax for multiline comments.

The warning is *not* emited neither for ellispis (...) since "f():
..." is a legit syntax for abstract function.

Are you ok with the new warning?


New behaviour:

haypo@smithers$ ./python
Python 3.6.0a0 (default:759a975e1230, Feb  8 2016, 18:21:23)

def f():

...  False
...
:2: SyntaxWarning: ignore constant statement


import dis; dis.dis(f)

   2   0 LOAD_CONST   0 (None)
   3 RETURN_VALUE


Old behaviour:

haypo@smithers$ python3
Python 3.4.3 (default, Jun 29 2015, 12:16:01)

def f():

...  False
...

import dis; dis.dis(f)

   2   0 LOAD_CONST   1 (False)
   3 POP_TOP
   4 LOAD_CONST   0 (None)
   7 RETURN_VALUE



Before strings and numbers were already ignored. Example:

haypo@smithers$ python3
Python 3.4.3 (default, Jun 29 2015, 12:16:01)


def f():

...  123
...

import dis; dis.dis(f)

   2   0 LOAD_CONST   0 (None)
   3 RETURN_VALUE


Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/tritium-list%40sdamon.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Yury Selivanov]

On 2016-02-08 5:19 PM, Terry Reedy wrote:

On 2/8/2016 4:51 PM, Victor Stinner wrote:

2016-02-08 22:28 GMT+01:00 Alexander Walters :

What incantation do you need to do to make that behavior apparent?


I didn't know. I just checked. It's assert used with a non-empty tuple:


assert ("tuple",)
:1: SyntaxWarning: assertion is always true, perhaps remove 
parentheses?


I think this should be left to linters also.



I agree.  I'd remove that warning.

Yury

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[John Mark Vandenberg]

On Tue, Feb 9, 2016 at 8:51 AM, Victor Stinner  wrote:
> 2016-02-08 22:28 GMT+01:00 Alexander Walters :
>> What incantation do you need to do to make that behavior apparent?
>
> I didn't know. I just checked. It's assert used with a non-empty tuple:
>
 assert ("tuple",)
> :1: SyntaxWarning: assertion is always true, perhaps remove 
> parentheses?

And pyflakes also has a check for this, but it is similarly tight.

https://github.com/pyflakes/pyflakes/pull/51

It seems that the pyflakes maintainers tend to only accepts patches
for scenarios that Python does emit a SyntaxWarning.
So it is a bit of catch-22 there wrt unused constants.

pylint of course reports these unused constants with its message id
"pointless-statement".

-- 
John Vandenberg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Terry Reedy]

On 2/8/2016 4:51 PM, Victor Stinner wrote:

2016-02-08 22:28 GMT+01:00 Alexander Walters :

What incantation do you need to do to make that behavior apparent?


I didn't know. I just checked. It's assert used with a non-empty tuple:


assert ("tuple",)

:1: SyntaxWarning: assertion is always true, perhaps remove parentheses?


I think this should be left to linters also.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Victor Stinner]

2016-02-08 22:28 GMT+01:00 Alexander Walters :
> What incantation do you need to do to make that behavior apparent?

I didn't know. I just checked. It's assert used with a non-empty tuple:

>>> assert ("tuple",)
:1: SyntaxWarning: assertion is always true, perhaps remove parentheses?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Chris Barker]

On Mon, Feb 8, 2016 at 3:48 PM, MRAB  wrote:

> help('assert')
>
> You'll see that in "assert (True,)", the tuple (an object) is the first
> condition (and probably a mistake), whereas in "assert True,", the True is
> the condition and the second expression (after the comma) is missing.


yes, of course, that explains it.

-CHB




>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[MRAB]

On 2016-02-08 23:21, Chris Barker wrote:

On Mon, Feb 8, 2016 at 1:51 PM, Victor Stinner > wrote:

I didn't know. I just checked. It's assert used with a non-empty tuple:

 >>> assert ("tuple",)


which is more interesting with a tuple without the parentheses:

t = In [*4*]: t = True,

In [*5*]: t

Out[*5*]: (True,)

works fine, but not if you use an assert:

In [*7*]: assert True,

   File "", line 1

 assert True,

 ^

SyntaxError:invalid syntax

I actually like the Warning with the note about the problem better:

:1: SyntaxWarning: assertion is always true, perhaps remove
parentheses?


And, of course, more relevant with something Falsey in the tuple:

In [*14*]: assert (False,)

:1: SyntaxWarning: assertion is always
true, perhaps remove parentheses?

   assert (False,)

But I am curious why you get a different error without the parens?


Try:

help('assert')

You'll see that in "assert (True,)", the tuple (an object) is the first 
condition (and probably a mistake), whereas in "assert True,", the True 
is the condition and the second expression (after the comma) is missing.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Chris Barker]

On Mon, Feb 8, 2016 at 1:51 PM, Victor Stinner 
wrote:

> I didn't know. I just checked. It's assert used with a non-empty tuple:
>
> >>> assert ("tuple",)
>

which is more interesting with a tuple without the parentheses:

t = In [*4*]: t = True,

In [*5*]: t

Out[*5*]: (True,)

works fine, but not if you use an assert:

In [*7*]: assert True,

  File "", line 1

assert True,

^

SyntaxError: invalid syntax
I actually like the Warning with the note about the problem better:

:1: SyntaxWarning: assertion is always true, perhaps remove
> parentheses?


And, of course, more relevant with something Falsey in the tuple:

In [*14*]: assert (False,)

:1: SyntaxWarning: assertion is always true,
perhaps remove parentheses?

  assert (False,)

But I am curious why you get a different error without the parens?

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Guido van Rossum]

The warning for 'assert (cond, msg)' was specifically put in because
this is a nasty trap. It's *always* a mistaken attempt to write
'assert cond, msg' -- usually in an attempt to break a long line
without using a backslash. I'd actually consider promoting it to a
syntax error rather than removing the warning.

Compared to other "lint warnings" this one is much nastier -- it is
also much more certain that it is a mistake. (Much more certain than
e.g. an undefined variable, which could still be legitimate code due
to dynamic updates to globals() or builtins.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #26204: compiler now emits a SyntaxWarning on constant statement

[Steven D'Aprano]

On Mon, Feb 08, 2016 at 05:43:25PM -0500, Yury Selivanov wrote:
> 
> 
> On 2016-02-08 5:19 PM, Terry Reedy wrote:
> >On 2/8/2016 4:51 PM, Victor Stinner wrote:
> >>2016-02-08 22:28 GMT+01:00 Alexander Walters :
> >>>What incantation do you need to do to make that behavior apparent?
> >>
> >>I didn't know. I just checked. It's assert used with a non-empty tuple:
> >>
> >assert ("tuple",)
> >>:1: SyntaxWarning: assertion is always true, perhaps remove 
> >>parentheses?
> >
> >I think this should be left to linters also.
> >
> 
> I agree.  I'd remove that warning.


Please don't remove the warning, it is very useful. 

Compare an assertion written correctly:

py> assert 1==2, "Error in arithmetic"
Traceback (most recent call last):
  File "", line 1, in 
AssertionError: Error in arithmetic


with the simple mistake of wrapping the "tuple" in parens:

py> assert (1==2, "Error in arithmetic")
:1: SyntaxWarning: assertion is always true, perhaps remove parentheses?
py> 


This especially hurts people who think that assert is a function. In 
Python 2.5 and older, you get no warning, and can write wrong code:

py> x = 2
py> assert(x==1, 'expected 1 but got %s' % x)
py>


Removing this warning would be a regression.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

[Victor Stinner]

2015-10-02 9:37 GMT+02:00 Nick Coghlan :
> Spell it as "sys.implementation.debug_build" and I'm in favour.

Oh, in fact, I don't have no preference between sys.debug_flag and
sys.implementation.debug_flag. If I understood correctly, Serhiy would
prefer sys.implementation.debug_flag because he doesn't want to add
yet another symbol to the sys namespace.

But Berker Peksag wrote:

"According to the sys.implementation documentation and PEP 421, we can
only add a private attribute without writing a PEP. But I find
sys.implementation._debug_build too long and ``from sys import
implementation; implementation._debug_build``(or ``from sys import
implementation as i; i._debug_build``) is also not easy to write. So
I'm +1 to sys.debug_build."

Should I write a PEP for a new field in sys.implementation?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

[Maciej Fijalkowski]

Speaking of other python implementations - why would you even care?
(the pypy debug build has very different properties and does very
different stuff for example). I would be very happy to have this
clearly marked as implementation-dependent and that's why it would be
cool to not be in sys (there are already 5 symbols there for this
reason, so hasattr totalrefcount is cool enough)

On Fri, Oct 2, 2015 at 2:19 PM, Victor Stinner  wrote:
> 2015-10-02 13:16 GMT+02:00 Nir Soffer :
>> Whats wrong with:
>>
> sysconfig.get_config_var('Py_DEBUG')
>> 0
>
> Again, refer to my first message "On the Internet, I found various
> recipes to check if Python is compiled is debug mode. Sadly, some of
> them are not portable."
>
> I don't think that sysconfig.get_config_var('Py_DEBUG') will work on
> other Python implementations.
>
> On Windows, there is no such file like "Makefile" used to fill
> syscofig.get_config_vars() :-( sysconfig._init_non_posix() only fills
> a few variables like BINDIR or INCLUDEPY, but not Py_DEBUG.
>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

[Barry Warsaw]

On Oct 02, 2015, at 11:46 AM, Victor Stinner wrote:

>Should I write a PEP for a new field in sys.implementation?

Specifically PEP 421 says that a PEP is needed if the new sys.implementation
attribute is required to be defined in all implementations, i.e. it's a new
required attribute.  Will debug_build be required of all implementations?

The PEP can be short.

https://www.python.org/dev/peps/pep-0421/#id30

If it's only relevant for CPython then an underscore-prefix symbol in
sys.implementation is the right place for it, and no PEP is needed.  Just open
an issue on the tracker.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

[Victor Stinner]

2015-10-02 13:16 GMT+02:00 Nir Soffer :
> Whats wrong with:
>
 sysconfig.get_config_var('Py_DEBUG')
> 0

Again, refer to my first message "On the Internet, I found various
recipes to check if Python is compiled is debug mode. Sadly, some of
them are not portable."

I don't think that sysconfig.get_config_var('Py_DEBUG') will work on
other Python implementations.

On Windows, there is no such file like "Makefile" used to fill
syscofig.get_config_vars() :-( sysconfig._init_non_posix() only fills
a few variables like BINDIR or INCLUDEPY, but not Py_DEBUG.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

[Nir Soffer]

Whats wrong with:

>>> sysconfig.get_config_var('Py_DEBUG')
0

Nir

On Fri, Oct 2, 2015 at 10:18 AM, Victor Stinner 
wrote:

> Hi,
>
> I created the issue "Add sys.debug_build public variable to check if
> Python was compiled in debug mode": http://bugs.python.org/issue25256
>
> I would like to add an obvious way to check if Python was compiled in
> debug mode, instead of having hacks/tips to check it. On the Internet,
> I found various recipes to check if Python is compiled is debug mode.
> Sadly, some of them are not portable. For example, 3 different checks
> are proposed on StackOverflow but 2 of them are specific to Windows:
>
> http://stackoverflow.com/questions/646518/python-how-to-detect-debug-interpreter
>
>  Even if the exact impact of a debug build depends on the Python
> implementation and the Python version, we can use it to have the same
> behaviour on all Python implementations. For example, the warnings
> module shows warnings by default if Python is compiled in debug mode:
> Extract of my patch:
>
> -if hasattr(sys, 'gettotalrefcount'):
> +if sys.debug_build:
>  resource_action = "always"
>  else:
>  resource_action = "ignore"
>
> Alternative: Add a new sys.implementation.debug_build flag. Problem:
> extending sys.implementation requires a new PEP, and I don't think
> that debug_build fits into this object.
>
> Berker Peksag likes the idea.
>
> Serhiy Storchaka dislike the new flag: "I don't like this. The sys
> module is one of most used module, but it has too many members, and
> adding yet one makes the situation worse." (sys has 81 symbols)
> "Checking for debug mode is not often needed, and mainly in tests.
> Current way ``hasattr(sys, 'gettotalrefcount')`` works good. You also
> can check ``'d' in sys.abiflags`` if it looks cleaner to you. Or add a
> member to test.support."
>
> The name "debug_build" comes from the existing
> sysconfig.is_python_build() function. There is a sys.flags.debug flag,
> so "sys.debug" can be confusing. I prefer to attach the "build"
> suffix.
>
> First I proposed a function sys.is_debug_build(), but a flag is
> simpler than a function. There is not need to compute a version it's
> known at build time.
>
> What do you think? Should we add sys.debug_build?
>
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/nirsof%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

[Serhiy Storchaka]

On 02.10.15 10:18, Victor Stinner wrote:

I would like to add an obvious way to check if Python was compiled in
debug mode, instead of having hacks/tips to check it. On the Internet,
I found various recipes to check if Python is compiled is debug mode.
Sadly, some of them are not portable.


I agree with Maciej. Why do you need to check if Python is compiled in 
debug mode? Because either you need to use some feature that is 
available only in debug mode (such as sys.gettotalrefcount), or to 
distinguish debug and non-debug binaries (in sysconfig and distutils), 
or make the behaviour different in debug mode (in warnings), or handle 
other behaviour differences (such as additional asserts or different 
stack consumption). In the first case you should just explicitly check 
the existence of related function. In the second case I suggest to use 
sys.abiflags as a suffix (likely you want to distinguish pymalloc from 
non-pymalloc build), or at least make a suffix depended on sys.abiflags. 
In the third case perhaps we have to set default warning level depending 
on sys.flags.debug, or sys.flags.verbose, or other dynamic flags. In the 
fourth case there is no good solution, but in any case this behaviour is 
implementation specific, and other implementation can have different 
consistency checks and different limits.




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #25256: Add sys.debug_build?

[Nick Coghlan]

On 2 October 2015 at 17:18, Victor Stinner  wrote:
> What do you think? Should we add sys.debug_build?

Spell it as "sys.implementation.debug_build" and I'm in favour.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #15014 - SMTP AUTH initial-response (beta exception requested)

[Nick Coghlan]

On 8 July 2015 at 05:12, Barry Warsaw ba...@python.org wrote:
 On Jul 07, 2015, at 02:53 PM, Terry Reedy wrote:

To me, the main question is whether you are sure that your proposal is the
right fix, or whether you might reasonably do something different (with the
new arguments) if changes were reverted for the present and you two took more
time to think about the problem.  My impression is that the latter is
unlikely because the problem is inherent in the new auth methods.

 I generally like the approach that initially added with issue #15014.  This is
 a subtle corner of the RFC and an unexpected regression from Python 3.4.

That strikes me as just the kind of
not-quite-as-finished-as-we-thought case that the beta cycle is
designed to flush out, so the minor further enhancement sounds like a
good idea to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #15014 - SMTP AUTH initial-response (beta exception requested)

[Barry Warsaw]

On Jul 09, 2015, at 08:47 PM, Nick Coghlan wrote:

That strikes me as just the kind of
not-quite-as-finished-as-we-thought case that the beta cycle is
designed to flush out, so the minor further enhancement sounds like a
good idea to me.

Cool.  RDM provided some good feedback in the review, so I'll be committing
this at some point today.

Cheers,
-Barry


pgpnvCoLggNc_.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #15014 - SMTP AUTH initial-response (beta exception requested)

[Barry Warsaw]

On Jul 07, 2015, at 02:53 PM, Terry Reedy wrote:

To me, the main question is whether you are sure that your proposal is the
right fix, or whether you might reasonably do something different (with the
new arguments) if changes were reverted for the present and you two took more
time to think about the problem.  My impression is that the latter is
unlikely because the problem is inherent in the new auth methods.

I generally like the approach that initially added with issue #15014.  This is
a subtle corner of the RFC and an unexpected regression from Python 3.4.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #15014 - SMTP AUTH initial-response (beta exception requested)

[Terry Reedy]

On 7/7/2015 1:52 PM, Barry Warsaw wrote:

Larry and others,

I'd like to bring your attention to issue #15014.  This issue added arbitrary
auth methods to smtplib, which is a good thing.  Implicitly though, a
regression was introduced w.r.t. RFC 4954's optional initial-response for the
AUTH command, for authentication methods that support it.

An example is AUTH PLAIN, which does not require a challenge.  It's possible
that SMTP servers are not prepared to handle challenge/responses for
authentication methods that support initial-response, and regressions have
been seen in the wild while testing against Python 3.5.  In Python 3.4, AUTH
PLAIN included an initial-response.

After discussion on the issue, RDM and I agreed on a refinement to the
authobject() protocol, such that they would be called first with
challenge=None.  If the auth method implemented by the authobject() supports
an initial response, it can return the bytes to be encoded and sent along with
AUTH METHOD.  If it returns None, then a challenge is required, and the
normal SMTP challenge/response can proceed.

A minor complication is that clients using smtplib might want to force
challenge/response instead of initial-response even for authentication methods
that support the latter.  There are various reasons for this, including test
suites (such as Python's own test_smtplib.py).

So I added an optional keyword argument called *initial_response_ok* to
SMTP.auth() and SMTP.login(), with a default value of True.  Thus for
authentication methods that support it, smtplib will by default send an
initial-response, but it can easily be overridden.  Defaulting to True
restores compatibility with Python 3.4.

Technically, this is a new feature


From what you said above, this is not an independent new feature, in 
that you would not have proposed it without the prior patch, but rather, 
in effect, an amendment to the original patch, to make the combined 
effect what the original patch should have been.



even though it addresses a regression in Python 3.5.


The main purpose of releases after feature freeze is to text and fix 
bugs and regressions in added features. The alternative fix would be to 
revert enough of the additions to avoid the regression, and defer such 
to 3.6.


To me, the main question is whether you are sure that your proposal is 
the right fix, or whether you might reasonably do something different 
(with the new arguments) if changes were reverted for the present and 
you two took more time to think about the problem.  My impression is 
that the latter is unlikely because the problem is inherent in the new 
auth methods.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 24285: regression for importing extensions in packages

[Stefan Behnel]

Nick Coghlan schrieb am 28.05.2015 um 05:02:
 On 28 May 2015 at 12:51, Ned Batchelder wrote:
 This issue has been fixed, but a day or two late for 3.5b1.
 
 Aye, we only found out about the missing test case via feedback *on*
 the beta. We had never needed to worry about it before, but it turns
 out all our extension modules in the standard library are top level
 modules and we didn't previously have an explicit test for the
 submodule case :(
 
 It prevents
 loading the coverage.py extension.  It'd be great to get a new beta release
 soon. :)
 
 Until your email, I hadn't fully thought through the consequences, but
 the bug is actually going to block a *lot* of potential testing of the
 beta release - anything that requires a C extension module that isn't
 a top level module isn't going to work with 3.5b1.

+1 for a quick beta 2 from me, too (obviously). I've already seen a bug
report because a Cython compiled package doesn't work in Py3.5. Having to
tell them to wait a while for beta 2 is annoying and discouraging for early
testers.

Stefan


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 24285: regression for importing extensions in packages

[Nick Coghlan]

On 28 May 2015 at 14:30, Larry Hastings la...@hastings.org wrote:
 On 05/27/2015 07:51 PM, Ned Batchelder wrote:

 This issue has been fixed, but a day or two late for 3.5b1.  It prevents
 loading the coverage.py extension.  It'd be great to get a new beta release
 soon. :)


 http://legacy.python.org/dev/peps/pep-0478/

Aye, it's the long gap from May 24 (3.5b1) to July 5 (3.5b2) that we
were hoping to see shortened if it was possible to get the release
team together.

The fact folks can't currently import extension modules that aren't at
top level is unfortunately going to limit the amount of 3.5b1 testing
the community can do in those 6 weeks :(

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 24285: regression for importing extensions in packages

[Yury Selivanov]

On 2015-05-27 11:02 PM, Nick Coghlan wrote:

It prevents
loading the coverage.py extension.  It'd be great to get a new beta release
soon. :)

Until your email, I hadn't fully thought through the consequences, but
the bug is actually going to block a*lot*  of potential testing of the
beta release - anything that requires a C extension module that isn't
a top level module isn't going to work with 3.5b1.


It would also be great to release the new beta shortly with
the new OrderedDict implemented in C and math.isclose().

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 24285: regression for importing extensions in packages

[Nathaniel Smith]

On Wed, May 27, 2015 at 9:54 PM, Nick Coghlan ncogh...@gmail.com wrote:

 On 28 May 2015 at 14:30, Larry Hastings la...@hastings.org wrote:
  On 05/27/2015 07:51 PM, Ned Batchelder wrote:
 
  This issue has been fixed, but a day or two late for 3.5b1.  It prevents
  loading the coverage.py extension.  It'd be great to get a new beta release
  soon. :)
 
 
  http://legacy.python.org/dev/peps/pep-0478/

 Aye, it's the long gap from May 24 (3.5b1) to July 5 (3.5b2) that we
 were hoping to see shortened if it was possible to get the release
 team together.

 The fact folks can't currently import extension modules that aren't at
 top level is unfortunately going to limit the amount of 3.5b1 testing
 the community can do in those 6 weeks :(

Just chiming in to confirm that yep, numpy (and thus the whole
numerical stack) are also missing on 3.5b1. Of course this was filed
as a bug report on numpy :-):

   https://github.com/numpy/numpy/issues/5915

I guess the good news is that people are in fact testing the beta, but
it sounds like pretty much the only code running on b1 is pure Python
code whose entire dependency chain is also pure Python code.

On the upside, compatibility between PyPy and CPython has improved greatly ;-).

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 24285: regression for importing extensions in packages

[Larry Hastings]

On 05/27/2015 07:51 PM, Ned Batchelder wrote:
This issue has been fixed, but a day or two late for 3.5b1.  It 
prevents loading the coverage.py extension.  It'd be great to get a 
new beta release soon. :)


http://legacy.python.org/dev/peps/pep-0478/


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 24285: regression for importing extensions in packages

[Larry Hastings]

On 05/27/2015 08:02 PM, Nick Coghlan wrote:

On 28 May 2015 at 12:51, Ned Batchelder n...@nedbatchelder.com wrote:

This issue has been fixed, but a day or two late for 3.5b1.

Aye, we only found out about the missing test case via feedback *on*
the beta. We had never needed to worry about it before, but it turns
out all our extension modules in the standard library are top level
modules and we didn't previously have an explicit test for the
submodule case :(


Well, certainly this sounds like something that needs to go into the 
regression test suite.  Can someone create the issue?


... and the patch?


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 24285: regression for importing extensions in packages

[Nick Coghlan]

On 28 May 2015 at 12:51, Ned Batchelder n...@nedbatchelder.com wrote:
 This issue has been fixed, but a day or two late for 3.5b1.

Aye, we only found out about the missing test case via feedback *on*
the beta. We had never needed to worry about it before, but it turns
out all our extension modules in the standard library are top level
modules and we didn't previously have an explicit test for the
submodule case :(

 It prevents
 loading the coverage.py extension.  It'd be great to get a new beta release
 soon. :)

Until your email, I hadn't fully thought through the consequences, but
the bug is actually going to block a *lot* of potential testing of the
beta release - anything that requires a C extension module that isn't
a top level module isn't going to work with 3.5b1.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 24285: regression for importing extensions in packages

[Larry Hastings]

On 05/27/2015 10:35 PM, Larry Hastings wrote:
Well, certainly this sounds like something that needs to go into the 
regression test suite.  Can someone create the issue?


... and the patch?


NM, the existing fix already added a test to the regression test suite.  
I should have read the issue first!



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 22619 at bugs.python.org

[Victor Stinner]

http://bugs.python.org/issue22619
Possible implementation of negative limit for traceback functions

I see that Serhiy Storchaka reviewed a patch.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 22619 at bugs.python.org

[Stefan Ring]

On Tue, Jan 6, 2015 at 8:52 AM, Dmitry Kazakov jsb...@gmail.com wrote:
 Greetings.

 I'm sorry if I'm too insistent, but it's not truly rewarding to
 constantly improve a patch that no one appears to need. Again, I
 understand people are busy working and/or reviewing critical patches,
 but 2 months of inactivity is not right. Yes, I posted a message
 yesterday, but no one seemed to be bothered. In any case, I'll respect
 your decision about this patch and will never ask for a review of this
 patch again.

The later patches seem to miss the Mercurial header that would allow
the integrated review functionality on bugs.python.org to kick in (I
presume) and thus make it much easier to review.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 22619 at bugs.python.org

[Mason Hill]

Thank you for pointing this out. That's what I meant when I asked to
say what's wrong with it  :-) I fixed the latest patch and uploaded
it.

On Tue, Jan 6, 2015 at 7:48 AM, Stefan Ring stefan...@gmail.com wrote:
 On Tue, Jan 6, 2015 at 8:52 AM, Dmitry Kazakov jsb...@gmail.com wrote:
 Greetings.

 I'm sorry if I'm too insistent, but it's not truly rewarding to
 constantly improve a patch that no one appears to need. Again, I
 understand people are busy working and/or reviewing critical patches,
 but 2 months of inactivity is not right. Yes, I posted a message
 yesterday, but no one seemed to be bothered. In any case, I'll respect
 your decision about this patch and will never ask for a review of this
 patch again.

 The later patches seem to miss the Mercurial header that would allow
 the integrated review functionality on bugs.python.org to kick in (I
 presume) and thus make it much easier to review.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Yates, Andy (CS Houston, TX)]

Thanks for all the good information.  We ended up building _ssl and _hashlib 
and dropping those into the existing Python on our build server.  That seems to 
be working fine. 

From my perspective ssl libraries are a special case. I think I could handle 
any other included library having a flaw for weeks or months, but my 
management and customers are sensitive to releasing software with known ssl 
vulnerabilities.  For Windows Python it looks like the only option for 
updating OpenSSL is to build from source. For us that turned out to be no big 
deal. However, it may be beyond the reach of some, either technically or due 
to the lack of access to Dev Studio.  There's also some concern that a custom 
build of Python may not have some secret sauce or complier switch that could 
cause unexpected behavior.

That said, I'd like to see Python spin within a short period of time after a 
recognized OpenSSL vulnerability is fixed if is statically linked.  This would 
limit exposure to the unsuspecting user who downloads Windows Python from 
Python.org. The next best thing would be to dynamically link to Windows OpenSSL 
DLLs allowing users to drop in which ever version they like.

Thanks again!!

Andy


-Original Message-
From: Python-Dev [mailto:python-dev-bounces+ayates=hp@python.org] On Behalf 
Of Benjamin Peterson
Sent: Tuesday, June 17, 2014 2:07 PM
To: Ned Deily; python-dev@python.org
Subject: Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h 
on Windows required

On Tue, Jun 17, 2014, at 12:03, Ned Deily wrote:
 In article
 81f84430ce0242e5bfa5b2264777d...@blupr03mb389.namprd03.prod.outlook.c
 om
 ,
  Steve Dower steve.do...@microsoft.com wrote:
  You'll only need to rebuild the _ssl and _hashlib extension modules 
  with the new OpenSSL version. The easiest way to do this is to build 
  from source (which has already been updated for 1.0.1h if you use 
  the externals scripts in Tools\buildbot), and you should just be 
  able to drop _ssl.pyd and _hashlib.pyd on top of a normal install.
 
 Should we consider doing a re-spin of the Windows installers for 2.7.7 
 with 1.0.1h?  Or consider doing a 2.7.8 in the near future to address 
 this and various 2.7.7 regressions that have been identified so far 
 (Issues 21652 and 21672)?

I think we should do a 2.7.8 soon to pick up the openssl upgrade and recent CGI 
security fix. I would like to see those two regressions fixed first, though.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/ayates%40hp.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Cory Benfield]

On 17 June 2014 17:41, Yates, Andy (CS Houston, TX) aya...@hp.com wrote:
 Is it possible to drop in new OpenSSL versions
 on Windows without rebuilding Python?

If you think this is a problem you're going to have more than once,
you'll want to look  hard at whether it's worth using pyOpenSSL
(either the egenix version or the PyCA one[1]) instead, and delivering
binary releases with a bundled copy of OpenSSL. PyOpenSSL from PyCA is
actually considering bundling OpenSSL on Windows anyway[2], so you
might find this problem goes away.

[1] https://github.com/pyca/pyopenssl
[2] https://github.com/pyca/cryptography/issues/1121
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Martin v. Löwis]

Am 17.06.14 18:41, schrieb Yates, Andy (CS Houston, TX):
 Python Dev,
 
 Andy here. I have a Windows product based on Python and I’m getting
 hammered to release a version that includes the fix in OpenSSL 1.0.1h. 
 My product is built on a Windows system using Python installed from the
 standard Python installer at Python.org.  I would be grateful if I could
 get some advice on my options. 

Can you please report
- what version of Python you are distributing?
- why it absolutely has to be 1.0.1h that is included?

According to the CVE, 0.9.8za and 1.0.0m would work as well (and in our
case, would be preferred for older versions of Python).

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Martin v. Löwis]

Am 17.06.14 20:27, schrieb Steve Dower:
 You'll only need to rebuild the _ssl and _hashlib extension modules
 with the new OpenSSL version. The easiest way to do this is to build
 from source (which has already been updated for 1.0.1h if you use the
 externals scripts in Tools\buildbot), and you should just be able to
 drop _ssl.pyd and _hashlib.pyd on top of a normal install.
 
 Aside: I wonder if it's worth changing to dynamically linking to
 OpenSSL? It would make this kind of in-place upgrade easier when
 people need to do it. Any thoughts? (Does OpenSSL even support it?)

We originally considered using prebuilt binaries, such as

http://slproweb.com/products/Win32OpenSSL.html

This is tricky because of CRT issues: they will likely bind to a
different version of the CRT, and
a) it is unclear whether this would reliably work, and
b) requires the Python installer to include a different version of
   the CRT, which we would not have a license to include (as the
   CRT redistribution license only applies to the version of the CRT
   that Python was built with)

There was also the desire to use the same compiler for all code
distributed, to use the same optimizations on all of it. In addition,
for OpenSSL, there is compile time configuration wrt. to the algorithms
built into the binaries where Python's build deviates from the default.

Having a separate project to build a DLL within pcbuild.sln was never
implemented. Doing so possibly increases the risk of DLL hell, if Python
picks up the wrong version of OpenSSL (e.g. if Python gets embedded
into some other application).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Steve Dower]

Yeah, the fact that it really has to be our own copy of the DLL negates the 
advantage. If someone can rebuild that, they could rebuild the modules that 
statically link it.

Cheers,
Steve

Top-posted from my Windows Phone

From: Martin v. Löwismailto:mar...@v.loewis.de
Sent: ‎6/‎18/‎2014 2:46
To: Steve Dowermailto:steve.do...@microsoft.com; Yates, Andy (CS Houston, 
TX)mailto:aya...@hp.com; Python-Dev@python.orgmailto:Python-Dev@python.org
Subject: Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h 
on Windows required

Am 17.06.14 20:27, schrieb Steve Dower:
 You'll only need to rebuild the _ssl and _hashlib extension modules
 with the new OpenSSL version. The easiest way to do this is to build
 from source (which has already been updated for 1.0.1h if you use the
 externals scripts in Tools\buildbot), and you should just be able to
 drop _ssl.pyd and _hashlib.pyd on top of a normal install.

 Aside: I wonder if it's worth changing to dynamically linking to
 OpenSSL? It would make this kind of in-place upgrade easier when
 people need to do it. Any thoughts? (Does OpenSSL even support it?)

We originally considered using prebuilt binaries, such as

http://slproweb.com/products/Win32OpenSSL.html

This is tricky because of CRT issues: they will likely bind to a
different version of the CRT, and
a) it is unclear whether this would reliably work, and
b) requires the Python installer to include a different version of
   the CRT, which we would not have a license to include (as the
   CRT redistribution license only applies to the version of the CRT
   that Python was built with)

There was also the desire to use the same compiler for all code
distributed, to use the same optimizations on all of it. In addition,
for OpenSSL, there is compile time configuration wrt. to the algorithms
built into the binaries where Python's build deviates from the default.

Having a separate project to build a DLL within pcbuild.sln was never
implemented. Doing so possibly increases the risk of DLL hell, if Python
picks up the wrong version of OpenSSL (e.g. if Python gets embedded
into some other application).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Steve Dower]

Yates, Andy (CS Houston, TX) wrote:
 Python Dev,
 Andy here. I have a Windows product based on Python and I'm getting hammered 
 to
 release a version that includes the fix in OpenSSL 1.0.1h. My product is built
 on a Windows system using Python installed from the standard Python installer 
 at
 Python.org. I would be grateful if I could get some advice on my options. Will
 Python.org be releasing a Windows installer with the fix any time soon or will
 it be at the next scheduled release in November? If it is November, there's no
 way I can wait that long. Now what? Would it be best to build my own Python? 
 Is
 it possible to drop in new OpenSSL versions on Windows without rebuilding
 Python? Looking for some guidance on how to handle these OpenSSL issues on
 Windows.

You'll only need to rebuild the _ssl and _hashlib extension modules with the 
new OpenSSL version. The easiest way to do this is to build from source (which 
has already been updated for 1.0.1h if you use the externals scripts in 
Tools\buildbot), and you should just be able to drop _ssl.pyd and _hashlib.pyd 
on top of a normal install.

Aside: I wonder if it's worth changing to dynamically linking to OpenSSL? It 
would make this kind of in-place upgrade easier when people need to do it. Any 
thoughts? (Does OpenSSL even support it?)

Cheers,
Steve

 Thanks!
 Andy Yates
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[M.-A. Lemburg]

On 17.06.2014 20:27, Steve Dower wrote:
 Yates, Andy (CS Houston, TX) wrote:
 Python Dev,
 Andy here. I have a Windows product based on Python and I'm getting hammered 
 to
 release a version that includes the fix in OpenSSL 1.0.1h. My product is 
 built
 on a Windows system using Python installed from the standard Python 
 installer at
 Python.org. I would be grateful if I could get some advice on my options. 
 Will
 Python.org be releasing a Windows installer with the fix any time soon or 
 will
 it be at the next scheduled release in November? If it is November, there's 
 no
 way I can wait that long. Now what? Would it be best to build my own Python? 
 Is
 it possible to drop in new OpenSSL versions on Windows without rebuilding
 Python? Looking for some guidance on how to handle these OpenSSL issues on
 Windows.
 
 You'll only need to rebuild the _ssl and _hashlib extension modules with the 
 new OpenSSL version. The easiest way to do this is to build from source 
 (which has already been updated for 1.0.1h if you use the externals scripts 
 in Tools\buildbot), and you should just be able to drop _ssl.pyd and 
 _hashlib.pyd on top of a normal install.
 
 Aside: I wonder if it's worth changing to dynamically linking to OpenSSL? It 
 would make this kind of in-place upgrade easier when people need to do it. 
 Any thoughts? (Does OpenSSL even support it?)

Yes, no problem at all, but you'd still have to either do a new release
every time a new OpenSSL problem is found (don't think that's an
option for Python) or provide new compiled versions
compatible with the Python modules needing the OpenSSL libs or
instructions on how to build these.

Note that the hash routines are rarely affected by these OpenSSL
bugs. They usually only affect the SSL/TLS protocol parts.

Alternatively, you could make use of our pyOpenSSL distribution,
which includes pyOpenSSL and the OpenSSL libs (also for Windows):

http://www.egenix.com/products/python/pyOpenSSL/

We created this to address the problem of having to update
OpenSSL rather often. It doesn't support Python 3 yet, but
on the plus side, you do get OpenSSL libs which are compiled
with the same compiler versions used for the Python.org
installers.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
 Python/Zope Consulting and Support ...http://www.egenix.com/
 mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
 mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Ned Deily]

In article 
81f84430ce0242e5bfa5b2264777d...@blupr03mb389.namprd03.prod.outlook.com
,
 Steve Dower steve.do...@microsoft.com wrote:
 You'll only need to rebuild the _ssl and _hashlib extension modules with the 
 new OpenSSL version. The easiest way to do this is to build from source 
 (which has already been updated for 1.0.1h if you use the externals scripts 
 in Tools\buildbot), and you should just be able to drop _ssl.pyd and 
 _hashlib.pyd on top of a normal install.

Should we consider doing a re-spin of the Windows installers for 2.7.7 
with 1.0.1h?  Or consider doing a 2.7.8 in the near future to address 
this and various 2.7.7 regressions that have been identified so far 
(Issues 21652 and 21672)?

 Aside: I wonder if it's worth changing to dynamically linking to OpenSSL? It 
 would make this kind of in-place upgrade easier when people need to do it. 
 Any thoughts? (Does OpenSSL even support it?)

OpenSSL is often dynamically linked in Python builds on various other 
platforms, for example, on Linux or OS X.

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Benjamin Peterson]

On Tue, Jun 17, 2014, at 12:03, Ned Deily wrote:
 In article 
 81f84430ce0242e5bfa5b2264777d...@blupr03mb389.namprd03.prod.outlook.com
 ,
  Steve Dower steve.do...@microsoft.com wrote:
  You'll only need to rebuild the _ssl and _hashlib extension modules with 
  the 
  new OpenSSL version. The easiest way to do this is to build from source 
  (which has already been updated for 1.0.1h if you use the externals scripts 
  in Tools\buildbot), and you should just be able to drop _ssl.pyd and 
  _hashlib.pyd on top of a normal install.
 
 Should we consider doing a re-spin of the Windows installers for 2.7.7 
 with 1.0.1h?  Or consider doing a 2.7.8 in the near future to address 
 this and various 2.7.7 regressions that have been identified so far 
 (Issues 21652 and 21672)?

I think we should do a 2.7.8 soon to pick up the openssl upgrade and
recent CGI security fix. I would like to see those two regressions fixed
first, though.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[Antoine Pitrou]

Le 17/06/2014 14:55, M.-A. Lemburg a écrit :


Alternatively, you could make use of our pyOpenSSL distribution,
which includes pyOpenSSL and the OpenSSL libs (also for Windows):

http://www.egenix.com/products/python/pyOpenSSL/

We created this to address the problem of having to update
OpenSSL rather often.


This is very nice, but does it also upgrade the OpenSSL version used by 
the _ssl and _hashlib modules?


Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 21671: CVE-2014-0224 OpenSSL upgrade to 1.0.1h on Windows required

[M.-A. Lemburg]

On 17.06.2014 22:36, Antoine Pitrou wrote:
 Le 17/06/2014 14:55, M.-A. Lemburg a écrit :

 Alternatively, you could make use of our pyOpenSSL distribution,
 which includes pyOpenSSL and the OpenSSL libs (also for Windows):

 http://www.egenix.com/products/python/pyOpenSSL/

 We created this to address the problem of having to update
 OpenSSL rather often.
 
 This is very nice, but does it also upgrade the OpenSSL version used by the 
 _ssl and _hashlib modules?

On Unix, tt will if you load pyOpenSSL before importing _ssl or
_hashlib (and those modules are built as shared libs).

Alternatively, you can set LD_LIBRARY_PATH to lib/python2.7/OpenSSL
to have the system linker use the embedded libs before starting
Python. Then it will always use the up-to-date libs.

On Windows, this won't work, because _ssl and _hashlib are
statically linked against the OpenSSL libs.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 17 2014)
 Python Projects, Consulting and Support ...   http://www.egenix.com/
 mxODBC.Zope/Plone.Database.Adapter ...   http://zope.egenix.com/
 mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2014-06-17: Released eGenix PyRun 2.0.0 ...   http://egenix.com/go58
2014-06-09: Released eGenix pyOpenSSL 0.13.3 ...  http://egenix.com/go57
2014-07-02: Python Meeting Duesseldorf ... 15 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #21205: add __qualname__ to generators

[Victor Stinner]

2014-06-11 18:17 GMT+02:00 Antoine Pitrou anto...@python.org:
 Le 11/06/2014 10:28, Victor Stinner a écrit :
 (...)
 Issues describing the problem, I attached a patch implementing my ideas:
 http://bugs.python.org/issue21205

 Would you be ok with these (minor) incompatible changes?

 +1 from me.

 Regards
 Antoine.

I asked myself if this change can cause issues with serialization. The
marshal and pickle modules cannot serialize a generator. Marshal only
supports a few types. For pickle, I found this explanation:
http://peadrop.com/blog/2009/12/29/why-you-cannot-pickle-generators/

So I consider that my change is safe. It changes the representation of
a generator, but repr() is usually only checked in unit tests, tests
can be fixed. It also changes the value of the __name__ attribute if
the name of the function was changed, but I don't think that anyone
relies on it. If you really want the original name of the code object,
you can still get gen.gi_code.co_name.

Another recent change in the Python API was the __wrapped__ attribute
set by functools.wraps(). It is now chain wrapper functions, and I'm
not aware of anyone complaining of such change. So I'm confident in my
change :)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #21205: add __qualname__ to generators

[Yury Selivanov]

Hello Victor,

On 2014-06-11, 10:28 AM, Victor Stinner wrote:

Hi,

I'm working on asyncio and it's difficult to debug code because
@asyncio.coroutine decorator removes the name of the function if the
function is not a generator (if it doesn't use yield from).

I propose to add new gi_name and gi_qualname fields to the C structure
PyGenObject, add a new __qualname__ (= gi_qualname) attribute to the
Python API of generator, and change how the default value of __name__
(= gi_name) of generators.

Instead of getting the name from the code object, I propose to get the
name from the function (if the generator was created from a function).
So if the function name was modified, you get the new name instead of
getting the name from the code object (as done in Python 3.4).

I also propose to display the qualified name in repr(generator)
instead of the name.

All these changes should make my life easier to debug asyncio, but it
should help any project using generators.

Issues describing the problem, I attached a patch implementing my ideas:
http://bugs.python.org/issue21205

Would you be ok with these (minor) incompatible changes?


I'm +1 for your proposal.

This change will indeed make debugging asyncio (and any generator-heavy 
code) easier.  I wouldn't worry too much about compatibility, as the 
change is fairly minimal, and the feature will only land in 3.5, where 
people expect new things and are generally OK with slightly updated 
behaviors.



Yury


By the way, it looks like generator attributes were never documented
:-( My patch also adds a basic documentation (at least, it lists all
attributes in the documentation of the inspect module).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue #21205: add __qualname__ to generators

[Antoine Pitrou]

Le 11/06/2014 10:28, Victor Stinner a écrit :

Hi,

I'm working on asyncio and it's difficult to debug code because
@asyncio.coroutine decorator removes the name of the function if the
function is not a generator (if it doesn't use yield from).

I propose to add new gi_name and gi_qualname fields to the C structure
PyGenObject, add a new __qualname__ (= gi_qualname) attribute to the
Python API of generator, and change how the default value of __name__
(= gi_name) of generators.

Instead of getting the name from the code object, I propose to get the
name from the function (if the generator was created from a function).
So if the function name was modified, you get the new name instead of
getting the name from the code object (as done in Python 3.4).

I also propose to display the qualified name in repr(generator)
instead of the name.

All these changes should make my life easier to debug asyncio, but it
should help any project using generators.

Issues describing the problem, I attached a patch implementing my ideas:
http://bugs.python.org/issue21205

Would you be ok with these (minor) incompatible changes?


+1 from me.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue with itertools leads the crash

[Mark Lawrence]

On 08/04/2014 17:30, MRAB wrote:

On 2014-04-08 16:31, Brett Cannon wrote:

Something for Python 3.5, maybe? :-)

It's not going to happen in Python 2.7; that's the end of the Python 2
series, and it's getting security fixes only.


According to http://legacy.python.org/dev/peps/pep-0373/ the final 
release of 2.7 is scheduled to be 2.7.9 in May 2015.  Did you mean to 
say that 2.7 isn't getting new features?


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue with itertools leads the crash

[MRAB]

On 2014-04-09 14:26, Mark Lawrence wrote:

On 08/04/2014 17:30, MRAB wrote:

On 2014-04-08 16:31, Brett Cannon wrote:

Something for Python 3.5, maybe? :-)

It's not going to happen in Python 2.7; that's the end of the Python 2
series, and it's getting security fixes only.


According to http://legacy.python.org/dev/peps/pep-0373/ the final
release of 2.7 is scheduled to be 2.7.9 in May 2015.  Did you mean to
say that 2.7 isn't getting new features?


Err, probably... :-(
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue with itertools leads the crash

[Ryan Gonzalez]

2015!?!? I was hoping it was a tad further off...the PyPy team is going to
have to start freaking out in about 12 months.


On Wed, Apr 9, 2014 at 8:26 AM, Mark Lawrence breamore...@yahoo.co.ukwrote:

 On 08/04/2014 17:30, MRAB wrote:

 On 2014-04-08 16:31, Brett Cannon wrote:

 Something for Python 3.5, maybe? :-)

 It's not going to happen in Python 2.7; that's the end of the Python 2
 series, and it's getting security fixes only.


 According to http://legacy.python.org/dev/peps/pep-0373/ the final
 release of 2.7 is scheduled to be 2.7.9 in May 2015.  Did you mean to say
 that 2.7 isn't getting new features?

 --
 My fellow Pythonistas, ask not what our language can do for you, ask what
 you can do for our language.

 Mark Lawrence

 ---
 This email is free from viruses and malware because avast! Antivirus
 protection is active.
 http://www.avast.com



 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe: https://mail.python.org/mailman/options/python-dev/
 rymg19%40gmail.com




-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue with itertools leads the crash

[Geoffrey Spear]

On Wed, Apr 9, 2014 at 1:53 PM, MRAB pyt...@mrabarnett.plus.com wrote:
 On 2014-04-09 14:26, Mark Lawrence wrote:

 On 08/04/2014 17:30, MRAB wrote:

 On 2014-04-08 16:31, Brett Cannon wrote:

 Something for Python 3.5, maybe? :-)

 It's not going to happen in Python 2.7; that's the end of the Python 2
 series, and it's getting security fixes only.


 According to http://legacy.python.org/dev/peps/pep-0373/ the final
 release of 2.7 is scheduled to be 2.7.9 in May 2015.  Did you mean to
 say that 2.7 isn't getting new features?

 Err, probably... :-(

Of course, this raises the question of whether making slice assignment
not go into an infinite loop when the programmer asks it to is a
bugfix or a new feature.

Calling:

list(itertools.cycle([0]))

exhibits the same behavior for the same reason, and I don't think
anyone would call that a bug in Python.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue with itertools leads the crash

[Steven D'Aprano]

On Wed, Apr 09, 2014 at 02:40:21PM -0400, Geoffrey Spear wrote:

 Of course, this raises the question of whether making slice assignment
 not go into an infinite loop when the programmer asks it to is a
 bugfix or a new feature.

Definitely a new feature.

 Calling:
 
 list(itertools.cycle([0]))
 
 exhibits the same behavior for the same reason, and I don't think
 anyone would call that a bug in Python.

It's not a bug, nevertheless it is a problem that can be amiliated. 
Some objects now have a __length_hint__  method. cycle could use that to 
indicate that it was infinitely long, and list could raise an exception 
*before* consuming all available memory.

http://legacy.python.org/dev/peps/pep-0424/


-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com