RE: dangerous class neighborhood

[Avi Gross]

Short answer to a short question.

 

What does a dot (dot dot) mean in pseudocode?

 

NOTHING and EVERYTHING.

 

Longer answer:

 

I think what you see may not be what I wrote. I used three dots in a row (an 
ellipsis) to mean that something is left out to be filled in with something 
appropriate.

 

A = …

B = …

 

That was meant to mean that it does not really matter exactly what code is 
used. Set A equal to SOMETHING and set B equal to something.

 

As it happens, there are computer languages where that is valid code. Python 
turns out to be one of them!

 

 

>>> A=...

>>> A

Ellipsis

 

In python you can often use “pass” in some contexts like:

 

>>> if 1: pass

>>> def name():

Pass

 

These don’t do anything except serve as a placeholder and should be replaced 
later. There are cases where you can write something like this:

 

>>> x = 5

>>> if x > 7:

pass

else:

print(x)

 



5

 

But although that works, you could simply use a NOT or invert the condition to 
say:

 

>>> if not (x > 7):

print(x)

 



5

>>> if (x <= 7):

print(x)

 



5

 

Well “…” can also be used in modern versions of python although it may not be 
novice-level material. It can serve as a placeholder in things like this:

 

>>> def name(x,y,z): pass

 

>>> name(1,2,3)

>>> name(1,3)

Traceback (most recent call last):

  File "", line 1, in 

name(1,3)

TypeError: name() missing 1 required positional argument: 'z'

>>> name(1,...,3)

 

You can see the third positional argument is still the third in this example:

 

>>> def name(x,y,z): print(x,z)

 

>>> name(1,...,3)

1 3

 

So what is the middle argument in this case?

 

>>> def name(x,y,z): print(x,y,z)

 

>>> name(1,...,3)

1 Ellipsis 3

>>> name(...,...,3)

Ellipsis Ellipsis 3

>>> name(...,...,...)

Ellipsis Ellipsis Ellipsis

 

I suspect that generally the intent is to eventually fill in the missing 
argument.

 

Here is a related anomaly. The single underscore has multiple uses but one is 
to say you are throwing away something.

 

>>> a, _, c = (1, "who cares", 3)

>>> a

1

>>> _

'who cares'

>>> c

3

 

But what I used the same variable name multiple times?

 

>>> _, _, c = (1, "who cares", 3)

>>> _

'who cares'

>>> c

3

 

To be fair, any variable used repeatedly in that context keeps only the 
rightmost assignment. But there is an idiom of using just an underscore to mean 
you don’t care about that and want something else whose name may be made 
meaningful.

 

>>> _, _, LastName = ("John", "J.", "Doe")

>>> LastName

'Doe'

 

So, my apologies. I won’t use … even in pseudocode.

 

 

From: Abdur-Rahmaan Janhangeer  
Sent: Friday, December 28, 2018 12:30 AM
To: Avi Gross 
Cc: Python 
Subject: Re: dangerous class neighborhood

 

btw what does

 

A = .

 

above means? the dot?

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club   | 
https://github.com/Abdur-rahmaanJ
Mauritius

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: dangerous class neighborhood

[Avi Gross]

Chris,

You pretty much did exactly what I expected. You took what I wrote and
replaced it with your words and suggested I had said that.

What I wrote this time did not mention python. By now I am far from being a
novice in python but do not claim deep experience or expertise either. In
time, perhaps. In many places I understand it well enough and after the many
replies and some earlier reading I do understand some aspects being
discussed. It is the fact that I do understand what I now think I do, and
ask questions to understand more, that lets me say that it is often a good
idea to bend when reality intrudes and find another way that IS supposed to
work. I posted a number of ways in an earlier message that seem like
reasonable alternatives to try IF your design needs are satisfied. Note that
what satisfies me might not satisfy you. Others have posted other ways we
can evaluate. There does not need to be just one way to do things. Sometimes
even a partial result that works almost all the time is also of value.

What I said next was conditional. FIRST find out why your initial method
fails. If it turns out it is because it is not legal use of python (or
whatever tool) then understand that and find methods that follow the rules.
If you cannot, and perhaps others cannot help you either, then maybe this is
not the right tool. Are you giving up at each stage? Sure. But giving up to
try something else is not the same as giving up and killing yourself. So I
said that when I encountered some problems, I reserve the flexibility to try
other paths to the point of switching tools. In the end, which can happen
upon repeated failures, I may decide it is not worth it.

Two examples and I stress these are examples and imply nothing that you may
choose to dump here.

One example is trying to solve a non-trivial problem using a computer when
any human on the street can solve it just by looking and your best algorithm
might work if run for a decade. Since you need to make a decision soon,
perhaps you can just look out the window and move on and solve other
problems that are more pressing and have no easy solution already available.
Perhaps you can revisit it some day when others have solved enough problems
with computer vision and other technologies and made changes to the legal
system so you can build a relatively safe car that drives itself while
killing few enough people. But doing that hundreds of years ago would not
have been easy, especially before there was a single car on the road. But
horses and carriages seemed available.

Another example is if you determine the problem is intractable. Can you make
an algorithm that simulates the entire universe (whatever that means) in
less than real time. I mean it keeps track of the initial position and
velocity of every particle (including neutrinos and photons and anything
else) to near-infinite precision in the entire  universe as well as the
curvature of space at every possible location in what may turn out to be an
infinite yet expanding realm and calculate how it will progress going
forward trillions of years while also calculating every possible outcome as
many events at the quantum level are in some deep sense probabilistic. Oh,
and do this on a computer that fits into a small room with a standard set of
computers, not some quantum computer. 

Most would agree that if you are working on your Ph.D. thesis and wish to
graduate, pick a different dissertation topic. The above probably can not be
solved with anything smaller than the entire universe which seemingly does
it in real time so asking for it to do it faster using less resources may be
a challenge. I have seen people fail with many smaller problems or find out
it had already been done by someone else and simply pick a new topic to work
on. That does not make them failures.

My point again is what I would do. I would stubbornly try various ways using
the first tool and on repeated failure see if perhaps there was a different
tool to use or build one or perhaps change the problem so it could be solved
and so on. But, ultimately, if daunted, I would not drop everything else in
my life but would indeed stop trying.

If you suggest you would do something better, feel free.

At some point a discussion or debate can switch to just insults and I
suggest I would prefer to have a meaningful interaction instead. I did learn
from this discussion and know ways I can write code to meet such needs. So I
fail to understand what you are saying. I make no assertions I can meet your
needs if they are not reasonable. No solution can exist if you want an
irresistible force to meet an unmovable object.

I suggest that if your attitude persists, you are the one making this a
dangerous neighborhood.

And further replies by me in public or private would serve no purpose. 

-Original Message-
From: Python-list  On
Behalf Of Chris Angelico
Sent: Friday, December 28, 2018 12:08 AM
To: Python 
Subject: Re: dangerous class neighborhood

Re: dangerous class neighborhood

[Abdur-Rahmaan Janhangeer]

btw what does

A = .

above means? the dot?

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: dangerous class neighborhood

[Chris Angelico]

On Fri, Dec 28, 2018 at 2:27 PM Avi Gross  wrote:
>
> Sometimes when I post something I get back comments and evaluate them and
> learn quite a bit. I then reply and debate every little point and it can
> continue for a few rounds.
>
> I don't seem to be in that mood today so let me simply restate my entire
> post in a few sentences with no examples, no lectures, no advice on what
> anyone else can do and very little for anyone to bother replying to. Here
> goes:
>
> Sometimes when I run up against a wall and find that a solution to a problem
> does not work because things may not work as I expected, I pause. I
> reconsider what I actually need to get done. Then I look to see if I can
> come up with other ways to do it that will work while still getting the
> important parts done. Failing that, I ask if perhaps there is another tool,
> such as another programming language that is a better fit for the task. And,
> if the work needed seems excessive, I ask if perhaps the problem does not
> really need to be solved by me and I move on.

So your entire original post can be summed up as "I don't understand
Python, and I don't care"? Why did you post it then? I was under the
impression that you wanted to learn what was going on.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: dangerous class neighborhood

[Avi Gross]

Sometimes when I post something I get back comments and evaluate them and
learn quite a bit. I then reply and debate every little point and it can
continue for a few rounds.

I don't seem to be in that mood today so let me simply restate my entire
post in a few sentences with no examples, no lectures, no advice on what
anyone else can do and very little for anyone to bother replying to. Here
goes:

Sometimes when I run up against a wall and find that a solution to a problem
does not work because things may not work as I expected, I pause. I
reconsider what I actually need to get done. Then I look to see if I can
come up with other ways to do it that will work while still getting the
important parts done. Failing that, I ask if perhaps there is another tool,
such as another programming language that is a better fit for the task. And,
if the work needed seems excessive, I ask if perhaps the problem does not
really need to be solved by me and I move on.

-Original Message-
From: Python-list  On
Behalf Of Chris Angelico
Sent: Thursday, December 27, 2018 5:11 PM
To: Python 
Subject: Re: dangerous class neighborhood

On Fri, Dec 28, 2018 at 8:47 AM Avi Gross  wrote:
> Question 2: Do you want the variables available at the class level or 
> at the instance level?

For constants, definitely put them on the class. They'll be available on
instances as well ("for free", if you like). For mutables, obviously you
need to decide on a case-by-case basis.

> Question 3: Which python variations on syntactic sugar, such as list 
> comprehensions, get expanded invisibly in ways that make the problem 
> happen by asking for variables to be found when no longer in the 
> visible
range?

The oddities with comprehensions were tackled partly during the discussion
of PEP 572. If you want to know exactly why this isn't changing, go read a
few hundred emails on the subject. A lot of the main points are summarized
in the PEP itself:

https://www.python.org/dev/peps/pep-0572/

> There may be matters of efficiency some would consider but some of the 
> examples seen recently seemed almost silly and easy to compute. The 
> people asking about this issue wanted to define a bunch of CONSTANTS, 
> or things that might as well be constants, like this:
>
>
>
> def Foo():
>
> A = ("male", "female", "other")
>
> B = [ kind[0] for kind in A ]# First letters
> only
>
> # And so on making more constants like a dictionary 
> mapping each string to a number or vice versa.
>
>
>
> All the above can be evaluated at the time the class is defined but 
> unintuitive scope rules make some operations fail as variables defined 
> in the scope become unavailable to other things that SEEM to be 
> embedded in the same scope.

If you write simple and Pythonic code, these will almost always work
perfectly. The recent thread citing an oddity worked just fine until it was
written to iterate over range(len(x)) instead of iterating directly.

> If they are ONLY to be used within an instance of Foo or invoked from 
> within there, there may be a fairly simple suggestion. If you already 
> have a __init__ method, then instantiate the variables there carefully 
> using the self object to reference those needed.

But why? __init__ should initialize an instance, not class-level constants.
A Python class is not restricted to just methods, and there's no reason to
avoid class attributes.

> Create a function either outside the class or defined within. Have it 
> do any internal calculations you need in which all internal variables 
> can play nicely with each other. Then let it return all the variables 
> in a tuple like
> this:
>
> def make_sexual_constants():
>
> A = .
>
> B = .
>
> C = f(A,B)
>
> D = .
>
> def Foo():
>
> (A, B, C, D) = make_sexual_constants():

Lovely. Now you have to define your variables once inside the function, then
name them a second time in that function's return statement, and finally
name them all a *third* time in the class statement (at least, I presume
"def Foo():" is meant to be "class Foo:"). A mismatch will create bizarre
and hard-to-debug problems.
What do you actually gain? Can you show me real-world code that would truly
benefit from this?

> Can we agree that the class Foo now has those 4 variables defined and 
> available at either the class level or sub-class or instance levels?
> But the values are created, again, in a unified safe environment?

Unified? No more so than the class statement itself. Safe? Definitely not,
because of the mandatory duplication of names.

> As noted in section 3, it would be good to know what python features 
> may be unsafe in this kind of context. I had an unrelated recent 
> discussion where it was mentioned that some proposed feature changes 
> might not be thread safe. Valid consideration when that may lead to
hard-to-explain anomalies.

Uhh. nope, 

Re: Undocumented issue: Open system call blocks on named pipes (and a feature request)

[Chris Angelico]

On Fri, Dec 28, 2018 at 1:38 PM Daniel Ojalvo via Python-list
 wrote:
>
> Hello,
>
> I've been working on a python3 project and I came across an issue with the 
> open system call that, at the very least, isn't documented. In my humble 
> opinion, the 
> documentation should 
> be updated because folks wouldn't expect open to be a blocking operation and 
> simply error out. Worse yet, open doesn't have an option to make itself 
> non-blocking. You have to use the os system calls to kludge a solution.
>

Hmm. I disagree that the docs are deceptive here; I would normally
expect open() to block if it needs to. But looking at this as a
feature request, it seems reasonable. Actually, it's not even that
hard to do, since open() is already pluggable:

rosuav@sikorsky:~/tmp$ rm rene_magritte
rosuav@sikorsky:~/tmp$ mkfifo rene_magritte
rosuav@sikorsky:~/tmp$ ls -l rene_magritte
prw-r--r-- 1 rosuav rosuav 0 Dec 28 14:05 rene_magritte
rosuav@sikorsky:~/tmp$ python3
Python 3.8.0a0 (heads/master:8b9c33ea9c, Nov 20 2018, 02:18:50)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> def nonblock(fn, mode): return os.open(fn, mode | os.O_NONBLOCK)
...
>>> open("rene_magritte", opener=nonblock)
<_io.TextIOWrapper name='rene_magritte' mode='r' encoding='UTF-8'>
>>> _.read(1)
''

> Here is how I reproduced the issue:
>
> root@beefy:~/sandbox# mkfifo this_is_a_pipe

(my example file name is a more subtle reference...)

> I'm doing this to get a fileobject and make it error out if we do have a 
> blocking special file:
> with os.fdopen(os.open(, os.O_RDONLY| os.O_NONBLOCK) , mode='rb') 
> as file_obj:
>
> I think this is mostly a documentation bug because this wouldn't be expected 
> behavior to someone reading the docs, but open is behaving as the fifo man 
> page is documented. The 
> feature request would be to add a non-blocking option to the default open 
> system call.
>

Honestly, I don't think there's a problem with it blocking by default.
Most of Python works that way. But it would be pretty straight-forward
to add "nonblocking=False" as another keyword-only parameter, and for
compatibility with existing versions (back as far as 3.3), the opener
should work just fine.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Undocumented issue: Open system call blocks on named pipes (and a feature request)

[Daniel Ojalvo via Python-list]

Hello,

I've been working on a python3 project and I came across an issue with the open 
system call that, at the very least, isn't documented. In my humble opinion, 
the documentation should 
be updated because folks wouldn't expect open to be a blocking operation and 
simply error out. Worse yet, open doesn't have an option to make itself 
non-blocking. You have to use the os system calls to kludge a solution.

Here is how I reproduced the issue:

root@beefy:~/sandbox# mkfifo this_is_a_pipe
root@beefy:~/sandbox# ls -l this_is_a_pipe
prw-r--r-- 1 root root 0 Dec 27 14:28 this_is_a_pipe
root@beefy:~/sandbox# python3 --version
Python 3.6.7
root@beefy:~/sandbox# python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> open("this_is_a_pipe")


The mitigation is to use the os calls and specify to be nonblocking when 
opening the file.

I'm doing this to get a fileobject and make it error out if we do have a 
blocking special file:
with os.fdopen(os.open(, os.O_RDONLY| os.O_NONBLOCK) , mode='rb') as 
file_obj:

I think this is mostly a documentation bug because this wouldn't be expected 
behavior to someone reading the docs, but open is behaving as the fifo man 
page is documented. The 
feature request would be to add a non-blocking option to the default open 
system call.

I've also found this with some named special character files, but I don't have 
a reproduction at the moment.

Thank you and have a good day!
Dan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ask for help about class variable scope (Re: Why doesn't a dictionary work in classes?)

[Chris Angelico]

On Fri, Dec 28, 2018 at 8:47 AM eryk sun  wrote:
>
> On 12/27/18, Chris Angelico  wrote:
> >
> > Class scope is special, and a generator expression within that class
> > scope is special too. There have been proposals to make these kinds of
> > things less special, but the most important thing to remember is that
> > when you create a generator expression, it is actually a function.
> > Remember that a function inside a class statement becomes a method,
> > and that inside the method, you have to use "self.X" rather than just
> > "X" to reference class attributes. That's what's happening here.
>
> A generator expression is implemented internally as a generator
> function that takes an iterator as its only parameter (named ".0") and
> gets called immediately to get a generator object. There's some inline
> bytecode in the defining scope that sets this up.
>
> A generator object has iterator methods (__iter__, __next__) and
> close, send, and throw methods. Its code and execution state in
> CPython uses a code object and a frame object:

A generator expression is an expression that creates and then calls a
function to create a generator object. This is exactly the same as any
other generator function being called to create a generator object.
Yes, the generator function and the generator object are very
different beasts, and yes, they're both called "a generator", and yes,
this is confusing. But it's the same whether it's a genexp or an
explicit function.

> Unlike a function object, a generator object is not a descriptor (i.e.
> it has no __get__ method) that could in principle be bound as either a
> class or instance method. Anyway, since the class doesn't exist yet,
> trying to bind and call a method at this point can't work.

This is correct. A genexp created at class scope would be
approximately equivalent to a generator function called as "Foo.f()"
not as "Foo().f()" - called on the class, not an instance of it.

> This is straying off topic, but note that a consequence of late
> binding is that super() can be broken by rebinding __class__:
>
> class C:
> def f(self):
> nonlocal __class__
> __class__ = None
> super()
>
> >>> C().f()
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "", line 5, in f
> RuntimeError: super(): __class__ is not a type (NoneType)
>
> It's not a bug or design flaw; just an example of code shooting itself
> in the foot by stepping on an informally reserved dunder name.

Technically only the no-arg form of super() can be broken that way,
because it's actually a bit of magic syntax that becomes
"super(__class__, self)", except that it doesn't actually use "self"
but "whatever the first argument to this function is". You can break
it in an even simpler way:

>>> class C:
... def f():
... super()
...
>>> C.f()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 3, in f
RuntimeError: super(): no arguments

"But Mum, I thought you said you wanted NO ARGUMENTS while you were out"

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: dangerous class neighborhood

[Chris Angelico]

On Fri, Dec 28, 2018 at 8:47 AM Avi Gross  wrote:
> Question 2: Do you want the variables available at the class level or at the
> instance level?

For constants, definitely put them on the class. They'll be available
on instances as well ("for free", if you like). For mutables,
obviously you need to decide on a case-by-case basis.

> Question 3: Which python variations on syntactic sugar, such as list
> comprehensions, get expanded invisibly in ways that make the problem happen
> by asking for variables to be found when no longer in the visible range?

The oddities with comprehensions were tackled partly during the
discussion of PEP 572. If you want to know exactly why this isn't
changing, go read a few hundred emails on the subject. A lot of the
main points are summarized in the PEP itself:

https://www.python.org/dev/peps/pep-0572/

> There may be matters of efficiency some would consider but some of the
> examples seen recently seemed almost silly and easy to compute. The people
> asking about this issue wanted to define a bunch of CONSTANTS, or things
> that might as well be constants, like this:
>
>
>
> def Foo():
>
> A = ("male", "female", "other")
>
> B = [ kind[0] for kind in A ]# First letters
> only
>
> # And so on making more constants like a dictionary mapping
> each string to a number or vice versa.
>
>
>
> All the above can be evaluated at the time the class is defined but
> unintuitive scope rules make some operations fail as variables defined in
> the scope become unavailable to other things that SEEM to be embedded in the
> same scope.

If you write simple and Pythonic code, these will almost always work
perfectly. The recent thread citing an oddity worked just fine until
it was written to iterate over range(len(x)) instead of iterating
directly.

> If they are ONLY to be used within an instance of Foo or invoked from within
> there, there may be a fairly simple suggestion. If you already have a
> __init__ method, then instantiate the variables there carefully using the
> self object to reference those needed.

But why? __init__ should initialize an instance, not class-level
constants. A Python class is not restricted to just methods, and
there's no reason to avoid class attributes.

> Create a function either outside the class or defined within. Have it do any
> internal calculations you need in which all internal variables can play
> nicely with each other. Then let it return all the variables in a tuple like
> this:
>
> def make_sexual_constants():
>
> A = .
>
> B = .
>
> C = f(A,B)
>
> D = .
>
> def Foo():
>
> (A, B, C, D) = make_sexual_constants():

Lovely. Now you have to define your variables once inside the
function, then name them a second time in that function's return
statement, and finally name them all a *third* time in the class
statement (at least, I presume "def Foo():" is meant to be "class
Foo:"). A mismatch will create bizarre and hard-to-debug problems.
What do you actually gain? Can you show me real-world code that would
truly benefit from this?

> Can we agree that the class Foo now has those 4 variables defined and
> available at either the class level or sub-class or instance levels? But the
> values are created, again, in a unified safe environment?

Unified? No more so than the class statement itself. Safe? Definitely
not, because of the mandatory duplication of names.

> As noted in section 3, it would be good to know what python features may be
> unsafe in this kind of context. I had an unrelated recent discussion where
> it was mentioned that some proposed feature changes might not be thread
> safe. Valid consideration when that may lead to hard-to-explain anomalies.

Uhh. nope, that's nothing but FUD. There is no reason to believe
that some language features would be "unsafe".

> We now hear that because a list comprehension can be unwound internally into
> a "while" loop and an "if" statement and that some parts may expand to calls
> to a "range" statement, perhaps some variables are now in more deeply
> embedded contexts that have no access to any class variables.

No idea what you're looking at. A comprehension can be unwound in a
fairly straight-forward way, although there are some subtleties to
them.

B = [ kind[0] for kind in A ]
# equivalent to, approximately:
def listcomp(iter):
result = []
for kind in iter:
result.append(kind[0])
return result
B = listcomp(A)

For casual usage, you can describe a list comp very simply and neatly:

B = [ kind[0] for kind in A ]
# equivalent to, more approximately:
B = []
for kind in A:
B.append(kind[0])

Nothing here expands to a call to range(), nothing has a while loop.
The only way you'll get an "if" is if you had one in the comprehension
itself.

> I think that
> is quite reasonable; hence my suggestion we need to know which ones to
> avoid, or use 

Re: Ask for help about class variable scope (Re: Why doesn't a dictionary work in classes?)

[eryk sun]

On 12/27/18, Chris Angelico  wrote:
>
> Class scope is special, and a generator expression within that class
> scope is special too. There have been proposals to make these kinds of
> things less special, but the most important thing to remember is that
> when you create a generator expression, it is actually a function.
> Remember that a function inside a class statement becomes a method,
> and that inside the method, you have to use "self.X" rather than just
> "X" to reference class attributes. That's what's happening here.

A generator expression is implemented internally as a generator
function that takes an iterator as its only parameter (named ".0") and
gets called immediately to get a generator object. There's some inline
bytecode in the defining scope that sets this up.

A generator object has iterator methods (__iter__, __next__) and
close, send, and throw methods. Its code and execution state in
CPython uses a code object and a frame object:

>>> g = (c for c in 'spam')
>>> g.gi_code.co_varnames
('.0', 'c')

Initially it hasn't run, so there's no 'c' value yet:

>>> sorted(g.gi_frame.f_locals)
['.0']

'c' is defined after it executes up to the first yield:

>>> next(g)
's'
>>> sorted(g.gi_frame.f_locals)
['.0', 'c']

Unlike a function object, a generator object is not a descriptor (i.e.
it has no __get__ method) that could in principle be bound as either a
class or instance method. Anyway, since the class doesn't exist yet,
trying to bind and call a method at this point can't work.

In contrast generator functions are commonly used for methods that can
access class attributes. But this is confusing matters since the class
__dict__ (and certainly not the instance __dict__) that's used for
attributes is not the locals() of the initial class statement
execution. In other words, class attribute access is not a closure
over the class statement scope. For methods, it depends on early
binding of the bound object to a method's __self__ attribute, which is
implicitly passed as the first argument to its __func__ function when
the method is called.

The execution locals of a class statement is a temporary namespace
that gets copied when the class object is instantiated. For example:

class C:
exec_dict = locals()

>>> C.s = 'spam'
>>> sorted(C.__dict__)
['__dict__', '__doc__', '__module__', '__weakref__', 'exec_dict', 's']

The original locals():

>>> sorted(C.exec_dict)
['__module__', '__qualname__', 'exec_dict']

What could be done is for the compiler to introduce nonlocal free
variables (like what's already done with __class__), and then capture
the current value to the locals() dict after it's done executing. For
example (a clumsy one; in practice it would be implicit, expanding on
how __class__ is implemented):

def make_Foo():
XS = None
class Foo:
nonlocal XS
XS = [15] * 4
Z5 = sum(XS[i] for i in range(len(XS)))
locals()['XS'] = XS
return Foo

>>> Foo = make_Foo()
>>> Foo.Z5
60
>>> Foo.XS
[15, 15, 15, 15]

However, this would be confusing in general, since there's no way to
keep the class attribute in sync with the cell variable. So a function
that's called as a class method may see different values for XS and
cls.XS. This is a bad idea.

This is straying off topic, but note that a consequence of late
binding is that super() can be broken by rebinding __class__:

class C:
def f(self):
nonlocal __class__
__class__ = None
super()

>>> C().f()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 5, in f
RuntimeError: super(): __class__ is not a type (NoneType)

It's not a bug or design flaw; just an example of code shooting itself
in the foot by stepping on an informally reserved dunder name.
-- 
https://mail.python.org/mailman/listinfo/python-list

dangerous class neighborhood

[Avi Gross]

There have been several discussions about unexpected behavior when people
write programs within a class definition. Some suggest various ways around
the problem and that is fine although I bet new people will keep
encountering this issue.

 

I have some questions about what people want to do with this kind of
programming and for each use, there may be suggested ways to get things done
and get the right result in a reasonably natural way.

 

I have some overall concerns regarding the aggressive approach taken in
Python toward supporting too many features in what is loosely considered
object-oriented programming to the point where some combinations interact in
mathematically intractable ways resulting in concessions and compromises
such as the one we see.

 

Question 1: Do you want evaluation at class definition time or at run time?

 

Question 2: Do you want the variables available at the class level or at the
instance level?

 

Question 3: Which python variations on syntactic sugar, such as list
comprehensions, get expanded invisibly in ways that make the problem happen
by asking for variables to be found when no longer in the visible range?

 

There may be matters of efficiency some would consider but some of the
examples seen recently seemed almost silly and easy to compute. The people
asking about this issue wanted to define a bunch of CONSTANTS, or things
that might as well be constants, like this:

 

def Foo():

A = ("male", "female", "other")

B = [ kind[0] for kind in A ]# First letters
only

# And so on making more constants like a dictionary mapping
each string to a number or vice versa.

 

All the above can be evaluated at the time the class is defined but
unintuitive scope rules make some operations fail as variables defined in
the scope become unavailable to other things that SEEM to be embedded in the
same scope.

 

So, are these constants being called as Foo.A or are they used after an
instance is created like:

 

Aleph = Foo()

# Use Aleph.A 

 

If they are ONLY to be used within an instance of Foo or invoked from within
there, there may be a fairly simple suggestion. If you already have a
__init__ method, then instantiate the variables there carefully using the
self object to reference those needed.

 

def Foo():

def __init__(self):

# calculate A and B and whatever.

self.A = A

self.B = B

# .

These calculations would not be at declaration time for class Foo but the
results would be created every time an object of class Foo came along. If
Foo is subclassed, it would be wise to make sure the initialization reaches
up to run Foo.__init__ properly too.

 

Within __init__ I presume all the USUAL scoping rules make sense so if
making a B includes using A, it would easily be found there first. Again,
not self.A, just A at that stage. You can create self.A near the end of the
initialization when all calculations needed are complete.

 

Second suggestion, even if what is being made may not be constants. 

 

Create a function either outside the class or defined within. Have it do any
internal calculations you need in which all internal variables can play
nicely with each other. Then let it return all the variables in a tuple like
this:

 

def make_sexual_constants():

A = .

B = .

C = f(A,B)

D = .

def Foo():

(A, B, C, D) = make_sexual_constants():

 

Can we agree that the class Foo now has those 4 variables defined and
available at either the class level or sub-class or instance levels? But the
values are created, again, in a unified safe environment?

 

As noted in section 3, it would be good to know what python features may be
unsafe in this kind of context. I had an unrelated recent discussion where
it was mentioned that some proposed feature changes might not be thread
safe. Valid consideration when that may lead to hard-to-explain anomalies.  

 

We now hear that because a list comprehension can be unwound internally into
a "while" loop and an "if" statement and that some parts may expand to calls
to a "range" statement, perhaps some variables are now in more deeply
embedded contexts that have no access to any class variables. I think that
is quite reasonable; hence my suggestion we need to know which ones to
avoid, or use a workaround like expanding it out ourselves and perhaps
carefully import variables into other contexts such as by passing the
variable into the function that otherwise cannot access it from a point it
can still be seen.

 

Least, but at least last, I ask if the need really exists for these
variables as constants versus functions. If creating C this way runs into
problems, but A and B are fine, consider making a method with some name like
Foo.get_C() that can see A and B and do the calculation and yet return the
value of C needed. L

Re: Ask for help about class variable scope (Re: Why doesn't a dictionary work in classes?)

[Chris Angelico]

On Fri, Dec 28, 2018 at 3:31 AM Ian Kelly  wrote:
>
> On Wed, Dec 26, 2018 at 11:21 PM Chris Angelico  wrote:
> >
> > On Thu, Dec 27, 2018 at 1:56 PM  wrote:
> > >
> > > I saw the code below at stackoverflow. I have a little idea about the 
> > > scope of a class, and list comprehension and generator expressions, but 
> > > still can't figure out why Z4 works and Z5 not. Can someone explain it? 
> > > (in a not-too-complicated way:-)
> > >
> > > class Foo():
> > > XS = [15, 15, 15, 15]
> > > Z4 = sum(val for val in XS)
> > > try:
> > > Z5 = sum(XS[i] for i in range(len(XS)))
> > > except NameError:
> > > Z5 = None
> > >
> > > print(Foo.Z4, Foo.Z5)
> > > >>> 60 None
> > >
> >
> > Class scope is special, and a generator expression within that class
> > scope is special too. There have been proposals to make these kinds of
> > things less special, but the most important thing to remember is that
> > when you create a generator expression, it is actually a function.
> > Remember that a function inside a class statement becomes a method,
> > and that inside the method, you have to use "self.X" rather than just
> > "X" to reference class attributes. That's what's happening here.
>
> Except you can't use "self" either because the class doesn't exist yet
> at the time the generator expression is being evaluated.

Well, yes. But that's the easiest way to highlight the difference of scope.

> Nor can you
> use "self" generally even if you wait until the class does exist
> before you iterate over the generator, because while a generator
> expression may scope like a function, and may under the hood be
> implemented as a function, a generator object is not a function object
> and will not create a method.

Hmm. It creates a function which is called at class scope. Based on
disassembly, I can't see a difference between the function created for
a genexp and the one created for a method. What IS true, though, is
that there's no way to pass arguments to the genexp, which means that
it won't be passed a 'self'. So if you wait till the class exists
before iterating over the generator, you can name the class, but you
can't use self:

>>> class Foo:
... word = "spam"
... genexp = (l.upper() for shim in [...] for l in Foo.word)
...
>>> list(Foo.genexp)
['S', 'P', 'A', 'M']

I would personally have preferred for the official Python language
definition to have described comprehensions and genexps as creating a
scope boundary, without baking into the language "they're hidden,
implicit functions". But that ship has sailed, and when I debated the
point, all Pandora's box got opened.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ask for help about class variable scope (Re: Why doesn't a dictionary work in classes?)

[Ian Kelly]

On Wed, Dec 26, 2018 at 11:21 PM Chris Angelico  wrote:
>
> On Thu, Dec 27, 2018 at 1:56 PM  wrote:
> >
> > I saw the code below at stackoverflow. I have a little idea about the scope 
> > of a class, and list comprehension and generator expressions, but still 
> > can't figure out why Z4 works and Z5 not. Can someone explain it? (in a 
> > not-too-complicated way:-)
> >
> > class Foo():
> > XS = [15, 15, 15, 15]
> > Z4 = sum(val for val in XS)
> > try:
> > Z5 = sum(XS[i] for i in range(len(XS)))
> > except NameError:
> > Z5 = None
> >
> > print(Foo.Z4, Foo.Z5)
> > >>> 60 None
> >
>
> Class scope is special, and a generator expression within that class
> scope is special too. There have been proposals to make these kinds of
> things less special, but the most important thing to remember is that
> when you create a generator expression, it is actually a function.
> Remember that a function inside a class statement becomes a method,
> and that inside the method, you have to use "self.X" rather than just
> "X" to reference class attributes. That's what's happening here.

Except you can't use "self" either because the class doesn't exist yet
at the time the generator expression is being evaluated. Nor can you
use "self" generally even if you wait until the class does exist
before you iterate over the generator, because while a generator
expression may scope like a function, and may under the hood be
implemented as a function, a generator object is not a function object
and will not create a method.
-- 
https://mail.python.org/mailman/listinfo/python-list

Facing an Error after migrating from python 3.4.1 to python 3.6.6 ( Failed to import the site module )

[sandeep . bayi6]

``` 
Error code: 
-- 


Traceback (most recent call last): 
  File 
"C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\site.py", line 
73, in  
from _collections_abc import MutableMapping 
  File 
"C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\_collections_abc.py",
 line 58 
async def _coro(): pass 
^ 
SyntaxError: invalid syntax 
Failed to import the site module 

 
After migrating from python 3.4.1 to python 3.6.6 
while Executing my project, I'm facing this issue and not able to resolve it. 
Can i get any solution for this issue? 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Getting Error after migrating from python 3.4.1 to python 3.6.6 ( Failed to import the site module )

[sandeep . bayi6]

On Thursday, December 27, 2018 at 7:49:16 PM UTC+5:30, sandee...@gmail.com 
wrote:
> ```
> Error code:
> --
> 
> 
> Traceback (most recent call last):
>   File 
> "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\site.py", 
> line 73, in  import os
>   File 
> "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\os.py", line 
> 652, in 
> from _collections_abc import MutableMapping
>   File 
> "C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\_collections_abc.py",
>  line 58
> async def _coro(): pass
> ^
> SyntaxError: invalid syntax
> Failed to import the site module
> 
> 
> After migrating from python 3.4.1 to python 3.6.6
> while Executing my project, I'm facing this issue and not able to resolve it. 
> Can i find any solution for this error?

-- 
https://mail.python.org/mailman/listinfo/python-list

Getting Error after migrating from python 3.4.1 to python 3.6.6 ( Failed to import the site module )

[sandeep . bayi6]

```
Error code:
--


Traceback (most recent call last):
  File 
"C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\site.py", line 
73, in 
from _collections_abc import MutableMapping
  File 
"C:\Users\sandeep\AppData\Local\Programs\Python\Python36-32\Lib\_collections_abc.py",
 line 58
async def _coro(): pass
^
SyntaxError: invalid syntax
Failed to import the site module


After migrating from python 3.4.1 to python 3.6.1
while Executing my project, I'm facing this issue and not able to resolve it

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do data descriptors (e.g. properties) take priority over instance attributes?

[Peter J. Holzer]

On 2018-12-17 21:57:22 +1100, Paul Baker wrote:
> class C:
> def __init__(self):
> self.__dict__['a'] = 1
> 
> @property
> def a(self):
> return 2
> 
> o = C()
> print(o.a)  # Prints 2

What version of Python is this? I get a different result:

Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
...   def __init__(self):
... self.__dict__['a'] = 1
... @property
... def a(self):
...   return 2
... 
>>> o = C()
>>> o.a
1

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do data descriptors (e.g. properties) take priority over instance attributes?

[Peter J. Holzer]

On 2018-12-27 14:17:34 +0100, Peter J. Holzer wrote:
> On 2018-12-17 21:57:22 +1100, Paul Baker wrote:
> > class C:
> > def __init__(self):
> > self.__dict__['a'] = 1
> > 
> > @property
> > def a(self):
> > return 2
> > 
> > o = C()
> > print(o.a)  # Prints 2
> 
> What version of Python is this? I get a different result:
> 
> Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
> [GCC 8.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> class C:
> ...   def __init__(self):
> ... self.__dict__['a'] = 1
> ... @property
> ... def a(self):
> ...   return 2
> ... 
> >>> o = C()
> >>> o.a
> 1

Oops. I messed up the indentation. 

With the same indentation I get the same result.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

next steps to make a python program more available

[ant]

  ok, i have the program going and installable for my
local Linux distribution.  aka i have it packaged and
on PyPI and it runs from my local terminal command line.

  i have people available for testing it out on Windows
and a Mac, but i don't have any way of knowing what to
do to make the program appear in their Menu system to
make it clickable.

  in MATE (which is what i run here) i can figure it out
as something to do with the desktop, so somehow i have
to do that, but i don't know what keywords to even search
for to do that, desktop icons i guess and menu, but is
there a more global way to set that up?  XDG seems to be
a part of that.

  i'm figuring if i get it working for MATE that it should
also work for Gnome.  i haven't touched KDE in quite a 
few years.

  Windows i'll figure out after the more Linux/Posix 
systems.

  so my priorties are about like:

  1. posix/other linux sytems (perhaps Macs fit into this anways)
  2. Macs
  3. Windows

  luckily i do have other examples of python 3 programs which
seem to be multiple platform oriented that i can look at and
see what they've done.  so i won't be forever lost in the
wilderness...

  thanks for help,  :)


  ant
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ask for help about class variable scope (Re: Why doesn't a dictionary work in classes?)

[eryk sun]

On 12/27/18, jf...@ms4.hinet.net  wrote:
>
> I still don't get it. When I change it to using list comprehension, the
> problem is still there. (it now has no late-binding variable, right? :-)
>
 class Too:
> ... XS = [15, 15, 15, 15]
> ... Z4 = [val for val in XS]
> ... Z5 = [XS[0] for val in XS]
> ...
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "", line 4, in Too
>   File "", line 4, in 
> NameError: name 'XS' is not defined
>
> The problem confuse me is that is XS a local variable of the list
> comprehension?

XS is not a local variable in the scope of either comprehension. XS is
local to the class statement's scope. For each comprehension, an
iterator for the current object referenced by XS gets instantiated
(early binding) and passed as an argument to the comprehension scope.
If we disassemble the comprehension code, we find that this iterator
argument has the creatively illegal name ".0". (The bytecode
references it by its fast-locals-array index, not its weird name.)

In the Z5 case, XS is a non-local variable (late binding) in the
loop-body expression (left-hand side) of the comprehension. That's
common for Python code. In every iteration of the loop, the
interpreter looks up the object referenced by the name "XS", which can
change at any time (e.g. by another thread).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ask for help about class variable scope (Re: Why doesn't a dictionary work in classes?)

[Terry Reedy]

On 12/26/2018 9:53 PM, jf...@ms4.hinet.net wrote:

I saw the code below at stackoverflow. I have a little idea about the scope of 
a class, and list comprehension and generator expressions, but still can't 
figure out why Z4 works and Z5 not. Can someone explain it? (in a 
not-too-complicated way:-)

class Foo():
 XS = [15, 15, 15, 15]
 Z4 = sum(val for val in XS)


The iterable passed in to the comprehension is XS.


 try:
 Z5 = sum(XS[i] for i in range(len(XS)))


The iterable passed in to the comprehension function is range(len(XS)). 
XS is a class name, not a global or outer function local name, hence 
invisible to the inner function.



 except NameError:
 Z5 = None

print(Foo.Z4, Foo.Z5)

60 None



--Jach




--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Ask for help about class variable scope (Re: Why doesn't a dictionary work in classes?)

[jfong]

eryk sun於 2018年12月27日星期四 UTC+8下午2時31分58秒寫道：
> On 12/26/18, jf...@ms4.hinet.net  wrote:
> > I saw the code below at stackoverflow. I have a little idea about the scope
> > of a class, and list comprehension and generator expressions, but still
> > can't figure out why Z4 works and Z5 not. Can someone explain it? (in a
> > not-too-complicated way:-)
> >
> > class Foo():
> > XS = [15, 15, 15, 15]
> > Z4 = sum(val for val in XS)
> > try:
> > Z5 = sum(XS[i] for i in range(len(XS)))
> > except NameError:
> > Z5 = None
> >
> > print(Foo.Z4, Foo.Z5)
>  60 None
> 
> Maybe rewriting it with approximately equivalent inline code and
> generator functions will clarify the difference:
> 
> class Foo:
> def genexpr1(iterable):
> for val in iterable:
> yield val
> 
> def genexpr2(iterable):
> for i in iterable:
> yield XS[i]
> 
> XS = [15, 15, 15, 15]
> Z4 = sum(genexpr1(XS))
> try:
> Z5 = sum(genexpr2(range(len(XS
> except NameError:
> Z5 = None
> 
> del genexpr1, genexpr2
> 
> >>> print(Foo.Z4, Foo.Z5)
> 60 None
> 
> In both cases, an iterable is passed to the generator function. This
> argument is evaluated in the calling scope (e.g. range(len(XS))). A
> generator expression has a similar implementation, except it also
> evaluates the iterator for the iterable to ensure an exception is
> raised immediately in the defining scope if it's not iterable. For
> example:
> 
> >>> (x for x in 1)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: 'int' object is not iterable
> 
> genexpr1 is working with local variables only, but genexpr2 has a
> non-local reference to variable XS, which we call late binding. In
> this case, when the generator code executes the first pass of the loop
> (whenever that is), it looks for XS in the global (module) scope and
> builtins scope. It's not there, so a NameError is raised.
> 
> With late-binding, the variable can get deleted or modified in the
> source scope while the generator gets evaluated. For example:
> 
> >>> x = 'spam'
> >>> g = (x[i] for i in range(len(x)))
> >>> next(g)
> 's'
> >>> del x
> >>> next(g)
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "", line 1, in 
> NameError: name 'x' is not defined
> 
> >>> x = 'spam'
> >>> g = (x[i] for i in range(len(x)))
> >>> next(g)
> 's'
> >>> x = 'eggs'
> >>> list(g)
> ['g', 'g', 's']

I still don't get it. When I change it to using list comprehension, the problem 
is still there. (it now has no late-binding variable, right? :-)

>>> class Too:
... XS = [15, 15, 15, 15]
... Z4 = [val for val in XS]
... Z5 = [XS[0] for val in XS]
...
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 4, in Too
  File "", line 4, in 
NameError: name 'XS' is not defined
>>>

The problem confuse me is that is XS a local variable of the list comprehension?
If it's then Z5 should work, if it's not then Z4 shouldn't work when it's 
written in generator expression.

Or, things is much more complex than I thought:-(


-- 
https://mail.python.org/mailman/listinfo/python-list