Re: Question regarding unexpected behavior in using __enter__ method

2023-04-26 Thread Mark Bourne

Lorenzo Catoni wrote:

Dear Python Mailing List members,

I am writing to seek your assistance in understanding an unexpected
behavior that I encountered while using the __enter__ method. I have
provided a code snippet below to illustrate the problem:

```

class X:

... __enter__ = int
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...

x

0
```
As you can see, the __enter__ method does not throw any exceptions and
returns the output of "int()" correctly. However, one would normally expect
the input parameter "self" to be passed to the function.

On the other hand, when I implemented a custom function in place of the
__enter__ method, I encountered the following TypeError:

```

def myint(*a, **kw):

... return int(*a, **kw)
...

class X:

... __enter__ = myint
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...
Traceback (most recent call last):
   File "", line 1, in 
   File "", line 2, in myint
TypeError: int() argument must be a string, a bytes-like object or a real
number, not 'X'
```
Here, the TypeError occurred because "self" was passed as an input
parameter to "myint". Can someone explain why this unexpected behavior
occurs only in the latter case?

I tested this issue on the following Python versions, and the problem
persists on all of them:
- Python 3.8.10 (default, Nov 14 2022, 12:59:47) [GCC 9.4.0] on linux
- Python 3.10.10 (main, Feb  8 2023, 14:50:01) [GCC 9.4.0] on linux
- Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933
64 bit (AMD64)] on win32

I appreciate any input or insights that you might have on this matter.

Thank you for your help in advance!


Aside from other explanations and suggestions, the following definition 
of X also works:


class X:
__enter__ = staticmethod(myint)
__exit__ = lambda *_: None

Wrapping `myint` in a call to `staticmethod` is the same as using 
`@staticmethod` as a decorator on a method within the class, so the 
`self` parameter doesn't get passed.  Equivalent to:


class X:
@staticmethod
def __enter__(*a, **kw):
return int(*a, **kw)
__exit__ = lambda *_: None

Which in turn is just a neater way of doing:

class X:
def __enter__(*a, **kw):
return int(*a, **kw)
__enter__ = staticmethod(__enter__)
__exit__ = lambda *_: None

Those equivalents are a bit pointless, since no arguments will be passed 
into `__enter__` anyway in normal usage, but perhaps make it a bit 
clearer that the second example behaves as would be expected for a 
method of X (where the instance is passed as the first argument), and 
that it's the first example (with `__enter__ = int`) that should be a 
bit more surprising.  (I'm not sure there's much practical use for the 
original `__enter__ = int` either, but presumably that's just used as a 
cut-down demonstration).


--
Mark.
--
https://mail.python.org/mailman/listinfo/python-list


RE: Question regarding unexpected behavior in using __enter__ method

2023-04-25 Thread avi.e.gross


I think you got that right, Rob. A method created in a class is normally 
expected to care about the class in the sense that it often wants to access 
internal aspects and is given a "this" or "self" or whatever name you choose as 
a first argument. As noted, it is sometimes possible to create a function 
attached not to an object but to the class itself  as in, I think, the math 
class that is not normally instantiated as an object but lets you use things 
like math.pi and math.cos() and so on.

A comment on dunder methods in python is that they have a sort of purpose 
albeit you can hijack some to do other things. The protocol for WITH is a bit 
slippery as __enter__() and __exit__ are expected to do some abstract things 
that loosely are intended to set up something at the start in a way that will 
be (guaranteed) to be done if the exit routine is called when done. This can be 
about opening a file, or network connection and later closing it, or setting up 
some data structure and freeing the memory at the end, but it could be ANYTHING 
you feel like. For example, it can turn logging of some kind on and off and 
also compress the log file at the end. Or it could set up changes to the object 
that are there for the duration of the WITH and then reset the changes back at 
the end. 

An imaginary example might be to start caching what some methods are doing or 
replace a method by another, then empty the cache at the end or put back the 
redirected one.

And if what you want done at the beginning or end is outside the object being 
worked on, fine. Consider wrapping your function call in a simple function that 
calls the one you want after ignoring or removing the first argument. There are 
decorators that can do things like that.

So if you want int() or some existing plain non-member function, define an f() 
whose body calls int() with all arguments passed along other than the first. 

I just wrote and tested a trivial example where for some reason you just want 
to call sum() either with an iterable argument or with a second unnamed or 
named argument that specified a start you can add to. If this is written as a 
class method, it would have a first argument of "self" to ignore so I simulate 
that here:

def plusone(first, *rest, **named):
  return(sum(*rest, **named))

If you call this as below with valid arguments, it sort of swallows the first 
argument and passes the rest along:

>>> plusone("ignore", [])
0
>>> plusone("ignore", [1,2,3])
6
>>> plusone("ignore", [1,2,3], 100)
106
>>> plusone("ignore", range(7), start=100)
121

Yes, anything like this adds overhead. It does add flexibility and allows you 
to hijack the WITH protocol to do other things perhaps never anticipated but 
that may make sense, such as changing a companion object rather than the 
current one. But you need to live within some rules to do things and that means 
knowing there will be a first argument.

Avi
-Original Message-
From: Python-list  On 
Behalf Of Rob Cliffe via Python-list
Sent: Saturday, April 22, 2023 9:56 AM
To: Lorenzo Catoni ; python-list@python.org
Subject: Re: Question regarding unexpected behavior in using __enter__ method

This puzzled me at first, but I think others have nailed it.  It is not 
to do with the 'with' statement, but with the way functions are defined.
When a class is instantiated, as in x=X():
 the instance object gets (at least in effect), as attributes, 
copies of functions defined *in the class* (using def or lambda) but 
they become "bound methods", i.e. bound to the instance.  Whenever they 
are called, they will be called with the instance as the first argument, 
aka self:
 class X(object):
 def func(*args, **kargs): pass
 x = X()
 y = ()
x.func and y.func are two *different" functions.  When x.func is called, 
x is added as the first argument.  When y.func is called. y is added as 
the first argument.
  boundFunc = y.func
 boundFunc() # Adds y as first argument.
Indeed, these functions have an attribute called __self__ whose value is 
... you guessed it ... the object they are bound to
When a function is defined outside of a class, it remains a simple 
function, not bound to any object.  It does not have a __self__ 
attribute.  Neither does a built-in type such as 'int'.
Nor for that matter does the class function X.func:
 X.func() # Called with no arguments

Best wishes
Rob Cliffe

On 20/04/2023 23:44, Lorenzo Catoni wrote:
> Dear Python Mailing List members,
>
> I am writing to seek your assistance in understanding an unexpected
> behavior that I encountered while using the __enter__ method. I have
> provided a code snippet below to illustrate the problem:
>
> ```
>>>> class X:
> ... __enter__ = int
> ... __exit__ = lambda *_: None
> ...
>>>> with X() as 

Re: Question regarding unexpected behavior in using __enter__ method

2023-04-25 Thread Rob Cliffe via Python-list
This puzzled me at first, but I think others have nailed it.  It is not 
to do with the 'with' statement, but with the way functions are defined.

When a class is instantiated, as in x=X():
    the instance object gets (at least in effect), as attributes, 
copies of functions defined *in the class* (using def or lambda) but 
they become "bound methods", i.e. bound to the instance.  Whenever they 
are called, they will be called with the instance as the first argument, 
aka self:

    class X(object):
        def func(*args, **kargs): pass
    x = X()
    y = ()
x.func and y.func are two *different" functions.  When x.func is called, 
x is added as the first argument.  When y.func is called. y is added as 
the first argument.

 boundFunc = y.func
    boundFunc() # Adds y as first argument.
Indeed, these functions have an attribute called __self__ whose value is 
... you guessed it ... the object they are bound to
When a function is defined outside of a class, it remains a simple 
function, not bound to any object.  It does not have a __self__ 
attribute.  Neither does a built-in type such as 'int'.

Nor for that matter does the class function X.func:
    X.func() # Called with no arguments

Best wishes
Rob Cliffe

On 20/04/2023 23:44, Lorenzo Catoni wrote:

Dear Python Mailing List members,

I am writing to seek your assistance in understanding an unexpected
behavior that I encountered while using the __enter__ method. I have
provided a code snippet below to illustrate the problem:

```

class X:

... __enter__ = int
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...

x

0
```
As you can see, the __enter__ method does not throw any exceptions and
returns the output of "int()" correctly. However, one would normally expect
the input parameter "self" to be passed to the function.

On the other hand, when I implemented a custom function in place of the
__enter__ method, I encountered the following TypeError:

```

def myint(*a, **kw):

... return int(*a, **kw)
...

class X:

... __enter__ = myint
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...
Traceback (most recent call last):
   File "", line 1, in 
   File "", line 2, in myint
TypeError: int() argument must be a string, a bytes-like object or a real
number, not 'X'
```
Here, the TypeError occurred because "self" was passed as an input
parameter to "myint". Can someone explain why this unexpected behavior
occurs only in the latter case?

I tested this issue on the following Python versions, and the problem
persists on all of them:
- Python 3.8.10 (default, Nov 14 2022, 12:59:47) [GCC 9.4.0] on linux
- Python 3.10.10 (main, Feb  8 2023, 14:50:01) [GCC 9.4.0] on linux
- Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933
64 bit (AMD64)] on win32

I appreciate any input or insights that you might have on this matter.

Thank you for your help in advance!

Best regards,
Lorenzo Catoni


--
https://mail.python.org/mailman/listinfo/python-list


Re: Question regarding unexpected behavior in using __enter__ method

2023-04-21 Thread aapost

On 4/20/23 18:44, Lorenzo Catoni wrote:

Here, the TypeError occurred because "self" was passed as an input


Instantiate X and observe it there

x2 = X()

>>> X.__enter__

>>> X.__exit__
 at 0x...>
>>> x2.__enter__

>>> x2.__exit__
 of <__main__.X object at 0x...>>


To receive self the method must be bound. __enter__ = int doesn't bind 
the int type/class on instantiation, so it never gets self. Your custom 
function binds and receives self.


I am not sure if there is documentation to explain better specifically 
what makes makes type different and not bindable.


--
https://mail.python.org/mailman/listinfo/python-list


Re: Question regarding unexpected behavior in using __enter__ method

2023-04-21 Thread Lorenzo Catoni
Thankyou for your answer,
i think i found the reason for this behavior, is has to do with the
function being user defined or not, rather than being a plain function or
type, as stated here
https://docs.python.org/3/reference/datamodel.html#:~:text=Also%20notice%20that%20this%20transformation%20only%20happens%20for%20user%2Ddefined%20functions%3B%20other%20callable%20objects%20(and%20all%20non%2Dcallable%20objects)%20are%20retrieved%20without%20transformation

Regards,
Lorenzo Catoni

On Fri, 21 Apr 2023 at 07:21, Cameron Simpson  wrote:

> On 21Apr2023 00:44, Lorenzo Catoni  wrote:
> >I am writing to seek your assistance in understanding an unexpected
> >behavior that I encountered while using the __enter__ method. I have
> >provided a code snippet below to illustrate the problem:
> >
> >```
>  class X:
> >... __enter__ = int
> >... __exit__ = lambda *_: None
> >...
>  with X() as x:
> >... pass
> >...
>  x
> >0
> >```
> >As you can see, the __enter__ method does not throw any exceptions and
> >returns the output of "int()" correctly. However, one would normally
> expect
> >the input parameter "self" to be passed to the function.
>
> My descriptor fu is weak, but I believe this is because `int` is not a
> plain function but a type.
>
> Consider this class definition:
>
>  class X:
> x = 1
> def y(self):
> return "y"
>
> When you define a class, the body of the class is run in a namespace,
> and on completion, the namespace is _used_ to construct the class.
> During that process, the various names are considered. Here we've got 2
> names: "x" and "y".
>
> "x" refers to an int and is just stored as a class attribute, unchanged.
>
> "y" refers to a function, and is promoted to a descriptor of an unbound
> method.
>
> So later: X.x return 1 but X.y returns a unbound method. If we make an
> instance:
>
>  objx = X()
>
> then obj.x returns 1 (by not fining an "x" on "obj", but finding one on
> "type(obj)" i.e. the class attribute.
>
> By contrast, obj.y returns a bound method, a function already curried
> with a leading parameter "obj" (which will be "self"). There's no "y"
> attribute directly on "obj" but there's an unbound method on
> "type(obj).y", which gets bound by saying "obj.y".
>
> The means that what happens to a name when you define the class depends
> on the typeof the value bound to the name.
>
> A plain function gets turned into an unbound instance method, but other
> things are left alone.
>
> When you went:
>
>  __enter__ = int
>
> That's not a plain function and so "obj.__enter__" doesn't turn into a
> bound method - it it just `int`.
>
> Cheers,
> Cameron Simpson 
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Question regarding unexpected behavior in using __enter__ method

2023-04-21 Thread Peter Otten

On 21/04/2023 00:44, Lorenzo Catoni wrote:

Dear Python Mailing List members,

I am writing to seek your assistance in understanding an unexpected
behavior that I encountered while using the __enter__ method. I have
provided a code snippet below to illustrate the problem:

```

class X:

... __enter__ = int
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...

x

0
```
As you can see, the __enter__ method does not throw any exceptions and
returns the output of "int()" correctly. However, one would normally expect
the input parameter "self" to be passed to the function.

On the other hand, when I implemented a custom function in place of the
__enter__ method, I encountered the following TypeError:

```

def myint(*a, **kw):

... return int(*a, **kw)
...

class X:

... __enter__ = myint
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...
Traceback (most recent call last):
   File "", line 1, in 
   File "", line 2, in myint
TypeError: int() argument must be a string, a bytes-like object or a real
number, not 'X'
```
Here, the TypeError occurred because "self" was passed as an input
parameter to "myint". Can someone explain why this unexpected behavior
occurs only in the latter case?


Cameron is right, it's the descriptor protocol. Technically

inst.attr

invokes attr.__get__(...) if it exists:

>>> class A:
def __get__(self, *args): return args


>>> class B: pass

>>> class X:
a = A()
b = B()


>>> x = X()
>>> x.b
<__main__.B object at 0x02C2E388>
>>> x.a
(<__main__.X object at 0x02C2E280>, )

Python functions support the descriptor protocol

>>> hasattr(lambda: None, "__get__")
True

while builtin functions don't:

>>> hasattr(ord, "__get__")
False


I'm unsure whether to regard int as a class or or function, but as there
is no __get__

>>> hasattr(int, "__get__")
False

it behaves like builtin functions in this case.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Question regarding unexpected behavior in using __enter__ method

2023-04-20 Thread Cameron Simpson

On 21Apr2023 00:44, Lorenzo Catoni  wrote:

I am writing to seek your assistance in understanding an unexpected
behavior that I encountered while using the __enter__ method. I have
provided a code snippet below to illustrate the problem:

```

class X:

... __enter__ = int
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...

x

0
```
As you can see, the __enter__ method does not throw any exceptions and
returns the output of "int()" correctly. However, one would normally expect
the input parameter "self" to be passed to the function.


My descriptor fu is weak, but I believe this is because `int` is not a 
plain function but a type.


Consider this class definition:

class X:
   x = 1
   def y(self):
   return "y"

When you define a class, the body of the class is run in a namespace, 
and on completion, the namespace is _used_ to construct the class.  
During that process, the various names are considered. Here we've got 2 
names: "x" and "y".


"x" refers to an int and is just stored as a class attribute, unchanged.

"y" refers to a function, and is promoted to a descriptor of an unbound 
method.


So later: X.x return 1 but X.y returns a unbound method. If we make an 
instance:


objx = X()

then obj.x returns 1 (by not fining an "x" on "obj", but finding one on 
"type(obj)" i.e. the class attribute.


By contrast, obj.y returns a bound method, a function already curried 
with a leading parameter "obj" (which will be "self"). There's no "y" 
attribute directly on "obj" but there's an unbound method on 
"type(obj).y", which gets bound by saying "obj.y".


The means that what happens to a name when you define the class depends 
on the typeof the value bound to the name.


A plain function gets turned into an unbound instance method, but other 
things are left alone.


When you went:

__enter__ = int

That's not a plain function and so "obj.__enter__" doesn't turn into a 
bound method - it it just `int`.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list


Re: Question regarding unexpected behavior in using __enter__ method

2023-04-20 Thread dn via Python-list

On 21/04/2023 10.44, Lorenzo Catoni wrote:

I am writing to seek your assistance in understanding an unexpected
behavior that I encountered while using the __enter__ method. I have
provided a code snippet below to illustrate the problem:


It is expected behavior - just not what WE might have expected!



class X:

... __enter__ = int
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...

x

0


Note that what is happening is the creation of an alias for the int 
built-in function.


The docs say:
«
class int(x=0)
class int(x, base=10)

Return an integer object constructed from a number or string x, or 
return 0 if no arguments are given. If x defines __int__(), int(x) 
returns x.__int__(). If x defines __index__(), it returns x.__index__(). 
If x defines __trunc__(), it returns x.__trunc__(). For floating point 
numbers, this truncates towards zero.

...
»

(https://docs.python.org/3/library/functions.html#int)

No argument is given. int() delivers as-promised. Hence, the x == 0 
result obtained.





As you can see, the __enter__ method does not throw any exceptions and
returns the output of "int()" correctly. However, one would normally expect
the input parameter "self" to be passed to the function.

On the other hand, when I implemented a custom function in place of the
__enter__ method, I encountered the following TypeError:

```

def myint(*a, **kw):

... return int(*a, **kw)
...

class X:

... __enter__ = myint
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...
Traceback (most recent call last):
   File "", line 1, in 
   File "", line 2, in myint
TypeError: int() argument must be a string, a bytes-like object or a real
number, not 'X'
```
Here, the TypeError occurred because "self" was passed as an input
parameter to "myint". Can someone explain why this unexpected behavior
occurs only in the latter case?

I tested this issue on the following Python versions, and the problem
persists on all of them:
- Python 3.8.10 (default, Nov 14 2022, 12:59:47) [GCC 9.4.0] on linux
- Python 3.10.10 (main, Feb  8 2023, 14:50:01) [GCC 9.4.0] on linux
- Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933
64 bit (AMD64)] on win32

I appreciate any input or insights that you might have on this matter.



(you know this next part!)

However, if int() is fed an X-object, which in no way represents a 
numeric or numeric-able value, then it crashes. After the class 
definition, try:


int( X )

=> int 0


Right-y-ho: the evidence.

Firstly, what happens when int() is called with no argument?

print( "int", int(), )


Note that whereas int and myint are both called with no argument(s), and 
therefore int() defaults to 0, myint is attempting to use the arguments 
as part of its return-value - "pass-through".


As identified, the first argument (the a-tuple's zero-th element) is 
going to be x's self - which is NOT numeric, stored in a tuple - which 
is NOT numeric...



Thus, if we "shadow" the built-in int() with a user-function, we can see 
what is being passed-in


def int( *a, **kw, ):
print( locals(), )
print( "a", a, type( a ), id( a ), )
print( len( a ), a[ 0 ], type( a[ 0 ], ) )
print( "kw", kw, type( kw ), id( kw ), )
return 42

class X:
__enter__ = int
__exit__ = lambda *_: None

with X() as x:
pass

print( "After first CM", x, "\n\n")

del( int )


def myint(*a, **kw):
print( locals(), )
print( "a", a, type( a ), id( a ), )
print( len( a ), a[ 0 ], type( a[ 0 ], ) )
print( "kw", kw, type( kw ), id( kw ), )
return int(*a, **kw)

class Y:
__enter__ = myint
__exit__ = lambda *_: None


print( Y, type( Y ), id( Y ), )

with Y() as y:
print( y, type( y ), id( y ), )
pass

print( y )


=>
{'a': (<__main__.X object at 0x7f9b6bf13b90>,), 'kw': {}}
a (<__main__.X object at 0x7f9b6bf13b90>,)  140305733882528
1 <__main__.X object at 0x7f9b6bf13b90> 
kw {}  140305734120576
After first CM 42


  93904023389520
{'a': (<__main__.Y object at 0x7f9b6bf2c0d0>,), 'kw': {}}
a (<__main__.Y object at 0x7f9b6bf2c0d0>,)  140305507712640
1 <__main__.Y object at 0x7f9b6bf2c0d0> 
kw {}  140305507621376
Traceback (most recent call last):
  File "/home/dn/Projects/analyzer/lorenzo.py", line 53, in 
with Y() as y:
  File "/home/dn/Projects/analyzer/lorenzo.py", line 44, in myint
return int(*a, **kw)
   ^
TypeError: int() argument must be a string, a bytes-like object or a 
real number, not 'Y'



If you remover the del() and leave my play-version of int(), Python can 
make it work...


(the second half of the output)

  94557671306576
{'a': (<__main__.Y object at 0x7f950ee2c0d0>,), 'kw': {}}
a (<__main__.Y object at 0x7f950ee2c0d0>,)  140278176579200
1 <__main__.Y object at 0x7f950ee2c0d0> 
kw {}  140278176487936
{'a': (<__main__.Y object at 0x7f950ee2c0d0>,), 'kw': {}}
a (<__main__.Y object at 0x7f950ee2c0d0>,)  140278176579152
1 <__main__.Y object at 0x7f950ee2c0d0> 

Question regarding unexpected behavior in using __enter__ method

2023-04-20 Thread Lorenzo Catoni
Dear Python Mailing List members,

I am writing to seek your assistance in understanding an unexpected
behavior that I encountered while using the __enter__ method. I have
provided a code snippet below to illustrate the problem:

```
>>> class X:
... __enter__ = int
... __exit__ = lambda *_: None
...
>>> with X() as x:
... pass
...
>>> x
0
```
As you can see, the __enter__ method does not throw any exceptions and
returns the output of "int()" correctly. However, one would normally expect
the input parameter "self" to be passed to the function.

On the other hand, when I implemented a custom function in place of the
__enter__ method, I encountered the following TypeError:

```
>>> def myint(*a, **kw):
... return int(*a, **kw)
...
>>> class X:
... __enter__ = myint
... __exit__ = lambda *_: None
...
>>> with X() as x:
... pass
...
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2, in myint
TypeError: int() argument must be a string, a bytes-like object or a real
number, not 'X'
```
Here, the TypeError occurred because "self" was passed as an input
parameter to "myint". Can someone explain why this unexpected behavior
occurs only in the latter case?

I tested this issue on the following Python versions, and the problem
persists on all of them:
- Python 3.8.10 (default, Nov 14 2022, 12:59:47) [GCC 9.4.0] on linux
- Python 3.10.10 (main, Feb  8 2023, 14:50:01) [GCC 9.4.0] on linux
- Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933
64 bit (AMD64)] on win32

I appreciate any input or insights that you might have on this matter.

Thank you for your help in advance!

Best regards,
Lorenzo Catoni
-- 
https://mail.python.org/mailman/listinfo/python-list