Re: __new__ and __init__ - why does this work?

[Ethan Furman]

On 08/09/2017 03:39 PM, Ian Kelly wrote:

On Wed, Aug 9, 2017 at 2:20 PM, Ethan Furman wrote:

On 08/09/2017 12:59 PM, Ian Pilcher wrote:



I do want to prevent frozenset.__init__ from being called *again* when
an existing instance is returned, so I've decided to take this
approach:

  def __new__(cls, *args, **kwargs):
  self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
  self._initialized = False
  return UniqueSet._registry.setdefault(self, self)

  def __init__(self, *args, **kwargs):
  if not self._initialized:
  super(UniqueSet, self).__init__(self, *args, **kwargs)
  self._initialized = True



Whom do you think is going to call __init__ a second time?


The __call__ method of the metaclass. Here's an example using a singleton class:

--> Singleton()
__init__ called 1 times
<__main__.Singleton object at 0x76b54a717518>
--> Singleton()
__init__ called 2 times
<__main__.Singleton object at 0x76b54a717518>
--> Singleton() is Singleton()
__init__ called 3 times
__init__ called 4 times
True


Ah, cool!  I hadn't known about that particular side effect of singletons.

--
~Ethan~

--
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Ian Kelly]

On Aug 9, 2017 4:39 PM, "Ian Kelly"  wrote:



By the way, "whom" is not the verb object in "Whom do you think is
going to call", so it should properly be "who do you think is going to
call". I point this out because although I don't really care when
people use "who" incorrectly, it looks pretty silly (and pretentious)
to use "whom" incorrectly.


Er, sorry, I'm wrong of course. I don't know why my brain parsed your
statement the way that it did.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Ian Kelly]

On Wed, Aug 9, 2017 at 2:20 PM, Ethan Furman  wrote:
> On 08/09/2017 12:59 PM, Ian Pilcher wrote:
>
>> I do want to prevent frozenset.__init__ from being called *again* when
>> an existing instance is returned, so I've decided to take this
>> approach:
>>
>>  def __new__(cls, *args, **kwargs):
>>  self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
>>  self._initialized = False
>>  return UniqueSet._registry.setdefault(self, self)
>>
>>  def __init__(self, *args, **kwargs):
>>  if not self._initialized:
>>  super(UniqueSet, self).__init__(self, *args, **kwargs)
>>  self._initialized = True
>
>
> Whom do you think is going to call __init__ a second time?

The __call__ method of the metaclass. Here's an example using a singleton class:

class Singleton:
  _instance = None

  def __new__(cls):
if cls._instance is None:
  cls._instance = super(Singleton, cls).__new__(cls)
return cls._instance

  def __init__(self):
self.count = getattr(self, 'count', 0) + 1
print('__init__ called %d times' % self.count)

>>> Singleton()
__init__ called 1 times
<__main__.Singleton object at 0x76b54a717518>
>>> Singleton()
__init__ called 2 times
<__main__.Singleton object at 0x76b54a717518>
>>> Singleton() is Singleton()
__init__ called 3 times
__init__ called 4 times
True

By the way, "whom" is not the verb object in "Whom do you think is
going to call", so it should properly be "who do you think is going to
call". I point this out because although I don't really care when
people use "who" incorrectly, it looks pretty silly (and pretentious)
to use "whom" incorrectly.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Ethan Furman]

On 08/09/2017 12:59 PM, Ian Pilcher wrote:


I do want to prevent frozenset.__init__ from being called *again* when
an existing instance is returned, so I've decided to take this
approach:

 def __new__(cls, *args, **kwargs):
 self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
 self._initialized = False
 return UniqueSet._registry.setdefault(self, self)

 def __init__(self, *args, **kwargs):
 if not self._initialized:
 super(UniqueSet, self).__init__(self, *args, **kwargs)
 self._initialized = True


Whom do you think is going to call __init__ a second time?

--
~Ethan~

--
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Ian Pilcher]

On 08/09/2017 07:54 AM, Steve D'Aprano wrote:

On Wed, 9 Aug 2017 10:08 am, Ian Pilcher wrote:


I have created a class to provide a "hash consing"[1] set.


Your footnote for [1] appears to be missing. What's a hash consing set? It
appears to be nothing more than frozen sets which you put in a cache so as to
confuse identity and value *wink*


Uugh.  Here's the link:

  https://en.wikipedia.org/wiki/Hash_consing


I doubt very much you're actually saving any time, since you create a temporary
frozen set before returning the one in the cache. You might save some memory
though.


Indeed.  This is all about using memory efficiently.


Your __init__ method does nothing. Get rid of it and save two lines of code :-)


Well, it prevents frozenset.__init__ from being called.


Also, there is at least theoretically the vague possibility that
frozenset.__init__ does something (phones home to Guido?) so you shouldn't
block it if you don't need to.


I do want to prevent frozenset.__init__ from being called *again* when
an existing instance is returned, so I've decided to take this
approach:

def __new__(cls, *args, **kwargs):
self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
self._initialized = False
return UniqueSet._registry.setdefault(self, self)

def __init__(self, *args, **kwargs):
if not self._initialized:
super(UniqueSet, self).__init__(self, *args, **kwargs)
self._initialized = True


--

Ian Pilcher arequip...@gmail.com
 "I grew up before Mark Zuckerberg invented friendship" 


--
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Ian Pilcher]

On 08/08/2017 10:19 PM, Ian Kelly wrote:

It's initialized by the superclass call to __new__. frozenset is
immutable, and __init__ methods of immutable types generally don't do
anything (if they could, then they wouldn't be immutable), which is
why it doesn't really matter that you didn't call it. At the same
time, it generally doesn't hurt to call it, and you probably shouldn't
even have an override of __init__ here if it doesn't do anything.


Thanks for the explanation.

I'll admit that I'm a bit paranoid about the potential effects of
"re-__init__-ing" an object, at least in the general case.  What do
you think of this?

def __new__(cls, *args, **kwargs):
self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
self._initialized = False
return UniqueSet._registry.setdefault(self, self)

def __init__(self, *args, **kwargs):
if not self._initialized:
super(UniqueSet, self).__init__(self, *args, **kwargs)
self._initialized = True


It seems a bit inefficient that you create *two* sets in __new__ and
then map one of them to the other in your registry. Why not just
create the UniqueSet and then map it to itself if it's not already
registered? Something like this (untested):

def __new__(cls, *args, **kwargs):
 self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
 return UniqueSet._registry.setdefault(self, self)


That was mainly me being unfamiliar with the frozenset API (and not
overly concerned about the size of the _registry, since I expect that
there will be a very small number of entries).  Your version is much
more elegant.

Thank you!

--

Ian Pilcher arequip...@gmail.com
 "I grew up before Mark Zuckerberg invented friendship" 


--
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Steve D'Aprano]

On Thu, 10 Aug 2017 12:39 am, Dennis Lee Bieber wrote:

> On Wed, 09 Aug 2017 22:54:00 +1000, Steve D'Aprano
>  declaimed the following:
> 
>>Its not just frozenset. Any mutable class must initialise its instances in
>>__new__. Immutable classes can use either __new__ or __init__, but for
>>historical reasons typically use __init__.
>>
> Think you swapped mutable and immutable there...


Doh! So I did.

This is what happens when you try typing a response while caring on a
conversation with the missus...



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Steve D'Aprano]

On Wed, 9 Aug 2017 10:08 am, Ian Pilcher wrote:

> I have created a class to provide a "hash consing"[1] set.

Your footnote for [1] appears to be missing. What's a hash consing set? It
appears to be nothing more than frozen sets which you put in a cache so as to
confuse identity and value *wink*

I doubt very much you're actually saving any time, since you create a temporary
frozen set before returning the one in the cache. You might save some memory
though.


>class UniqueSet(frozenset):
>_registry = dict()
>def __new__(cls, *args, **kwargs):
>set = frozenset(*args, **kwargs)
>try:
>return UniqueSet._registry[set]
>except KeyError:
>self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
>UniqueSet._registry[set] = self
>return self
> 
>def __init__(self, *args, **kwargs):
>pass
> 
> I can't figure out how it works, though.  In particular, I can't figure
> out how the call to __new__ actually initializes the set (since my
> __init__ never calls the superclass __init__).

Since frozensets are immutable, they have to be initialised in the __new__
constructor, because by the time __init__ is called the instance is immutable
and cannot be updated.

Your call to super() above creates a new instance. Its effectively a frozenset,
except that the class of it is set to your subclass ("cls", in this case). So
all the initalisation of the frozenset happens inside frozenset.__new__, which
you call via super().

Your __init__ method does nothing. Get rid of it and save two lines of code :-)

Also, there is at least theoretically the vague possibility that
frozenset.__init__ does something (phones home to Guido?) so you shouldn't
block it if you don't need to.

> Is this a particular behavior of frozenset, or am I missing something
> about the way that __new__ and __init__ interact?

Its not just frozenset. Any mutable class must initialise its instances in
__new__. Immutable classes can use either __new__ or __init__, but for
historical reasons typically use __init__.

The way __new__ and __init__ are called is:

(1) __new__ is called;

(2) if it returns an instance of cls, then instance.__init__ is called


(It is not mandatory for __new__ to return a new instance of its class; it can
return whatever you like. That is a feature, and there are occasional uses for
it.)



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Peter Otten]

Ian Pilcher wrote:

> I have created a class to provide a "hash consing"[1] set.
> 
>class UniqueSet(frozenset):
> 
>_registry = dict()
> 
>def __new__(cls, *args, **kwargs):
>set = frozenset(*args, **kwargs)
>try:
>return UniqueSet._registry[set]
>except KeyError:
>self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
>UniqueSet._registry[set] = self
>return self
> 
>def __init__(self, *args, **kwargs):
>pass
> 
> I can't figure out how it works, though.  In particular, I can't figure
> out how the call to __new__ actually initializes the set (since my
> __init__ never calls the superclass __init__).
> 
> Is this a particular behavior of frozenset, or am I missing something
> about the way that __new__ and __init__ interact?

I think __init__() is called to initialise the object after it has been 
created with __new__(), roughly the following, run by the metaclass:

obj = UniqueSet_.__new__(UniqueSet, ...)
if isinstance(obj, UniqueSet):
obj.__init__(...)

As Ian says, for immutable classes __init__() usually does nothing, as by 
definition the instance cannot be changed once created:

>>> t = tuple.__new__(tuple, "ab")
>>> t
('a', 'b')
>>> t.__init__(None)
>>> t.__init__(x=42)
>>> t.__init__("whatever", "you", "like")
>>> t
('a', 'b')

I'm a bit surprised that the signature isn't restricted to a single 
positional argument...

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: __new__ and __init__ - why does this work?

[Ian Kelly]

On Tue, Aug 8, 2017 at 6:08 PM, Ian Pilcher  wrote:
> I have created a class to provide a "hash consing"[1] set.
>
>   class UniqueSet(frozenset):
>
>   _registry = dict()
>
>   def __new__(cls, *args, **kwargs):
>   set = frozenset(*args, **kwargs)
>   try:
>   return UniqueSet._registry[set]
>   except KeyError:
>   self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
>   UniqueSet._registry[set] = self
>   return self
>
>   def __init__(self, *args, **kwargs):
>   pass
>
> I can't figure out how it works, though.  In particular, I can't figure
> out how the call to __new__ actually initializes the set (since my
> __init__ never calls the superclass __init__).
>
> Is this a particular behavior of frozenset, or am I missing something
> about the way that __new__ and __init__ interact?

It's initialized by the superclass call to __new__. frozenset is
immutable, and __init__ methods of immutable types generally don't do
anything (if they could, then they wouldn't be immutable), which is
why it doesn't really matter that you didn't call it. At the same
time, it generally doesn't hurt to call it, and you probably shouldn't
even have an override of __init__ here if it doesn't do anything.

It seems a bit inefficient that you create *two* sets in __new__ and
then map one of them to the other in your registry. Why not just
create the UniqueSet and then map it to itself if it's not already
registered? Something like this (untested):

def __new__(cls, *args, **kwargs):
self = super(UniqueSet, cls).__new__(cls, *args, **kwargs)
return UniqueSet._registry.setdefault(self, self)
-- 
https://mail.python.org/mailman/listinfo/python-list