Re: __new__ and __init__ - why does this work?
On 08/09/2017 03:39 PM, Ian Kelly wrote: On Wed, Aug 9, 2017 at 2:20 PM, Ethan Furman wrote: On 08/09/2017 12:59 PM, Ian Pilcher wrote: I do want to prevent frozenset.__init__ from being called *again* when an existing instance is returned, so I've decided to take this approach: def __new__(cls, *args, **kwargs): self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) self._initialized = False return UniqueSet._registry.setdefault(self, self) def __init__(self, *args, **kwargs): if not self._initialized: super(UniqueSet, self).__init__(self, *args, **kwargs) self._initialized = True Whom do you think is going to call __init__ a second time? The __call__ method of the metaclass. Here's an example using a singleton class: --> Singleton() __init__ called 1 times <__main__.Singleton object at 0x76b54a717518> --> Singleton() __init__ called 2 times <__main__.Singleton object at 0x76b54a717518> --> Singleton() is Singleton() __init__ called 3 times __init__ called 4 times True Ah, cool! I hadn't known about that particular side effect of singletons. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
On Aug 9, 2017 4:39 PM, "Ian Kelly"wrote: By the way, "whom" is not the verb object in "Whom do you think is going to call", so it should properly be "who do you think is going to call". I point this out because although I don't really care when people use "who" incorrectly, it looks pretty silly (and pretentious) to use "whom" incorrectly. Er, sorry, I'm wrong of course. I don't know why my brain parsed your statement the way that it did. -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
On Wed, Aug 9, 2017 at 2:20 PM, Ethan Furmanwrote: > On 08/09/2017 12:59 PM, Ian Pilcher wrote: > >> I do want to prevent frozenset.__init__ from being called *again* when >> an existing instance is returned, so I've decided to take this >> approach: >> >> def __new__(cls, *args, **kwargs): >> self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) >> self._initialized = False >> return UniqueSet._registry.setdefault(self, self) >> >> def __init__(self, *args, **kwargs): >> if not self._initialized: >> super(UniqueSet, self).__init__(self, *args, **kwargs) >> self._initialized = True > > > Whom do you think is going to call __init__ a second time? The __call__ method of the metaclass. Here's an example using a singleton class: class Singleton: _instance = None def __new__(cls): if cls._instance is None: cls._instance = super(Singleton, cls).__new__(cls) return cls._instance def __init__(self): self.count = getattr(self, 'count', 0) + 1 print('__init__ called %d times' % self.count) >>> Singleton() __init__ called 1 times <__main__.Singleton object at 0x76b54a717518> >>> Singleton() __init__ called 2 times <__main__.Singleton object at 0x76b54a717518> >>> Singleton() is Singleton() __init__ called 3 times __init__ called 4 times True By the way, "whom" is not the verb object in "Whom do you think is going to call", so it should properly be "who do you think is going to call". I point this out because although I don't really care when people use "who" incorrectly, it looks pretty silly (and pretentious) to use "whom" incorrectly. -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
On 08/09/2017 12:59 PM, Ian Pilcher wrote: I do want to prevent frozenset.__init__ from being called *again* when an existing instance is returned, so I've decided to take this approach: def __new__(cls, *args, **kwargs): self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) self._initialized = False return UniqueSet._registry.setdefault(self, self) def __init__(self, *args, **kwargs): if not self._initialized: super(UniqueSet, self).__init__(self, *args, **kwargs) self._initialized = True Whom do you think is going to call __init__ a second time? -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
On 08/09/2017 07:54 AM, Steve D'Aprano wrote: On Wed, 9 Aug 2017 10:08 am, Ian Pilcher wrote: I have created a class to provide a "hash consing"[1] set. Your footnote for [1] appears to be missing. What's a hash consing set? It appears to be nothing more than frozen sets which you put in a cache so as to confuse identity and value *wink* Uugh. Here's the link: https://en.wikipedia.org/wiki/Hash_consing I doubt very much you're actually saving any time, since you create a temporary frozen set before returning the one in the cache. You might save some memory though. Indeed. This is all about using memory efficiently. Your __init__ method does nothing. Get rid of it and save two lines of code :-) Well, it prevents frozenset.__init__ from being called. Also, there is at least theoretically the vague possibility that frozenset.__init__ does something (phones home to Guido?) so you shouldn't block it if you don't need to. I do want to prevent frozenset.__init__ from being called *again* when an existing instance is returned, so I've decided to take this approach: def __new__(cls, *args, **kwargs): self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) self._initialized = False return UniqueSet._registry.setdefault(self, self) def __init__(self, *args, **kwargs): if not self._initialized: super(UniqueSet, self).__init__(self, *args, **kwargs) self._initialized = True -- Ian Pilcher arequip...@gmail.com "I grew up before Mark Zuckerberg invented friendship" -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
On 08/08/2017 10:19 PM, Ian Kelly wrote: It's initialized by the superclass call to __new__. frozenset is immutable, and __init__ methods of immutable types generally don't do anything (if they could, then they wouldn't be immutable), which is why it doesn't really matter that you didn't call it. At the same time, it generally doesn't hurt to call it, and you probably shouldn't even have an override of __init__ here if it doesn't do anything. Thanks for the explanation. I'll admit that I'm a bit paranoid about the potential effects of "re-__init__-ing" an object, at least in the general case. What do you think of this? def __new__(cls, *args, **kwargs): self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) self._initialized = False return UniqueSet._registry.setdefault(self, self) def __init__(self, *args, **kwargs): if not self._initialized: super(UniqueSet, self).__init__(self, *args, **kwargs) self._initialized = True It seems a bit inefficient that you create *two* sets in __new__ and then map one of them to the other in your registry. Why not just create the UniqueSet and then map it to itself if it's not already registered? Something like this (untested): def __new__(cls, *args, **kwargs): self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) return UniqueSet._registry.setdefault(self, self) That was mainly me being unfamiliar with the frozenset API (and not overly concerned about the size of the _registry, since I expect that there will be a very small number of entries). Your version is much more elegant. Thank you! -- Ian Pilcher arequip...@gmail.com "I grew up before Mark Zuckerberg invented friendship" -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
On Thu, 10 Aug 2017 12:39 am, Dennis Lee Bieber wrote: > On Wed, 09 Aug 2017 22:54:00 +1000, Steve D'Aprano >declaimed the following: > >>Its not just frozenset. Any mutable class must initialise its instances in >>__new__. Immutable classes can use either __new__ or __init__, but for >>historical reasons typically use __init__. >> > Think you swapped mutable and immutable there... Doh! So I did. This is what happens when you try typing a response while caring on a conversation with the missus... -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
On Wed, 9 Aug 2017 10:08 am, Ian Pilcher wrote: > I have created a class to provide a "hash consing"[1] set. Your footnote for [1] appears to be missing. What's a hash consing set? It appears to be nothing more than frozen sets which you put in a cache so as to confuse identity and value *wink* I doubt very much you're actually saving any time, since you create a temporary frozen set before returning the one in the cache. You might save some memory though. >class UniqueSet(frozenset): >_registry = dict() >def __new__(cls, *args, **kwargs): >set = frozenset(*args, **kwargs) >try: >return UniqueSet._registry[set] >except KeyError: >self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) >UniqueSet._registry[set] = self >return self > >def __init__(self, *args, **kwargs): >pass > > I can't figure out how it works, though. In particular, I can't figure > out how the call to __new__ actually initializes the set (since my > __init__ never calls the superclass __init__). Since frozensets are immutable, they have to be initialised in the __new__ constructor, because by the time __init__ is called the instance is immutable and cannot be updated. Your call to super() above creates a new instance. Its effectively a frozenset, except that the class of it is set to your subclass ("cls", in this case). So all the initalisation of the frozenset happens inside frozenset.__new__, which you call via super(). Your __init__ method does nothing. Get rid of it and save two lines of code :-) Also, there is at least theoretically the vague possibility that frozenset.__init__ does something (phones home to Guido?) so you shouldn't block it if you don't need to. > Is this a particular behavior of frozenset, or am I missing something > about the way that __new__ and __init__ interact? Its not just frozenset. Any mutable class must initialise its instances in __new__. Immutable classes can use either __new__ or __init__, but for historical reasons typically use __init__. The way __new__ and __init__ are called is: (1) __new__ is called; (2) if it returns an instance of cls, then instance.__init__ is called (It is not mandatory for __new__ to return a new instance of its class; it can return whatever you like. That is a feature, and there are occasional uses for it.) -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
Ian Pilcher wrote: > I have created a class to provide a "hash consing"[1] set. > >class UniqueSet(frozenset): > >_registry = dict() > >def __new__(cls, *args, **kwargs): >set = frozenset(*args, **kwargs) >try: >return UniqueSet._registry[set] >except KeyError: >self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) >UniqueSet._registry[set] = self >return self > >def __init__(self, *args, **kwargs): >pass > > I can't figure out how it works, though. In particular, I can't figure > out how the call to __new__ actually initializes the set (since my > __init__ never calls the superclass __init__). > > Is this a particular behavior of frozenset, or am I missing something > about the way that __new__ and __init__ interact? I think __init__() is called to initialise the object after it has been created with __new__(), roughly the following, run by the metaclass: obj = UniqueSet_.__new__(UniqueSet, ...) if isinstance(obj, UniqueSet): obj.__init__(...) As Ian says, for immutable classes __init__() usually does nothing, as by definition the instance cannot be changed once created: >>> t = tuple.__new__(tuple, "ab") >>> t ('a', 'b') >>> t.__init__(None) >>> t.__init__(x=42) >>> t.__init__("whatever", "you", "like") >>> t ('a', 'b') I'm a bit surprised that the signature isn't restricted to a single positional argument... -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ and __init__ - why does this work?
On Tue, Aug 8, 2017 at 6:08 PM, Ian Pilcherwrote: > I have created a class to provide a "hash consing"[1] set. > > class UniqueSet(frozenset): > > _registry = dict() > > def __new__(cls, *args, **kwargs): > set = frozenset(*args, **kwargs) > try: > return UniqueSet._registry[set] > except KeyError: > self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) > UniqueSet._registry[set] = self > return self > > def __init__(self, *args, **kwargs): > pass > > I can't figure out how it works, though. In particular, I can't figure > out how the call to __new__ actually initializes the set (since my > __init__ never calls the superclass __init__). > > Is this a particular behavior of frozenset, or am I missing something > about the way that __new__ and __init__ interact? It's initialized by the superclass call to __new__. frozenset is immutable, and __init__ methods of immutable types generally don't do anything (if they could, then they wouldn't be immutable), which is why it doesn't really matter that you didn't call it. At the same time, it generally doesn't hurt to call it, and you probably shouldn't even have an override of __init__ here if it doesn't do anything. It seems a bit inefficient that you create *two* sets in __new__ and then map one of them to the other in your registry. Why not just create the UniqueSet and then map it to itself if it's not already registered? Something like this (untested): def __new__(cls, *args, **kwargs): self = super(UniqueSet, cls).__new__(cls, *args, **kwargs) return UniqueSet._registry.setdefault(self, self) -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
On Thu, 12 Mar 2015 16:31:12 +1100, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Mario Figueiredo wrote: If this is supposed to be a singleton, you can't create more instances. The point of a singleton that there is only one instance (or perhaps a small number, two or three say). Why do you need two different ways to create instances if you only have one instance? The map is instantiated from the contents of a saved map file on the main application. A different application, a map editor, needs to also instantiate an object of the class Map. But in this case the map needs to either be empty (if the user wants to create a new map), or loaded from the saved map file (if the user wants to edit an existing map). I added the following method to the class definition, above: @classmethod def generate(cls, width, height, fill=terrain[6]): if Map._instance is None: Map._instance = super(Map, cls).__new__(cls) else: raise Exception('Cannot generate an instance of Map.') Map._instance.author = None Map._instance.name = None Since this method modifies the singleton instance in place, it doesn't generate a new instance. It shouldn't be called generate(). No sure what you mean. That method either creates a new instance or raises an exception. It doesn't modify an instance in-place. Map._instance.description = None # etc... self.cells = [Cell(fill)] * width * height return Map._instance That's not your actual code, since the indentation is wrong. Map._instance.description = None # etc... self.cells = [Cell(fill)] * width * height return Map._instance -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
On Wed, 11 Mar 2015 16:47:32 -0700, Ethan Furman et...@stoneleaf.us wrote: You're code is good. Thanks for taking a weight off my shoulder. The only question is if you /really/ need a singleton -- and only you can answer that (although plenty of folks will tell you you don't ;) . Yeah. I debated that myself. But in the end the singleton won. It's just a cheap global, since is ubiquitous throughout the entire application, does behave like a singleton, and is a bit too expensive to create. A full map in the main application takes 3 or 4 seconds to instantiate and occupies around 2 Mb of memory. -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
Mario Figueiredo wrote: It's just a cheap global, since is ubiquitous throughout the entire application, does behave like a singleton, and is a bit too expensive to create. A full map in the main application takes 3 or 4 seconds to instantiate and occupies around 2 Mb of memory. There's nothing wrong with having only one instance. The quesion is whether it's a good idea to make calling Map() be the way to get hold of that instance. I would say it's counterproductive. The implementation is convoluted, and it makes code that calls Map() confusing, because it looks like it's creating a new instance when it really isn't. I would just provide a function: _map = None def get_map(): global _map if _map is None: _map = Map() return _map and document the fact that you shouldn't call Map() directly. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
Mario Figueiredo wrote: A different application, a map editor, needs to also instantiate an object of the class Map. But in this case the map needs to either be empty (if the user wants to create a new map), or loaded from the saved map file (if the user wants to edit an existing map). Then you have two functions for creating maps: new_empty_map() load_map_from_file(filename) Both of these can complain if there is already a Map instance. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
Mario Figueiredo wrote: But PyCharm flags the assignment with a warning telling me that generate() does not return anything and the I lose code completion on the mmap variable. My guess is that there is a syntax error somewhere in your code that's confusing the IDE. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
On Thu, 12 Mar 2015 21:40:03 +1300, Gregory Ewing greg.ew...@canterbury.ac.nz wrote: Mario Figueiredo wrote: A different application, a map editor, needs to also instantiate an object of the class Map. But in this case the map needs to either be empty (if the user wants to create a new map), or loaded from the saved map file (if the user wants to edit an existing map). Then you have two functions for creating maps: new_empty_map() load_map_from_file(filename) Both of these can complain if there is already a Map instance. new_empty_map(width, height, fill=terrain['sea']) load_map_from_file(filename) These would be the most likely signatures for those functions. The first function does not need to perform any of code to do with opening and reading a binary file, verifying an hash for possible file data corruption, filling the map header information and populating the cells. Both functions need to have knowledge of the Map class internals. Is it pythonic to create those functions in the same module of the Map class. Or because of how they have access to some private details of the class they should instead be made static methods as a means to signal that much? Map.new() Map.load() -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
On Thu, 12 Mar 2015 21:38:00 +1300, Gregory Ewing greg.ew...@canterbury.ac.nz wrote: I would just provide a function: _map = None def get_map(): global _map if _map is None: _map = Map() return _map and document the fact that you shouldn't call Map() directly. Oh, you are so right! Been so long since I last created a singleton I forgot all about the idiomatic approaches to the pattern. Thanks Greg. That is much, much, better. -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
Mario Figueiredo wrote: It's just a cheap global, since is ubiquitous throughout the entire application, does behave like a singleton, and is a bit too expensive to create. A full map in the main application takes 3 or 4 seconds to instantiate and occupies around 2 Mb of memory. 2MB is not that big. 3-4 seconds to instantiate is a bit worrying, but you should look at improving the efficiency of loading a map rather than insisting that there should be only one map instance. Particularly in the map editor, what if the user wants to copy parts of one map into a second map? I've often wanted to do that with games. Every other editor, from music editors to text editors, allows you to have multiple documents open. Why should game level editors be stuck in the 1980s? While Map is a singleton, editing multiple maps at once is impossible (or at least tiresomely difficult). -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
On Thu, 12 Mar 2015 21:41:16 +1300, Gregory Ewing greg.ew...@canterbury.ac.nz wrote: Mario Figueiredo wrote: But PyCharm flags the assignment with a warning telling me that generate() does not return anything and the I lose code completion on the mmap variable. My guess is that there is a syntax error somewhere in your code that's confusing the IDE. It's a PyCharm bug actually. Confirmed here: https://youtrack.jetbrains.com/issue/PY-11990 (sidenote: I wasn't finding that before. YouTrack search engine really needs to handle better word contractions. It was only when I was filling a bug report myself that it came up with that one in the list of possible related issues.) -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
On Thu, 12 Mar 2015 22:04:30 +1100, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: 3-4 seconds to instantiate is a bit worrying, but you should look at improving the efficiency of loading a map rather than insisting that there should be only one map instance. Particularly in the map editor, what if the user wants to copy parts of one map into a second map? I've often wanted to do that with games. Every other editor, from music editors to text editors, allows you to have multiple documents open. Why should game level editors be stuck in the 1980s? While Map is a singleton, editing multiple maps at once is impossible (or at least tiresomely difficult). Those scnearios don't apply here. And the map editor isn't even a first-class application in the system. Just a patch to corrupt map files that will thus stand a chance to be recovered. I can understand your resistance to the idea of yet another singleton being born to this world. I myself despise the pattern. I think the GoF were drunk by the time they reached the Singleton. The whole Creational Patterns section was really only going downhill, anyways. Starts strong and ends up with the justin bieber of patterns. And I'm not using it because of some ulterior design principle. I don't plan to justify it because I have a *real* reason to use a singleton. I don't. I'm just using it because it is really accomodating for me to have this global without having to trace it around function calls arguments. That's evil. I suppose I could just make it really global. But to me that's even more evil and I can't consider it. Of course. What is a singleton today can't be twins tomorrow, triplets the day after and a crowd the next. I realize this. But... yeah... I'm a sinner. Besides it's not that I never had to revisit my code before because of Best Practices (tm). In truth the road to damnation starts when we decide to write our first line of code. -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
On Thu, 12 Mar 2015 22:29:24 +1100, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: I would have a loadfile() method which takes a filename on disk, opens the file and passes the contents (or the open file object) to another method, load() to do the actual work: class Map: def __new__(cls, width, height, fill, treasure=None): # Include additional args as needed. map = super().__new__.(cls) map.width = width map.height = height map.fill = fill map.treasure = treasure return map @classmethod def loadfile(cls, filename): with open(filename, 'r') as f: map = cls.load(f) return map @classmethod def load(cls, f): # validate and parse file contents # extract width, height, fill, etc. map = cls(width, height, fill, treasure) return map To get an empty map, you pass the load method a file-like object that contains whatever an empty map data file will contain. That could literally be an external file, if you so choose, or you could simply read it from a global constant or even an embedded string/bytes object: @classmethod def new(cls): Return a new empty map. emptymap = bblah blah blah width=100 height=100 fill=1 f = io.StringIO(emptymap) return cls.load(f) This, naturally, assumes that your map format is simple enough and small enough that an empty map can be embedded into your code. If an empty map is 4MB on disk, you probably don't want to do this :-) Alternatively, set the default arguments to the __new__ constructor to be whatever values an empty map uses: def __new__(cls, width=100, height=100, fill=1, treasure=None): ... and now your create empty map command just calls Map() with no arguments. Your solution is excelent! Leveraging the __new__ method is something I wasn't considering. It gracefully deals with the absence of __init__, not forcing me to raise an exception or document the class to warn against its use. Thanks a bunch. -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
Mario Figueiredo wrote: On Thu, 12 Mar 2015 16:31:12 +1100, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Mario Figueiredo wrote: If this is supposed to be a singleton, you can't create more instances. The point of a singleton that there is only one instance (or perhaps a small number, two or three say). Why do you need two different ways to create instances if you only have one instance? The map is instantiated from the contents of a saved map file on the main application. A different application, a map editor, needs to also instantiate an object of the class Map. But in this case the map needs to either be empty (if the user wants to create a new map), or loaded from the saved map file (if the user wants to edit an existing map). An empty map is just a special case of an existing map, one with no features added. I would have a loadfile() method which takes a filename on disk, opens the file and passes the contents (or the open file object) to another method, load() to do the actual work: class Map: def __new__(cls, width, height, fill, treasure=None): # Include additional args as needed. map = super().__new__.(cls) map.width = width map.height = height map.fill = fill map.treasure = treasure return map @classmethod def loadfile(cls, filename): with open(filename, 'r') as f: map = cls.load(f) return map @classmethod def load(cls, f): # validate and parse file contents # extract width, height, fill, etc. map = cls(width, height, fill, treasure) return map To get an empty map, you pass the load method a file-like object that contains whatever an empty map data file will contain. That could literally be an external file, if you so choose, or you could simply read it from a global constant or even an embedded string/bytes object: @classmethod def new(cls): Return a new empty map. emptymap = bblah blah blah width=100 height=100 fill=1 f = io.StringIO(emptymap) return cls.load(f) This, naturally, assumes that your map format is simple enough and small enough that an empty map can be embedded into your code. If an empty map is 4MB on disk, you probably don't want to do this :-) Alternatively, set the default arguments to the __new__ constructor to be whatever values an empty map uses: def __new__(cls, width=100, height=100, fill=1, treasure=None): ... and now your create empty map command just calls Map() with no arguments. I added the following method to the class definition, above: @classmethod def generate(cls, width, height, fill=terrain[6]): if Map._instance is None: Map._instance = super(Map, cls).__new__(cls) else: raise Exception('Cannot generate an instance of Map.') Map._instance.author = None Map._instance.name = None Since this method modifies the singleton instance in place, it doesn't generate a new instance. It shouldn't be called generate(). No sure what you mean. That method either creates a new instance or raises an exception. It doesn't modify an instance in-place. My mistake, I was confused. You're right, if the instance already exists, it raises an exception. Map._instance.description = None # etc... self.cells = [Cell(fill)] * width * height return Map._instance That's not your actual code, since the indentation is wrong. Map._instance.description = None # etc... self.cells = [Cell(fill)] * width * height return Map._instance What's self? That's a classmethod, it's not usual to have a self local variable. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
On 03/11/2015 04:33 PM, Mario Figueiredo wrote: The following code runs just fine. But PyCharm flags the assignment with a warning telling me that generate() does not return anything and the I lose code completion on the mmap variable. if __name__ == '__main__': mmap = Map.generate(12, 24) print(mmap.width, mmap.height, mmap.author) I need to understand if this is just a glitch of the IDE or I am doing indeed something that is frowned upon and ended up caught in a misleading static analysis warning. You're code is good. The only question is if you /really/ need a singleton -- and only you can answer that (although plenty of folks will tell you you don't ;) . -- ~Ethan~ signature.asc Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__() does not return anything, on singletong pattern
Mario Figueiredo wrote: I'm fairly new to Python, so I don't know if the following is me abusing the programming language idioms, or simply a mistake of my IDE code inspection routine. I have a singleton Map class which is defined like so: class Map: _instance = None def __new__(cls): if Map._instance is None: Map._instance = super(Map, cls).__new__(cls) return Map._instance In Python 2, you need to inherit from object for __new__ to be called. In Python 3, it doesn't matter. def __init__(self, filename): # Instantiates from the contents of a binary file I am now trying to add another way of constructing an instance of this class. (I need to be able to create a dirty empty instance that is going to be used by the separate map editor script). If this is supposed to be a singleton, you can't create more instances. The point of a singleton that there is only one instance (or perhaps a small number, two or three say). Why do you need two different ways to create instances if you only have one instance? I added the following method to the class definition, above: @classmethod def generate(cls, width, height, fill=terrain[6]): if Map._instance is None: Map._instance = super(Map, cls).__new__(cls) else: raise Exception('Cannot generate an instance of Map.') Map._instance.author = None Map._instance.name = None Since this method modifies the singleton instance in place, it doesn't generate a new instance. It shouldn't be called generate(). In fact since this is a singleton, the method shouldn't exist at all. Just put the initialisation code in __new__, drop the __init__ and generate methods, and call Map(argument). def __new__(cls, width, height, fill=terrain[6]): if Map._instance is None: instance = Map._instance = super(Map, cls).__new__(cls) instance.author = None instance.name = None instance.description = None instance.cells = [Cell(fill)] * width * height else: # Ignore the arguments. Why are you passing different arguments # to a singleton class? Bleh. instance = Map._instance return instance Map._instance.description = None # etc... self.cells = [Cell(fill)] * width * height return Map._instance That's not your actual code, since the indentation is wrong. What does your actual code look like? You are running PyCharm on one piece of code, then showing us *different* (edited) code. We cannot see what PyCharm sees, so how do we know why PyCharm says what it says? The following code runs just fine. But PyCharm flags the assignment with a warning telling me that generate() does not return anything and the I lose code completion on the mmap variable. if __name__ == '__main__': mmap = Map.generate(12, 24) print(mmap.width, mmap.height, mmap.author) I need to understand if this is just a glitch of the IDE or I am doing indeed something that is frowned upon and ended up caught in a misleading static analysis warning. Probably. -- Steve -- https://mail.python.org/mailman/listinfo/python-list
Re: __new__ woes with list
macaronikazoo [EMAIL PROTECTED] writes: i'm having a hell of a time getting this to work. basically I want to be able to instantiate an object using either a list, or a string, but the class inherits from list. if the class is instantiated with a string, then run a method over it to tokenize it in a meaningful way. so how come this doesn't work??? if I do this: a=TMP( 'some string' ) it does nothing more than list('some string') and seems to be ignoring the custom __new__ method. def convertDataToList( data ): return [1,2,3] class TMP(list): def __new__( cls, data ): if isinstance(data, basestring): new = convertDataToList( data ) return list.__new__( cls, new ) if isinstance(data, list): return list.__new__( cls, data ) A list is mutable, its initialisation is done in __init__() not __new__(). There was a recent post about this (in the last couple of weeks). -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__ woes with list
ok thansk - i will search again. i did try searching but didn't find anything relevant... -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__ woes with list
macaronikazoo [EMAIL PROTECTED] writes: ok thansk - i will search again. i did try searching but didn't find anything relevant... Here's a link to the thread on google groups: http://groups.google.com/group/comp.lang.python/browse_thread/thread/7aff4eabc6182858 Unfortunately two threads seem to be intertwined: scroll down to when the thread is renamed python bug when subclassing list?. HTH -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
Rhamphoryncus wrote: On Aug 4, 11:46 am, Ethan Furman [EMAIL PROTECTED] wrote: Mel wrote: Ethan Furman wrote: Emile van Sebille wrote: Ethan Furman wrote: -- d25._int = (1, 5) Python considers names that start with a leading underscore as internal or private, and that abuse is the burden of the abuser... Is bytecodehacks still around? That was serious abuse :) Good point. What I'm curious about, though, is the comment in the code about making the Decimal instance immutable. I was unable to find docs on that issue. There's something in the Language Reference, chapter 3.1 'Objects, Values and Types'. Mel. Thanks, Mel. I had actually read that once already, but your post caused me to reread it, and evidently the ideas there had had enough time to percolate through my mind. -- from decimal import Decimal -- d25 = Decimal(25) -- d25 Decimal(25) -- d25.testing = 'immutable' Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'Decimal' object has no attribute 'testing' Decimals are immutable in that we cannot add new attributes to them. The documentation in Language Reference 3.4.2.4 '__slots__' has this to say: If defined in a new-style class, __slots__ reserves space for the declared variables and prevents the automatic creation of __dict__ and __weakref__ for each instance. and Without a __dict__ variable, instances cannot be assigned new variables not listed in the __slots__ definition. Attempts to assign to an unlisted variable name raises AttributeError. So the question I have now is this: is __new__ necessary, or would __init__ have also worked? Let's see... class tester(object): __slots__ = ['test1', 'test2', 'test3'] def __init__(self, value1, value2, value3): self.test1 = value1 self.test2 = value2 self.test3 = value3 -- import tester -- testee = tester.tester(1, 2, 3) -- testee tester.tester object at 0x009E7328 -- dir(testee) ['__class__', '__delattr__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__slots__', '__str__', 'test1', 'test2', 'test3'] -- testee.test4 = 4 Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'tester' object has no attribute 'test4' For this simple test, it looks like __init__ works just fine. So, besides consistency (which is important) am I missing any other reasons to use __new__ instead of __init__? If you subclass a builtin immutable like int then you need to override __new__, as __init__ has no effect. Decimal is written in python, so this is irrelevant, but if there are plans to rewrite it in C (which I believe there are) then it'd need to use __new__ at that point. Using __new__ even in the python version then becomes a way to future-proof the API. Thanks, Rhamphoryncus! -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
Calvin Spealman wrote: [snip] ask if you really feel the need to know. I am. ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
Ethan Furman wrote: Emile van Sebille wrote: Ethan Furman wrote: -- d25._int = (1, 5) Python considers names that start with a leading underscore as internal or private, and that abuse is the burden of the abuser... Is bytecodehacks still around? That was serious abuse :) Good point. What I'm curious about, though, is the comment in the code about making the Decimal instance immutable. I was unable to find docs on that issue. There's something in the Language Reference, chapter 3.1 'Objects, Values and Types'. Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
Mel wrote: Ethan Furman wrote: Emile van Sebille wrote: Ethan Furman wrote: -- d25._int = (1, 5) Python considers names that start with a leading underscore as internal or private, and that abuse is the burden of the abuser... Is bytecodehacks still around? That was serious abuse :) Good point. What I'm curious about, though, is the comment in the code about making the Decimal instance immutable. I was unable to find docs on that issue. There's something in the Language Reference, chapter 3.1 'Objects, Values and Types'. Mel. Thanks, Mel. I had actually read that once already, but your post caused me to reread it, and evidently the ideas there had had enough time to percolate through my mind. -- from decimal import Decimal -- d25 = Decimal(25) -- d25 Decimal(25) -- d25.testing = 'immutable' Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'Decimal' object has no attribute 'testing' Decimals are immutable in that we cannot add new attributes to them. The documentation in Language Reference 3.4.2.4 '__slots__' has this to say: If defined in a new-style class, __slots__ reserves space for the declared variables and prevents the automatic creation of __dict__ and __weakref__ for each instance. and Without a __dict__ variable, instances cannot be assigned new variables not listed in the __slots__ definition. Attempts to assign to an unlisted variable name raises AttributeError. So the question I have now is this: is __new__ necessary, or would __init__ have also worked? Let's see... class tester(object): __slots__ = ['test1', 'test2', 'test3'] def __init__(self, value1, value2, value3): self.test1 = value1 self.test2 = value2 self.test3 = value3 -- import tester -- testee = tester.tester(1, 2, 3) -- testee tester.tester object at 0x009E7328 -- dir(testee) ['__class__', '__delattr__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__slots__', '__str__', 'test1', 'test2', 'test3'] -- testee.test4 = 4 Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'tester' object has no attribute 'test4' For this simple test, it looks like __init__ works just fine. So, besides consistency (which is important) am I missing any other reasons to use __new__ instead of __init__? ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
On Aug 4, 11:46 am, Ethan Furman [EMAIL PROTECTED] wrote: Mel wrote: Ethan Furman wrote: Emile van Sebille wrote: Ethan Furman wrote: -- d25._int = (1, 5) Python considers names that start with a leading underscore as internal or private, and that abuse is the burden of the abuser... Is bytecodehacks still around? That was serious abuse :) Good point. What I'm curious about, though, is the comment in the code about making the Decimal instance immutable. I was unable to find docs on that issue. There's something in the Language Reference, chapter 3.1 'Objects, Values and Types'. Mel. Thanks, Mel. I had actually read that once already, but your post caused me to reread it, and evidently the ideas there had had enough time to percolate through my mind. -- from decimal import Decimal -- d25 = Decimal(25) -- d25 Decimal(25) -- d25.testing = 'immutable' Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'Decimal' object has no attribute 'testing' Decimals are immutable in that we cannot add new attributes to them. The documentation in Language Reference 3.4.2.4 '__slots__' has this to say: If defined in a new-style class, __slots__ reserves space for the declared variables and prevents the automatic creation of __dict__ and __weakref__ for each instance. and Without a __dict__ variable, instances cannot be assigned new variables not listed in the __slots__ definition. Attempts to assign to an unlisted variable name raises AttributeError. So the question I have now is this: is __new__ necessary, or would __init__ have also worked? Let's see... class tester(object): __slots__ = ['test1', 'test2', 'test3'] def __init__(self, value1, value2, value3): self.test1 = value1 self.test2 = value2 self.test3 = value3 -- import tester -- testee = tester.tester(1, 2, 3) -- testee tester.tester object at 0x009E7328 -- dir(testee) ['__class__', '__delattr__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__slots__', '__str__', 'test1', 'test2', 'test3'] -- testee.test4 = 4 Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'tester' object has no attribute 'test4' For this simple test, it looks like __init__ works just fine. So, besides consistency (which is important) am I missing any other reasons to use __new__ instead of __init__? If you subclass a builtin immutable like int then you need to override __new__, as __init__ has no effect. Decimal is written in python, so this is irrelevant, but if there are plans to rewrite it in C (which I believe there are) then it'd need to use __new__ at that point. Using __new__ even in the python version then becomes a way to future-proof the API. -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
Ethan Furman wrote: -- d25._int = (1, 5) Python considers names that start with a leading underscore as internal or private, and that abuse is the burden of the abuser... Is bytecodehacks still around? That was serious abuse :) Emile -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
Emile van Sebille wrote: Ethan Furman wrote: -- d25._int = (1, 5) Python considers names that start with a leading underscore as internal or private, and that abuse is the burden of the abuser... Is bytecodehacks still around? That was serious abuse :) Emile Good point. What I'm curious about, though, is the comment in the code about making the Decimal instance immutable. I was unable to find docs on that issue. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
its a good point you make. if its not _technically_ immutable, why use __new__ when __init__ would work just as fine? well, if it should be treated as immutable, then we should do what we can to follow that, even in internal code that knows otherwise. Besides, maybe down the road, protections will be added to disallow assignment to _int, and the author thought to future proof it. In any case, the author can only know, perhaps. ask if you really feel the need to know. On Sun, Aug 3, 2008 at 2:32 PM, Ethan Furman [EMAIL PROTECTED] wrote: Emile van Sebille wrote: Ethan Furman wrote: -- d25._int = (1, 5) Python considers names that start with a leading underscore as internal or private, and that abuse is the burden of the abuser... Is bytecodehacks still around? That was serious abuse :) Emile Good point. What I'm curious about, though, is the comment in the code about making the Decimal instance immutable. I was unable to find docs on that issue. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
James Stroud wrote: Hello All, I'm running 2.3.4 I was reading the documentation for classes types http://www.python.org/2.2.3/descrintro.html And stumbled on this paragraph: __new__ must return an object. There's nothing that requires that it return a new object that is an instance of its class argument, although that is the convention. If you return an existing object, the constructor call will still call its __init__ method. If you return an object of a different class, its __init__ method will be called. Any reason why you're looking at 2.2 documentation when you're running 2.3? Anyway, the current docs corrected this mistake[1] If __new__() returns an instance of cls, then the new instance's __init__() method will be invoked like __init__(self[, ...]), where self is the new instance and the remaining arguments are the same as were passed to __new__(). If __new__() does not return an instance of cls, then the new instance's __init__() method will not be invoked. [1]http://docs.python.org/ref/customization.html STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
James Stroud wrote: I'm running 2.3.4 I was reading the documentation for classes types http://www.python.org/2.2.3/descrintro.html And stumbled on this paragraph: __new__ must return an object. There's nothing that requires that it return a new object that is an instance of its class argument, although that is the convention. If you return an existing object, the constructor call will still call its __init__ method. If you return an object of a different class, its __init__ method will be called. The quote implies that when I call carol, b.__init__ should be called. However, this does not seem to be the case (see code below). What am I not understanding? Shouldn't the interpreter call b.__init__ when b is returned from carol.__new__? Here's what Python in a Nutshell (p84) says: Each new-style class has a static method named __new__. When you call C(*args, **kwds) to create a new instance of a new-style class C, Python invokes C.__new__(C, *args, **kwds). Python uses __new__'s return value x as the newly created instance. Then, Python calls C.__init__(x, *args, **kwds), but only when x is indeed an instance of C (otherwise, x's state is as __new__ had left it). Thus, for a new-style class C, the statement x=C(23) is equivlent to the following code: x = C.__new__(C, 23) if isinstance(x, C): C.__init__(x, 23) If the following code says what I think it does class A(object): def __init__(self): print init A def __new__(cls): return b class B(A): def __init__(self): print init B b = object.__new__(B) print --- A() # prints 'init B' A.__init__(b)# prints 'init A' print --- b = 42 A() # prints nothing the Nutshell example should be changed to x = C.__new__(C, 23) if isinstance(x, C): x.__init__(23) to comply with the current implementation (I used Python 2.4). Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
Peter Otten [EMAIL PROTECTED] wrote: ... is as __new__ had left it). Thus, for a new-style class C, the statement x=C(23) is equivlent to the following code: x = C.__new__(C, 23) if isinstance(x, C): C.__init__(x, 23) ... the Nutshell example should be changed to x = C.__new__(C, 23) if isinstance(x, C): x.__init__(23) to comply with the current implementation (I used Python 2.4). Hmmm -- not quite, because in the new-style object model special methods are taken from the type, NOT from the instance as the latter is saying. E.g, if you change class B into: class B(A): def __init__(self): print init B def f(): print 'from instance' self.__init__ = f you'll see that while b.__init__() would print 'from instance', instantiating A will not. I think the right correction to the Nutshell is therefore: x = C.__new__(C, 23) if isinstance(x, C): type(x).__init__(x, 23) and this is how I plan to have it in the 2nd edition. Alex -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__
James Stroud wrote: The quote implies that when I call carol, b.__init__ should be called. However, this does not seem to be the case (see code below). What am I not understanding? Shouldn't the interpreter call b.__init__ when b is returned from carol.__new__? James py class bob(object): ... def __init__(self): ... print self.x ... x = 2 ... py class carol(object): ... def __new__(cls): ... return b ... py b=bob() py b.x 2 py c = carol() # should print 2 py c __main__.bob object at 0x404333cc It seems to produce the output you expected for me (Python 2.4.1 on Windows XP), but this has nothing to do with carol. How are bob and carol related? Code: class bob(object): def __init__(self): print self.x x = 2 class carol(object): def __new__(cls): return b b=bob() print b.x c = carol() c Output: 2 2 This code produces the same output: class bob(object): def __init__(self): print self.x x = 2 ##class carol(object): ##def __new__(cls): ##return b b=bob() print b.x ##c = carol() ##c I am interested in the underlying subject but think your code was mispasted into the e-mail... -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__ does not call __init__ as described in descrintro.html (WAS: Can __new__ prevent __init__ from being called?)
In article [EMAIL PROTECTED], Steven Bethard [EMAIL PROTECTED] wrote: Yeah, I saw the same thing in playing around with this. Don't know what to make of it. I wonder if we should file a documentation bug? I can't find __new__ explained anywhere in the Language Reference. Can documentation bugs be filed for descrintro.html? Absolutely! And they're especially welcome if they contain suggested replacement text. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death. --GvR -- http://mail.python.org/mailman/listinfo/python-list
Re: __new__ does not call __init__ as described in descrintro.html (WAS: Can __new__ prevent __init__ from being called?)
Aahz wrote: In article [EMAIL PROTECTED], Steven Bethard [EMAIL PROTECTED] wrote: Yeah, I saw the same thing in playing around with this. Don't know what to make of it. I wonder if we should file a documentation bug? I can't find __new__ explained anywhere in the Language Reference. Can documentation bugs be filed for descrintro.html? Absolutely! And they're especially welcome if they contain suggested replacement text. Cool. I added a documentation bug: http://sourceforge.net/tracker/?func=detailaid=1123716group_id=5470atid=105470 Steve -- http://mail.python.org/mailman/listinfo/python-list