Re: questions ( answers) about object, type, builtin types, class, metaclass and __getattribute__
On Aug 22, 1:57 pm, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: The relationship between type and object is somewhat special, and needs to be bootstrapped by the CPython virtual machine. Since you are talking about CPython, I'm wondering how it is bootstraped since you can easly reference PyType in PyObject that part is not hard. 2) type is its own metaclass : type(type) is type ? Yes. Another bit of bootstrapping that the compiler does. self reference is easy same as referencing PyType from PyObject and PyObject from PyType. 5) type(any_object) == last_metaclass_..., which is, most of the time, type ? I'm not sure what you mean by last_metaclass. But no. The type of an object is its class: see this code for example proove my point: class meta_a(type): def __new__(cls, *args, **kwargs): return type.__new__(cls, *args, **kwargs) # see (¤) class meta_b(meta_a): def __new___(cls, *args, **kwargs): return meta_a.__new__(cls, *args, **kwargs) # same as above class ClassWithTypeMetaA(object): __metaclass__ = meta_a class ClassWithTypeMetaB(object): __metaclass__ = meta_b type(ClassWithTypeMetaA) == meta_a type(ClassWithTypeMetaB) == meta_b [¤] super call doesn't work here, anyone can say why ? Regards, Amirouche -- http://mail.python.org/mailman/listinfo/python-list
Re: questions ( answers) about object, type, builtin types, class, metaclass and __getattribute__
On Aug 22, 5:41 pm, Stephen Hansen me+list/pyt...@ixokai.io wrote: 3) object's type is type : object.__class__ is type 4) type parent object is object : type.__bases__ == (object,) Saying type and parent and the like for new-style classes is something of a misnomer. For type and object, these things aren't constructed like this. What you have here is technically true if you go poke at it in the interpreter, but it doesn't really /mean/ anything because its not how these objects came to be and is circular and a bit confusing. These fundamental objects are created special. The code snippet is here to illustrate how it is visible in the interpreter. But you are right. 2) type is its own metaclass : type(type) is type ? Only in a purely theoretical way. It doesn't actually mean anything; moreover, type(something) is NOT how you determine somethings metaclass. Its how you determine somethings type. see the answer to Steven D'Aprano. type(class_object) == a_meta_class_object The two concepts may be very distinct. Lots of things don't have metaclasses. All object in new style class have a metaclass at least type. 3) object's metaclass is type ? Again, only theoretically. and again the famous bootstrapping make it like type created object. From the outside world of the Python implementation object looks like a type instance. 5) type(any_object) == last_metaclass_..., which is, most of the time, type ? Not necessarily at all. In fact, there is no way I'm aware of to determine if a metaclass was involved in a classes construction unless said metaclass wants to provide such a mechanism. Metaclasses are kind of a hack. They are a way to hook into the class construction that's normally done and do something, anything you want, (even hijack the whole procedure and NOT construct a class at all, but play a song if you want) before its all finished. For example, this is a metaclass I've used: PageTypes = {} class _PageRegistration(type): def __new__(cls, name, bases, dct): klass = type.__new__(cls, name, bases, dct) typename = name[:-9].lower() if not typename: typename = None PageTypes[typename] = klass klass.Type = typename return klass class QueuePage(sc.SizedPanel): __metaclass__ = _PageRegistration Note, the fact that my _PageRegistration metaclass inherits is itself a class which inherits from type is just one convenient way to write metaclasses. It could as simply have been just a function. Metaclasses are somewhat poorly named in that they are really, creation hooks. It the same issue in django, views are only function, until you need complex behavior and you want a namespace to put everything in it. IMO that's why class based views exists for complex cases. That said being able to declare a metaclass only as a functions is neat. C) type vs class 1) Type is the metaclass of most classes Yes and no. Yes, in that most classes are created using the default mechanism inside CPython. The class body is executed in a scope, the resulting dictionary is bound to a new class object, bases and the like are set, and such. No in that it really just, IIUC, skips the whole metaclass part of the process because this 'default mechanism' doesn't need to call out into other code to do its job. At least, I think-- May be wrong here, metaclasses are something of a dark voodoo and I'm not 100% entirely familiar with the internal workings of CPython. But functionally, a metaclass is the chunk of code responsible for the actual physical construction of the class object. For me it takes some variables, namely ``bases``, ``class_dict`` and ``configuration class_name`` and do something with it, probably creating a class_object which behaviour is parametred with the context. I did not know Python before new-style class, so probably for most people explainning that metaclasses are a creation hook is easier for them... 4) It's in type.__call__ that happens calls to __new__ and __init__ Again, translates to is suggesting this is what happens when you do X, which I don't know if is strictly true. CPython inside may be optimizing this whole process. Especially when it comes to magic methods, __x__ and the like -- CPython rarely uses __get*_ for those. It just calls the methods directly on the class object. IIUC the code of Jython tells me what I've written. If the first part of the algorithm is lookup for special methods (what you seem to say) then we both agree that we agree, isn't it ? Moreover I'm not looking in this part to understand how CPython works internally, but how Python works. Since I'm most proeffencient in Python I translate it to Python. *Translates* means it's a shortcut for. 5) 3) = classes are instance of type 6) Since type.__call__ is used to instantiate instance of instance
questions ( answers) about object, type, builtin types, class, metaclass and __getattribute__
I'm learning a bit of python internals lately and I'm trying to figure out the relationship between type, objects, class, callables and __getattribute__ resolution. While understanding Python mechanics/concepts, I'm trying to figure how it translates in CPython. This post is Python centric. Questions about the implementation of this concepts might be the subject of a future post [1]. I will proceed this way: I will write statements about each subject. Don't hesitate to pick one and provide more information or better phrasing, or explaining why it's not true. Be aware that I'm considering only new-style class. A) type vs object - 1) object is the base object, it has no bases : len(object.__bases__) == 0 2) every object in python inherit object : any_object_except_object.__bases__[-1] is object 3) object's type is type : object.__class__ is type 4) type parent object is object : type.__bases__ == (object,) B) type vs metaclass 1) type is the first metaclass ? 2) type is its own metaclass : type(type) is type ? 3) object's metaclass is type ? 4) other metaclasses *MUST* inherit type ? 5) type(any_object) == last_metaclass_..., which is, most of the time, type ? C) type vs class 1) Type is the metaclass of most classes 2) The class statement:: class MyClass(object): attribute = 1 def method(self): pass translates to:: MyClass = type('MyClass', (object,), {'attribute': 1, 'method': def method: pass }) 3) Instantiation of any class ``MyClass(*args, **kwargs)`` translates to:: type(MyClass).__call__(MyClass, *args, **kwargs) This is due to __getattribute__ algorithm (see E) 4) It's in type.__call__ that happens calls to __new__ and __init__ 5) 3) = classes are instance of type 6) Since type.__call__ is used to instantiate instance of instance of type (rephrased: __call__ is used to instantiate classes) where is the code which is executed when we write ``type(myobject)`` or ``type('MyClass', bases, attributes)`` __getattribute__ resolution algorithm (see E) tells me that it should be type.__call__ but type.__call__ is already used to class instatiation. C') class vs class instances aka. objects - 1) A class type is a metaclass : issubclass(type(MyClass), type), MyClass.__class__ == type(MyClass) 2) An object type is a class : most of the time isinstance(type(my_object), type) generally issubclass(type(type(my_object)), type) D) builtin types 1) builtin types are their own metaclass ? 2) why function builtin type can not be subclassed ? 3) how does builtin function type relate to callable objects ? 4) int(1) is the same as int.__call__(1), since type(int) is type, shouldn't int(1) translates to type.__call__(int, 1) ? E) __getattribute__ --- 1) ``my_object.attribute`` always translates to ``my_object.__getattribute__('attribute')`` 2) Is the following algorithm describing __getattribute__ correct [2], beware that I've added a comment: a) If attrname is a special (i.e. Python-provided) attribute for objectname, return it. # what does it mean to be Python-provided ? b ) Check objectname.__class__.__dict__ for attrname. If it exists and is a data-descriptor, return the descriptor result. Search all bases of objectname.__class__ for the same case. c) Check objectname.__dict__ for attrname, and return if found. d) If it is a class and a descriptor exists in it or its bases, return the descriptor result. d) Check objectname.__class__.__dict__ for attrname. If it exists and is a non-data descriptor, return the descriptor result. If it exists, and is not a descriptor, just return it. If it exists and is a data descriptor, we shouldn't be here because we would have returned at point 2. Search all bases of objectname.__class__ for same case. e) Raise AttributeError Thanks in advance, Regards, Amirouche [1] or maybe you can point me to the right direction before I ask stupid questions. I'm a bit familiar with Jython code, which seems to be easier than PyPy and CPython to read even if there is some magic due to the fact that Jython use some JVM API that hides somewhat how it works. [2] this is a rewritten version of http://www.cafepy.com/article/python_attributes_and_methods/python_attributes_and_methods.html#attribute-search-summary -- http://mail.python.org/mailman/listinfo/python-list
Re: questions ( answers) about object, type, builtin types, class, metaclass and __getattribute__
Amirouche B. wrote: A) type vs object - 1) object is the base object, it has no bases : len(object.__bases__) == 0 Correct, but for reference, a more direct test is: object.__bases__ == () (no need for len). 2) every object in python inherit object : any_object_except_object.__bases__[-1] is object Excluding old-style objects, I believe you are correct. 3) object's type is type : object.__class__ is type 4) type parent object is object : type.__bases__ == (object,) The relationship between type and object is somewhat special, and needs to be bootstrapped by the CPython virtual machine. Arbitrary types (classes) inherit from object. That means the type is a subclass of object: class K(object):pass ... issubclass(K, object) True What's less obvious is that types are themselves objects, and therefore are instances of object: isinstance(K, object) True Since classes are objects, they have a type, namely ``type``. This includes ``type`` itself: * type is an instance of object * object is an instance of type * type is a subclass of object * but object is NOT a subclass of type B) type vs metaclass 1) type is the first metaclass ? Excluding old-style classes, yes, all custom classes (those you create with the class statement) have a default metaclass of type. 2) type is its own metaclass : type(type) is type ? Yes. Another bit of bootstrapping that the compiler does. type(type) is type True 3) object's metaclass is type ? Yes. 4) other metaclasses *MUST* inherit type ? No. Metaclasses can be anything that mimics type. def meta(name, bases, dict): ... class X(object): ... pass ... return X ... class K(object): ... __metaclass__ = meta ... a = 1 ... K class '__main__.X' They don't even need to return a type/class. Like decorators, they can return anything. def meta(name, bases, dict): ... return spam ... class K(object): ... __metaclass__ = meta ... K 'spam' 5) type(any_object) == last_metaclass_..., which is, most of the time, type ? I'm not sure what you mean by last_metaclass. But no. The type of an object is its class: type(42) = int type(spam) = str type(1.23) = float However, the type of a class is *usually* type. C) type vs class 1) Type is the metaclass of most classes Yes. 2) The class statement:: class MyClass(object): attribute = 1 def method(self): pass translates to:: MyClass = type('MyClass', (object,), {'attribute': 1, 'method': def method: pass }) Except that the syntax won't work, the idea is broadly correct. 3) Instantiation of any class ``MyClass(*args, **kwargs)`` translates to:: type(MyClass).__call__(MyClass, *args, **kwargs) Like any function call, MyClass(...) becomes type(MyClass).__call__(self, ...) with self=MyClass. Since type(MyClass) is usually ``type``, that gives: type.__call__(MyClass, ...) This is due to __getattribute__ algorithm (see E) 4) It's in type.__call__ that happens calls to __new__ and __init__ If type were written in pure Python, it would probably look something like this: class Type(object): # ... # other methods # ... def __call__(cls, *args, **kwargs): instance = cls.__new__(cls, *args, **kwargs) if isinstance(instance, cls): instance.__init__(*args, **kwargs) return instance But see further on, for more complication. Note that __new__ is special-cased as a staticmethod, hence it needs the first argument to be passed directly. 5) 3) = classes are instance of type 6) Since type.__call__ is used to instantiate instance of instance of type (rephrased: __call__ is used to instantiate classes) where is the code which is executed when we write ``type(myobject)`` or ``type('MyClass', bases, attributes)`` You would need to check the C implementation of type, but if I were doing this in pure Python, I'd have something like this: class Type(object): def __call__(self, *args, **kwargs): if self is Type: if kwargs: raise TypeError('unexpected keyword arguments') # calling type(...) directly if len(args) == 1: # Single argument call, like type(x) return x.__class__ else: # Like type(name, bases, dict) name, bases, dict = *args cls = Type.__new__(Type, name, bases, dict) if isinstance(cls, Type): cls.__init__(name, bases, dict) else: # called from MyClass(...) # which becomes type(MyClass).__call__(MyClass, ...) # self here equals MyClass, which is an instance of type but # not type itself instance = self.__new__(self, *args, **kwargs) if isinstance(instance, self):
Re: questions ( answers) about object, type, builtin types, class, metaclass and __getattribute__
On 8/22/11 3:02 AM, Amirouche B. wrote: A) type vs object - 1) object is the base object, it has no bases : len(object.__bases__) == 0 2) every object in python inherit object : any_object_except_object.__bases__[-1] is object Not exactly. Python has two somewhat different object models, old style classes and new style classes, with slightly different behavior and internal structure. class Foo: pass is an old-style class, dated back to Python's ancient past. This all relates to the fact that 'type' and 'class' used to be two pretty different things, where one was something you mostly did only in C, and one was something you did (mostly) only in Python. They are largely the same now. 3) object's type is type : object.__class__ is type 4) type parent object is object : type.__bases__ == (object,) Saying type and parent and the like for new-style classes is something of a misnomer. For type and object, these things aren't constructed like this. What you have here is technically true if you go poke at it in the interpreter, but it doesn't really /mean/ anything because its not how these objects came to be and is circular and a bit confusing. These fundamental objects are created special. B) type vs metaclass 1) type is the first metaclass ? Type is the basic, default metaclass, yes. A metaclass is a callable that constructs class objects. 2) type is its own metaclass : type(type) is type ? Only in a purely theoretical way. It doesn't actually mean anything; moreover, type(something) is NOT how you determine somethings metaclass. Its how you determine somethings type. The two concepts may be very distinct. Lots of things don't have metaclasses. 3) object's metaclass is type ? Again, only theoretically. 4) other metaclasses *MUST* inherit type ? Absolutely not. Any callable can be a metaclasss. Despite its name, it doesn't have to be itself a class or anything. Just something you can call with er, 3 (I forget exactly) arguments, and which returns a constructed class object. 5) type(any_object) == last_metaclass_..., which is, most of the time, type ? Not necessarily at all. In fact, there is no way I'm aware of to determine if a metaclass was involved in a classes construction unless said metaclass wants to provide such a mechanism. Metaclasses are kind of a hack. They are a way to hook into the class construction that's normally done and do something, anything you want, (even hijack the whole procedure and NOT construct a class at all, but play a song if you want) before its all finished. For example, this is a metaclass I've used: PageTypes = {} class _PageRegistration(type): def __new__(cls, name, bases, dct): klass = type.__new__(cls, name, bases, dct) typename = name[:-9].lower() if not typename: typename = None PageTypes[typename] = klass klass.Type = typename return klass class QueuePage(sc.SizedPanel): __metaclass__ = _PageRegistration Note, the fact that my _PageRegistration metaclass inherits is itself a class which inherits from type is just one convenient way to write metaclasses. It could as simply have been just a function. Metaclasses are somewhat poorly named in that they are really, creation hooks. C) type vs class 1) Type is the metaclass of most classes Yes and no. Yes, in that most classes are created using the default mechanism inside CPython. The class body is executed in a scope, the resulting dictionary is bound to a new class object, bases and the like are set, and such. No in that it really just, IIUC, skips the whole metaclass part of the process because this 'default mechanism' doesn't need to call out into other code to do its job. At least, I think-- May be wrong here, metaclasses are something of a dark voodoo and I'm not 100% entirely familiar with the internal workings of CPython. But functionally, a metaclass is the chunk of code responsible for the actual physical construction of the class object. 2) The class statement:: class MyClass(object): attribute = 1 def method(self): pass translates to:: MyClass = type('MyClass', (object,), {'attribute': 1, 'method': def method: pass }) Translates to, I don't know about that. Is functionally equivalent, yes. It is more or less what happens. 3) Instantiation of any class ``MyClass(*args, **kwargs)`` translates to:: type(MyClass).__call__(MyClass, *args, **kwargs) This is due to __getattribute__ algorithm (see E) 4) It's in type.__call__ that happens calls to __new__ and __init__ Again, translates to is suggesting this is what happens when you do X, which I don't know if is strictly true. CPython inside may be optimizing this whole process. Especially when it comes to magic methods, __x__ and the like -- CPython rarely uses __get*_ for
Re: questions ( answers) about object, type, builtin types, class, metaclass and __getattribute__
On Mon, Aug 22, 2011 at 4:41 PM, Stephen Hansen me+list/pyt...@ixokai.io wrote: Not exactly. Python has two somewhat different object models, old style classes and new style classes, with slightly different behavior and internal structure. class Foo: pass is an old-style class, dated back to Python's ancient past. This all relates to the fact that 'type' and 'class' used to be two pretty different things, where one was something you mostly did only in C, and one was something you did (mostly) only in Python. They are largely the same now. And now includes everything in Python 3, where a class implicitly derives from object if no other subclassing is given. ChrisA -- http://mail.python.org/mailman/listinfo/python-list