Dag Sverre Seljebotn <d.s.seljeb...@astro.uio.no> wrote:
>On 05/28/2012 01:24 PM, Nathaniel Smith wrote: >> On Mon, May 28, 2012 at 12:09 PM, mark florisson >> <markflorisso...@gmail.com> wrote: >>> On 28 May 2012 12:01, Nathaniel Smith<n...@pobox.com> wrote: >>>> On Mon, May 28, 2012 at 11:55 AM, mark florisson >>>> <markflorisso...@gmail.com> wrote: >>>>> On 28 May 2012 11:41, Nathaniel Smith<n...@pobox.com> wrote: >>>>>> On Mon, May 28, 2012 at 10:13 AM, mark florisson >>>>>> <markflorisso...@gmail.com> wrote: >>>>>>> On 28 May 2012 09:54, mark florisson<markflorisso...@gmail.com> >wrote: >>>>>>>> On 27 May 2012 23:12, Nathaniel Smith<n...@pobox.com> wrote: >>>>>>>>> On Sun, May 27, 2012 at 10:24 PM, Dag Sverre Seljebotn >>>>>>>>> <d.s.seljeb...@astro.uio.no> wrote: >>>>>>>>>> On 05/18/2012 10:30 AM, Dag Sverre Seljebotn wrote: >>>>>>>>>>> >>>>>>>>>>> On 05/18/2012 12:57 AM, Nick Coghlan wrote: >>>>>>>>>>>> >>>>>>>>>>>> I think the main things we'd be looking for would be: >>>>>>>>>>>> - a clear explanation of why a new metaclass is considered >too complex a >>>>>>>>>>>> solution >>>>>>>>>>>> - what the implications are for classes that have nothing >to do with the >>>>>>>>>>>> SciPy/NumPy ecosystem >>>>>>>>>>>> - how subclassing would behave (both at the class and >metaclass level) >>>>>>>>>>>> >>>>>>>>>>>> Yes, defining a new metaclass for fast signature exchange >has its >>>>>>>>>>>> challenges - but it means that *our* concerns about >maintaining >>>>>>>>>>>> consistent behaviour in the default object model and >avoiding adverse >>>>>>>>>>>> effects on code that doesn't need the new behaviour are >addressed >>>>>>>>>>>> automatically. >>>>>>>>>>>> >>>>>>>>>>>> Also, I'd consider a functioning reference implementation >using a custom >>>>>>>>>>>> metaclass a requirement before we considered modifying type >anyway, so I >>>>>>>>>>>> think that's the best thing to pursue next rather than a >PEP. It also >>>>>>>>>>>> has the virtue of letting you choose which Python versions >to target and >>>>>>>>>>>> iterating at a faster rate than CPython. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This seems right on target. I could make a utility code C >header for >>>>>>>>>>> such a metaclass, and then the different libraries can all >include it >>>>>>>>>>> and handshake on which implementation becomes the real one >through >>>>>>>>>>> sys.modules during module initialization. That way an >eventual PEP will >>>>>>>>>>> only be a natural incremental step to make things more >polished, whether >>>>>>>>>>> that happens by making such a metaclass part of the standard >library or >>>>>>>>>>> by extending PyTypeObject. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> So I finally got around to implementing this: >>>>>>>>>> >>>>>>>>>> https://github.com/dagss/pyextensibletype >>>>>>>>>> >>>>>>>>>> Documentation now in a draft in the NumFOCUS SEP repo, which >I believe is a >>>>>>>>>> better place to store cross-project standards like this. (The >NumPy >>>>>>>>>> docstring standard will be SEP 100). >>>>>>>>>> >>>>>>>>>> https://github.com/numfocus/sep/blob/master/sep200.rst >>>>>>>>>> >>>>>>>>>> Summary: >>>>>>>>>> >>>>>>>>>> - No common runtime dependency >>>>>>>>>> >>>>>>>>>> - 1 ns overhead per lookup (that's for the custom slot >*alone*, no >>>>>>>>>> fast-callable signature matching or similar) >>>>>>>>>> >>>>>>>>>> - Slight annoyance: Types that want to use the metaclass >must be a >>>>>>>>>> PyHeapExtensibleType, to make the binary layout work with how >CPython makes >>>>>>>>>> subclasses from Python scripts >>>>>>>>>> >>>>>>>>>> My conclusion: I think the metaclass approach should work >really well. >>>>>>>>> >>>>>>>>> Few quick comments on skimming the code: >>>>>>>>> >>>>>>>>> The complicated nested #ifdef for __builtin_expect could be >simplified to >>>>>>>>> #if defined(__GNUC__)&& (__GNUC__> 2 || __GNUC_MINOR__> >95) >>>>>>>>> >>>>>>>>> PyCustomSlots_Check should be called PyCustomSlots_CheckExact, >surely? >>>>>>>>> And given that, how can this code work if someone does >subclass this >>>>>>>>> metaclass? >>>>>>>> >>>>>>>> I think we should provide a wrapper for PyType_Ready, which >just >>>>>>>> copies the pointer to the table and the count directly into the >>>>>>>> subclass. If a user then wishes to add stuff, the user can >allocate a >>>>>>>> new memory region dynamically, memcpy the base class' stuff in >there, >>>>>>>> and append some entries. >>>>>>> >>>>>>> Maybe we should also allow each custom type to set a >deallocator, >>>>>>> since they are then heap types which can go out of scope. The >>>>>>> metaclass can then call this deallocator to deallocate the >table. >>>>>> >>>>>> Custom types are plain old Python objects, they can use >tp_dealloc. >>>>>> >>>>> If I set etp_custom_slots to something allocated on the heap, then >the >>>>> (shared) metaclass would have to deallocate it. The tp_dealloc of >the >>>>> type itself would be called for its instances (which can be used >to >>>>> deallocate dynamically allocated memory in the objects if you use >a >>>>> custom slot "pointer offset"). >>>> >>>> Oh, I see. Right, the natural way to handle this would be have each >>>> user define their own metaclass with the behavior they want. >Another >>>> argument for supporting multiple metaclasses simultaneously I >guess... >>>> >>>> - N >>>> _______________________________________________ >>>> cython-devel mailing list >>>> cython-devel@python.org >>>> http://mail.python.org/mailman/listinfo/cython-devel >>> >>> That bludgeons your constant time type check. >> >> Not if you steal a flag, like the interpreter already does with >> Py_TPFLAGS_INT_SUBCLASS, Py_TPFLAGS_STRING_SUBCLASS, etc. I was >> referring to that argument I made earlier :-) >> >>> It's easier to just >>> reserve an extra slot for a deallocator pointer :) It would probably >>> be set to NULL in the common case anyway, since you allocate your >>> slots statically. > >Subclassing: Note that even if all types has to have a PyHeapTypeObject > >structure, they are still statically allocated! So for statically >created subclasses (which should be the majority of the cases), there's > >not going to be any deallocator... > >I agree that there should be a PyExtensibleType_Ready. To keep >allocating statically I propose that the subclass should leave some >room >open for slots from the superclass: > >PyCustomSlot subclass_custom_slots[10] = { > {SLOT_C, foo}, {SLOT_D, BAR}, {0,0}, ... >} > >Then, fill in etp_count=2, etp_custom_slots=subclass_custom_slots, and >then call PyExtensibleType_Ready(&Subclass_Type, 10); i.e., the number >of total elements in etp_custom_slots is passed in. > >One should always leave more room than one thinks one needs if the >superclass is from another library... > >Then, inheritance happens according to the following rules: > > - Slots are inherited from superclass > - Slots in subclass with same ID overwrites superclass > - Slots from superclass are put before slots from subclass > - Exception raised if the number of final slots is larger than the >limit passed in to PyExtensibleType_Ready. > >(Whenever this is not sufficient, you can always manually munge the >table after PyExtensibleType_Ready.) > >Question: How to deal with possible flag bits in the ID? > >Three approaches: > >a) Forget about the flags-in-ID idea; if you want flags, stick them in >the data > > b) Embed a seperate variable for flags in every PyCustomSlot > > c) Standardize on a *hard* requirement on the bottom 8 bits being >flags while the top 24 bits indicate incompatible slots; so for the >purposes of inheritance, 0x12345601 would overwrite 0x12345600. > >To me, b) is OK, but the 32 bit ID space is already so ridiculously >huge >that c) is a "why not"? -1 on a), it'd be rather tedious if the payload > I guess the sane thing to do is make the custom slot (id, flags, data); and have id and flags be 32 bits on all platforms. Otherwise 32 bits are wasted to padding on 64 bit platforms anyway. Is there a type one can safely use everywhere to get a 32 bit unsigned int? Does MSVC support stdint.h? Dag >is an offset to the PyObject*. > >Subclassing in heap-allocated types (subclasses Python side): It'd >certainly be nice to completely ignore this for now and require making >a >sub-metaclass to support this (e.g., have tp_new parse some >__customslots__ attribute in the class dict). > >Hijacking a TP_FLAG: We could make the branch for a direct hit on >metaclass comparison likely(), so that the branch checking tp_base on >the metaclass unlikely(), which with branch prediction I think makes it > >very likely that there's no penalty for allowing sub-metaclasses (when >you don't use them -- when you do, there's a slight penalty). > >But here's another great argument in favour of a TP_FLAG bit: Consumers > >would then not need to import the metaclass or contain its definition >(which is really only around in case the user imports the consumer >before the provider...). This would make the header file the consumers >need to bundle much lighter. So I think I'm +1. > >At any rate, I would like the metaclass rendezvous to keep happening >just because it's less confusing if "extensibletype is extensibletype" >in general. > >Anyway, the metaclass checking is a nice fallback if CPython uses all >their flag bits. > >Dag >_______________________________________________ >cython-devel mailing list >cython-devel@python.org >http://mail.python.org/mailman/listinfo/cython-devel -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. _______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
