Re: [Python-3000] PyUnicodeObject implementation

2008-09-26 Thread Guido van Rossum
On Fri, Sep 26, 2008 at 7:42 AM, M.-A. Lemburg <[EMAIL PROTECTED]> wrote: > On 2008-09-26 12:08, Antoine Pitrou wrote: >> So what would be the outcome of this discussion, and should a decision (and >> which one) be taken? > > I'm still -1 on changing Unicode objects to PyVarObjects for the > reason

Re: [Python-3000] PyUnicodeObject implementation

2008-09-26 Thread M.-A. Lemburg
On 2008-09-26 12:08, Antoine Pitrou wrote: > So what would be the outcome of this discussion, and should a decision (and > which one) be taken? I'm still -1 on changing Unicode objects to PyVarObjects for the reasons already stated in various postings on this thread and on the ticket. I'd much ra

Re: [Python-3000] PyUnicodeObject implementation

2008-09-26 Thread Antoine Pitrou
So what would be the outcome of this discussion, and should a decision (and which one) be taken? Regards Antoine. ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python

Re: [Python-3000] PyUnicodeObject implementation

2008-09-09 Thread Martin v. Löwis
> Coming back to this: Why is this done anyway? Can't the new instance of the > unicode-subtype just steal the buffer pointer of the already allocated > unicode > object? Only if the refcount of the tmp object is 1. But then, yes, it could. You then also need to change unicode_dealloc, to only

Re: [Python-3000] PyUnicodeObject implementation

2008-09-09 Thread Antoine Pitrou
Hello, M.-A. Lemburg egenix.com> writes: > > It turned out that the patch only provides a marginal performance > improvement, so the perceived main argument for the PyVarObject > implementation doesn't turn out to be a real advantage. Uh, while the results are not always overwhelming, they are

Re: [Python-3000] PyUnicodeObject implementation

2008-09-09 Thread Stefan Behnel
Stefan Behnel wrote: > Antoine Pitrou wrote: >> Stefan Behnel behnel.de> writes: >>> From a Cython perspective, I find the lack of efficient subclassing after >>> such a change particularly striking. >> what do you call "efficient subclassing"? if you look at the current >> implementation of unico

Re: [Python-3000] PyUnicodeObject implementation

2008-09-09 Thread M.-A. Lemburg
Before jumping to conclusions, please read the discussion on the patch ticket: http://bugs.python.org/issue1943 It turned out that the patch only provides a marginal performance improvement, so the perceived main argument for the PyVarObject implementation doesn't turn out to be a real advant

Re: [Python-3000] PyUnicodeObject implementation

2008-09-09 Thread Stefan Behnel
Stefan Behnel wrote: > (s + s->tp_basicsize) I (obviously) meant (s + Py_TYPE(s)->tp_basicsize) so the impact is another bit bigger. Stefan ___ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-

Re: [Python-3000] PyUnicodeObject implementation

2008-09-09 Thread Stefan Behnel
Greg Ewing canterbury.ac.nz> writes: > > We create a new struct for the type that contains the parent-struct > > as first field, and then we add the new attributes of the new type behind > > that. > > I seem to remember there's a field in the type called tp_basicsize > that's meant to indicate ho

Re: [Python-3000] PyUnicodeObject implementation

2008-09-08 Thread Stefan Behnel
Martin v. Löwis wrote: >> I wouldn't mind letting Cython special case subtypes of str (or unicode in >> Py3) *somehow*, as long as this "somewhow" proves to be a viable solution >> that >> only applies to exactly those types *and* can be done realiably for subtypes >> of subtypes. I'm just not awa

Re: [Python-3000] PyUnicodeObject implementation

2008-09-08 Thread Martin v. Löwis
> How about putting the variable sized data _before_ the struct? That won't work for container objects (such as tuples); they already have the GC structure before the PyObject, whose size and layout is opaque to the objects. Regards, Martin ___ Python-3

Re: [Python-3000] PyUnicodeObject implementation

2008-09-08 Thread Greg Ewing
Stefan Behnel wrote: We create a new struct for the type that contains the parent-struct as first field, and then we add the new attributes of the new type behind that. I seem to remember there's a field in the type called tp_basicsize that's meant to indicate how big the base part of the stru

Re: [Python-3000] PyUnicodeObject implementation

2008-09-08 Thread Eric Smith
Martin v. Löwis wrote: I wouldn't mind letting Cython special case subtypes of str (or unicode in Py3) *somehow*, as long as this "somewhow" proves to be a viable solution that only applies to exactly those types *and* can be done realiably for subtypes of subtypes. I'm just not aware of such a s

Re: [Python-3000] PyUnicodeObject implementation

2008-09-08 Thread Martin v. Löwis
> base_address + base_address[ob_len]*elem_size - more_fields_size The subtraction is wrong, of course - it's still an addition. I was just confused by tp_dictoffset being negative in that case; the sign is but a mere flag in that case, and the offset is still positive. Regards, Martin

Re: [Python-3000] PyUnicodeObject implementation

2008-09-08 Thread Martin v. Löwis
> I wouldn't mind letting Cython special case subtypes of str (or unicode in > Py3) *somehow*, as long as this "somewhow" proves to be a viable solution that > only applies to exactly those types *and* can be done realiably for subtypes > of subtypes. I'm just not aware of such a solution. As peop

Re: [Python-3000] PyUnicodeObject implementation

2008-09-08 Thread Antoine Pitrou
Stefan Behnel behnel.de> writes: > > cdef class MyListSubType(PyListObject): > cdef int some_additional_int_field > cdef my_struct* some_struct > > def __init__(self): > self.some_struct = get_the_struct_pointer(...) > self.some_additional_int_

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Stefan Behnel
Martin v. Löwis wrote: >>> Can you explain this a bit more? I presume you're talking about >>> subclassing in C >> Yes, I mentioned Cython above. > > Can you please still elaborate? I have never used Cython before Should have been clearer, sorry. C-level subtyping in Cython/Pyrex works as follows

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Martin v. Löwis
> Given that you can, today, subclass str in Python, without wasting an > extra 4/8 bytes of memory, or adding anything new to the class object, > why wouldn't anyone who really wanted to make a hypothetical optimized > subclass just use the same mechanism (putting your additional data > *after* th

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Martin v. Löwis
>> Can you explain this a bit more? I presume you're talking about >> subclassing in C > > Yes, I mentioned Cython above. Can you please still elaborate? I have never used Cython before, but if it cannot efficiently subclass str, isn't that a bug in Cython? >> I do note that the mechanisms that

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread James Y Knight
On Sep 7, 2008, at 6:55 PM, Guido van Rossum wrote: One possibility that occurs to me is to use a PyVarObject variant that allocates space for an additional void pointer before the variable sized section of the object. The builtin type would leave that pointer NULL, but subtypes could perfor

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Guido van Rossum
On Sun, Sep 7, 2008 at 2:23 PM, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: >> All in all, given the advantage (half the number of allocations) of >> the proposal I think there would have to be *very* good arguments >> against before we reject this outright. I'd like to unders

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Nick Coghlan
Guido van Rossum wrote: > All in all, given the advantage (half the number of allocations) of > the proposal I think there would have to be *very* good arguments > against before we reject this outright. I'd like to understand > Marc-Andre's reasons too. As Stefan notes, because of the frequency w

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Stefan Behnel
Hi, Guido van Rossum wrote: > On Sun, Sep 7, 2008 at 12:15 AM, Stefan Behnel <[EMAIL PROTECTED]> wrote: >> Antoine Pitrou wrote: >>> Also note that Marc-André Lemburg (one of the authors of the unicode >>> implementation) is opposed to that change. See the discussion in the bug >>> tracker >>> is

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Stefan Behnel
Antoine Pitrou wrote: > Stefan Behnel behnel.de> writes: >> From a Cython perspective, I find the lack of efficient subclassing after >> such >> a change particularly striking. That seriously bit me in Py2 when I tried >> making XML text content a bit more intelligent in lxml (i.e. make it rememb

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Guido van Rossum
On Sun, Sep 7, 2008 at 12:15 AM, Stefan Behnel <[EMAIL PROTECTED]> wrote: > Antoine Pitrou wrote: >> Also note that Marc-André Lemburg (one of the authors of the unicode >> implementation) is opposed to that change. See the discussion in the bug >> tracker >> issue for the details. > > From a Cyth

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Antoine Pitrou
Stefan Behnel behnel.de> writes: > > From a Cython perspective, I find the lack of efficient subclassing after such > a change particularly striking. That seriously bit me in Py2 when I tried > making XML text content a bit more intelligent in lxml (i.e. make it remember > what XML element it ori

Re: [Python-3000] PyUnicodeObject implementation

2008-09-07 Thread Stefan Behnel
Antoine Pitrou wrote: > Also note that Marc-André Lemburg (one of the authors of the unicode > implementation) is opposed to that change. See the discussion in the bug > tracker > issue for the details. From a Cython perspective, I find the lack of efficient subclassing after such a change partic

Re: [Python-3000] PyUnicodeObject implementation

2008-09-06 Thread Antoine Pitrou
Jeremy Kloth gmail.com> writes: > > I don't know if this is too late to do before the final release, but > shouldn't > the implementation of PyUnicodeObject be updated to match the much more > efficient old PyStringObject layout? I mean eliminating the double malloc > that is currently requi

Re: [Python-3000] PyUnicodeObject implementation

2008-09-05 Thread Guido van Rossum
This is an excellent idea that fell by the wayside. Since it is a major coding project there's no way it can be done for 3.0 -- the risk of introducing new instabilities or leaks is just too high. We *might* be able to get it in for 3.0.1, if the code is reviewed really well. Though it might be saf

[Python-3000] PyUnicodeObject implementation

2008-09-05 Thread Jeremy Kloth
I don't know if this is too late to do before the final release, but shouldn't the implementation of PyUnicodeObject be updated to match the much more efficient old PyStringObject layout? I mean eliminating the double malloc that is currently required for each unicode string. PyStringObject is