Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
Bob Ippolito wrote: It doesn't for reasons I care not to explain in depth, again. Search the pythonmac-sig archives for longer explanations. The gist is that you specifically do not want to link directly to the framework at all when building extensions. Because an Apple-built extension then may pick up a user-installed Python? Why can this problem not be solved by adding -F options, as Jack Jansen proposed? This is not the wrong way to do it. I'm not convinced. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
Andrew McNamara wrote: There's a bunch of jobs we (CSV module maintainers) have been putting off - attached is a list (in no particular order): * unicode support (this will probably uglify the code considerably). Can you please elaborate on that? What needs to be done, and how is that going to be done? It might be possible to avoid considerable uglification. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
Martin v. Löwis wrote: Andrew McNamara wrote: There's a bunch of jobs we (CSV module maintainers) have been putting off - attached is a list (in no particular order): * unicode support (this will probably uglify the code considerably). Can you please elaborate on that? What needs to be done, and how is that going to be done? It might be possible to avoid considerable uglification. Indeed. The trick is to convert to Unicode early and to use Unicode literals instead of string literals in the code. Note that the only real-life Unicode format in use is UTF-16 (with BOM mark) written by Excel. Note that there's no standard for specifying the encoding in CSV files, so this is also the only feasable format. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 05 2005) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On 5-jan-05, at 9:33, Martin v. Löwis wrote: Bob Ippolito wrote: It doesn't for reasons I care not to explain in depth, again. Search the pythonmac-sig archives for longer explanations. The gist is that you specifically do not want to link directly to the framework at all when building extensions. Because an Apple-built extension then may pick up a user-installed Python? Why can this problem not be solved by adding -F options, as Jack Jansen proposed? It gets worse when you have a user-installed python 2.3 and a user-installed python 2.4. Those will be both be installed as /Library/Frameworks/Python.framework. This means that you cannot use the -F flag to select which one you want to link to, '-framework Python' will only link to the python that was installed the latest. This is an issue on Mac OS X 10.2. Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
>> Andrew McNamara wrote: >>> There's a bunch of jobs we (CSV module maintainers) have been putting >>> off - attached is a list (in no particular order): >>> * unicode support (this will probably uglify the code considerably). >> >Martin v. Löwis wrote: >> Can you please elaborate on that? What needs to be done, and how is >> that going to be done? It might be possible to avoid considerable >> uglification. I'm not altogether sure there. The parsing state machine is all written in C, and deals with signed chars - I expect we'll need two versions of that (or one version that's compiled twice using pre-processor macros). Quite a large job. Suggestions gratefully received. M.-A. Lemburg wrote: >Indeed. The trick is to convert to Unicode early and to use Unicode >literals instead of string literals in the code. Yes, although it would be nice to also retain the 8-bit versions as well. >Note that the only real-life Unicode format in use is UTF-16 >(with BOM mark) written by Excel. Note that there's no standard >for specifying the encoding in CSV files, so this is also the only >feasable format. Yes - that's part of the problem I hadn't really thought about yet - the csv module currently interacts directly with files as iterators, but it's clear that we'll need to decode as we go. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
Andrew McNamara wrote: Andrew McNamara wrote: There's a bunch of jobs we (CSV module maintainers) have been putting off - attached is a list (in no particular order): * unicode support (this will probably uglify the code considerably). Martin v. Löwis wrote: Can you please elaborate on that? What needs to be done, and how is that going to be done? It might be possible to avoid considerable uglification. I'm not altogether sure there. The parsing state machine is all written in C, and deals with signed chars - I expect we'll need two versions of that (or one version that's compiled twice using pre-processor macros). Quite a large job. Suggestions gratefully received. M.-A. Lemburg wrote: Indeed. The trick is to convert to Unicode early and to use Unicode literals instead of string literals in the code. Yes, although it would be nice to also retain the 8-bit versions as well. You can do so by using latin-1 as default encoding. Works great ! Note that the only real-life Unicode format in use is UTF-16 (with BOM mark) written by Excel. Note that there's no standard for specifying the encoding in CSV files, so this is also the only feasable format. Yes - that's part of the problem I hadn't really thought about yet - the csv module currently interacts directly with files as iterators, but it's clear that we'll need to decode as we go. Depends on your needs: CSV files tend to be small enough to do the decoding in one call in memory. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 05 2005) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
>> Yes, although it would be nice to also retain the 8-bit versions as well. > >You can do so by using latin-1 as default encoding. Works great ! Yep, although that means we wear the cost of decoding and encoding for all 8 bit input. What does the _sre.c code do? >Depends on your needs: CSV files tend to be small enough >to do the decoding in one call in memory. We are routinely dealing with multi-gigabyte csv files - which is why the original 2001 vintage csv module was written as a C state machine. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
Andrew McNamara wrote: Yes, although it would be nice to also retain the 8-bit versions as well. You can do so by using latin-1 as default encoding. Works great ! Yep, although that means we wear the cost of decoding and encoding for all 8 bit input. Right, but it makes the code very clean and straight forward. Again, it depends on what you need. If performance is critical then you probably need a C version written using the same trick as _sre.c... What does the _sre.c code do? It comes in two versions: one for 8-bit the other for Unicode. Depends on your needs: CSV files tend to be small enough to do the decoding in one call in memory. We are routinely dealing with multi-gigabyte csv files - which is why the original 2001 vintage csv module was written as a C state machine. I see, but are you sure that the typical Python user will have the same requirements to make it worth the effort (and complexity) ? I've written a few CSV parsers and writers myself over the years and the requirements were different every time, in terms of being flexible in the parsing phase, the interfaces and the performance needs. Haven't yet found a one fits all solution and don't really expect to any more :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 05 2005) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
>> Yep, although that means we wear the cost of decoding and encoding for >> all 8 bit input. > >Right, but it makes the code very clean and straight forward. I agree it makes for a very clean solution, and 99% of the time I'd chose that option. >Again, it depends on what you need. If performance is critical >then you probably need a C version written using the same trick >as _sre.c... > >> What does the _sre.c code do? > >It comes in two versions: one for 8-bit the other for Unicode. That's what I thought. I think the motivations here are similar to those that drove the _sre developers. >> We are routinely dealing with multi-gigabyte csv files - which is why the >> original 2001 vintage csv module was written as a C state machine. > >I see, but are you sure that the typical Python user will have >the same requirements to make it worth the effort (and >complexity) ? This is open source, so I scratch my own itch (and that of my employers) - we need fast csv parsing more than we need unicode... 8-) Okay, assuming we go the "produce two versions via evil macro tricks" path, it's still not quite the same situation as _sre.c, which only has to deal with the internal unicode representation. One way to approach this would be to add an "encoding" keyword argument to the readers and writers. If given, the parser would decode the input stream to the internal representation before passing it through the unicode state machine, which would yield tuples of unicode objects. That leaves us with a bit of a problem where the source is already unicode (eg, a list of unicode strings)... hmm. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: [Csv] csv module TODO list
>Also, review comments from Neal Norwitz, 22 Mar 2003 (some of these should >already have been addressed): I should apologise to Neal here for not replying to him at the time. Okay, going though the issues Neal raised... >* remove TODO comment at top of file--it's empty Was fixed. >* is CSV going to be maintained outside the python tree? > If not, remove the 2.2 compatibility macros for: > PyDoc_STR, PyDoc_STRVAR, PyMODINIT_FUNC, etc. Does anyone thing we should continue to maintain this 2.2 compatibility? >* inline the following functions since they are used only in one place >get_string, set_string, get_nullchar_as_None, set_nullchar_as_None, >join_reset (maybe) It was done that way as I felt we would be adding more getters and setters to the dialect object in future. >* rather than use PyErr_BadArgument, should you use assert? >(first example, Dialect_set_quoting, line 218) You mean C assert()? I don't think I'm really following you here - where would the type of the object be checked in a way the user could recover from? >* is it necessary to have Dialect_methods, can you use 0 for tp_methods? I was assuming I would need to add methods at some point (in fact, I did have methods, but removed them). >* remove commented out code (PyMem_DEL) on line 261 >Have you used valgrind on the test to find memory overwrites/leaks? No, valgrind wasn't used. >* PyString_AsString()[0] on line 331 could return NULL in which case >you are dereferencing a NULL pointer Was fixed. >* note sure why there are casts on 0 pointers >lines 383-393, 733-743, 1144-1154, 1164-1165 To make it easier when the time comes to add one of those members. >* Reader_getiter() can be removed and use PyObject_SelfIter() Okay, wasn't aware of PyObject_SelfIter - will fix. >* I think you need PyErr_NoMemory() before returning on line 768, 1178 The examples I looked at in the Python core didn't do this - are you sure? (now lines 832 and 1280). >* is PyString_AsString(self->dialect->lineterminator) on line 994 >guaranteed not to return NULL? If not, it could crash by >passing to memmove. >* PyString_AsString() can return NULL on line 1048 and 1063, >the result is passed to join_append() Looking at the PyString_AsString implementation, it looks safe (we ensure it's really a string elsewhere)? >* iteratable should be iterable? (line 1088) Sorry, I don't know what you're getting at here? (now line 1162). >* why doesn't csv_writerows() have a docstring? csv_writerow does Was fixed. >* any PyUnicode_* methods should be protected with #ifdef Py_USING_UNICODE Was fixed. >* csv_unregister_dialect, csv_get_dialect could use METH_O >so you don't need to use PyArg_ParseTuple Was fixed. >* in init_csv, recommend using >PyModule_AddIntConstant and PyModule_AddStringConstant >where appropriate Was fixed. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
On 2005 Jan 05, at 04:42, Barry Warsaw wrote: On Tue, 2005-01-04 at 18:01, Jack Jansen wrote: But I'm more worried about losing the other information in an unbound method, specifically im_class. I would guess that info is useful to class browsers and such, or are there other ways to get at that? That would be my worry too. OTOH, we have function attributes now, so why couldn't we just stuff the class on the function's im_class attribute? Who'd be the wiser? (Could the same be done for im_self and im_func for backwards compatibility?) Hmmm, seems to me we'd need copies of the function object for this purpose: def f(*a): pass class C(object): pass class D(object): pass C.f = D.f = f If now we want C.f.im_class to differ from D.f.im_class then we need f to get copied implicitly when it's assigned to C.f (or, of course, when C.f is accessed... but THAT might be substantial overhead). OK, I guess, as long as we don't expect any further attribute setting on f to affect C.f or D.f (and I don't know of any real use case where that would be needed). Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On 2005 Jan 05, at 00:06, Jack Jansen wrote:
...
We've solved this issue for the trunk and we can solve it for 2.4.1:
if MACOSX_DEPLOYMENT_TARGET isn't set and we're on 10.3 we force it to
10.3. Moreover, when it is 10.3 or higher (possibly after being
forced) we use the dynamic_lookup way of linking
Not having followed Python/Mac developments closely (my fault, sigh), I
would like to understand what this would imply for the forthcoming 10.4
("Tiger") release of MacOS -- and that in turn depends, I assume, on
what Python release will come with it. Anybody who's under
nondisclosure should of course keep mum, but can somebody help e.g. by
telling me what Python is included in the current "development
previews" versions of Tiger? I'm not gonna spend $500 to become a
highly-ranked enough "apple developer" to get those previews.
Considering Apple's habitual timings, I'm sort of resigned to us being
stuck with 2.3 for Tiger, but I would at least hope they'd get as late
a 2.3.* as they can. So, assuming Tiger's Python is going to be, say,
2.3.4 or 2.3.5, would the change you're proposing make it MORE
attractive to Apple to go for 2.3.5, LESS so, or is it indifferent from
their POV...?
Thanks in advance for any help in getting the tradeoffs about this
clearer in my mind!
Alex
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On 5-jan-05, at 12:28, Alex Martelli wrote:
On 2005 Jan 05, at 00:06, Jack Jansen wrote:
...
We've solved this issue for the trunk and we can solve it for 2.4.1:
if MACOSX_DEPLOYMENT_TARGET isn't set and we're on 10.3 we force it
to 10.3. Moreover, when it is 10.3 or higher (possibly after being
forced) we use the dynamic_lookup way of linking
Not having followed Python/Mac developments closely (my fault, sigh),
I would like to understand what this would imply for the forthcoming
10.4 ("Tiger") release of MacOS -- and that in turn depends, I assume,
on what Python release will come with it. Anybody who's under
nondisclosure should of course keep mum, but can somebody help e.g. by
telling me what Python is included in the current "development
previews" versions of Tiger?
The Tiger that was released at WWDC included a patched version of
Python 2.3.3. See:
http://www.opensource.apple.com/darwinsource/WWDC2004/.
Ronald
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please help complete the AST branch
On 2005 Jan 04, at 22:11, Brett C. wrote: ... Speaking for myself, I have a burning interest in the AST branch (though I can't seem to get it correctly downloaded so far, I guess it's just my usual CVS-clumsiness and I'll soon find out what I'm doing wrong & fix it) See http://www.python.org/dev/devfaq.html#how-can-i-check-out-a-tagged- branch on how to do a checkout of a tagged branch. Done! Believe it or not, I _had_ already tried following those very instructions -- and I kept omitting the word 'python' at the end and/or mispelling the tag as ast_branch (while it wants a dash, NOT an underscore...). I guess having the instructions recommended to me again prompted me to doublecheck character by character extracarefully, so, thanks!-) Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: csv module TODO list
>Quite a while ago I posted some material to the csv-list about >problems using the csv module on Unix-style colon-separated files -- >it just doesn't deal properly with backslash escaping and is quite >useless for this kind of file. I seem to recall the general view was >that it wasn't intended for this kind of thing -- only the sort of csv >that Microsoft Excel outputs/inputs, but if I am mistaken about this, >perhaps fixing this issue might be put on the TODO-list? I'll be happy >to re-send or summarize the relevant emails, if needed. I think a related issue was included in my TODO list: >* Address or document Francis Avila's issues as mentioned in this posting: > >http://www.google.com.au/groups?selm=vsb89q1d3n5qb1%40corp.supernews.com -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On 2005 Jan 05, at 12:40, Ronald Oussoren wrote: ... The Tiger that was released at WWDC included a patched version of Python 2.3.3. See: http://www.opensource.apple.com/darwinsource/WWDC2004/. Thanks! So, since WWDC was on June 28 and 2.3.4 had been released on May 27, we get some first sense of the speed or lack thereof of 2.3.x releases' entrance in Tiger's previews... Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > Bob Ippolito wrote: >> It doesn't for reasons I care not to explain in depth, again. >> Search the pythonmac-sig archives for longer explanations. The >> gist is that you specifically do not want to link directly to the >> framework at all when building extensions. > > Because an Apple-built extension then may pick up a user-installed > Python? Why can this problem not be solved by adding -F options, > as Jack Jansen proposed? > >> This is not the wrong way to do it. > > I'm not convinced. Martin, can you please believe that Jack, Bob, Ronald et al know what they are talking about here? Cheers, mwh -- Q: Isn't it okay to just read Slashdot for the links? A: No. Reading Slashdot for the links is like having "just one hit" off the crack pipe. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On Jan 5, 2005, at 3:33 AM, Martin v. Löwis wrote: Bob Ippolito wrote: It doesn't for reasons I care not to explain in depth, again. Search the pythonmac-sig archives for longer explanations. The gist is that you specifically do not want to link directly to the framework at all when building extensions. Because an Apple-built extension then may pick up a user-installed Python? Why can this problem not be solved by adding -F options, as Jack Jansen proposed? This is not the wrong way to do it. I'm not convinced. Then you haven't done the appropriate research by searching pythonmac-sig. Do you even own a Mac? -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
On Tue, 2005-01-04 at 22:12 -0500, Bob Ippolito wrote: > If you have a class hierarchy where this is a problem, it's probably > pretty fragile to begin with, and you should think about making it > simpler. I agree with James's rant almost entirely, but I like super() anyway. I think it is an indication not of a new weakness of super(), but of a long-standing weakness of __init__. One approach I have taken in order to avoid copiously over-documenting every super() using class is to decouple different phases of initialization by making __init__ as simple as possible (setting a few attributes, resisting the temptation to calculate things), and then providing class methods like '.fromString' or '.forUnserialize' that create instances that have been completely constructed for a particular purpose. That way the signatures are much more likely to line up across inheritance hierarchies. Perhaps this should be a suggested "best practice" when using super() as well? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
On Wed, 2005-01-05 at 12:11 +0100, Alex Martelli wrote: > Hmmm, seems to me we'd need copies of the function object for this > purpose: For the stated use-case of serialization, only one copy would be necessary, and besides - even *I* don't use idioms as weird as the one you are suggesting very often ;). I think it would be reasonable to assign im_class only to functions defined in class scope. The only serialization that would break in that case is if your example had a 'del f' at the end. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
Hi Jim, On Tue, Jan 04, 2005 at 02:44:43PM -0500, Jim Fulton wrote: > >Actually, unbound builtin methods are a different type than bound > >builtin methods: > > Of course, but conceptually they are similar. You would still > encounter the concept if you got an unbound builtin method. There are no such things as unbound builtin methods: >>> list.append is list.__dict__['append'] True In other words 'list.append' just returns exactly the same object as stored in the list type's dict. Guido's proposal is to make Python methods behave in the same way. Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please help complete the AST branch
I'd like to help here on the AST branch, if it's not too late. (Especially I'm interested with the generator expression part.) If I want to volunteer, do I just begin to work with it? Or do I need to read something or discuss with someone? Thanks. Jiwon. On Mon, 3 Jan 2005 23:03:33 -0500, Jeremy Hylton <[EMAIL PROTECTED]> wrote: > On Mon, 03 Jan 2005 18:02:52 -0800, Brett C. <[EMAIL PROTECTED]> wrote: > > Plus there is the running tradition of sprinting on the AST branch at > > PyCon. I > > was planning on shedding my bug fixing drive at PyCon this year and > > sprinting > > with (hopefully) Jeremy, Neal, Tim, and Neil on the AST branch as a prep for > > working on it afterwards for my class credit. > > I'd like to sprint on it before PyCon; we'll have to see what my > schedule allows. > > > If anyone would like to see the current code, check out ast-branch from CVS > > (read the dev FAQ on how to check out a branch from CVS). Read > > Python/compile.txt for an overview of how the thing works and such. > > > > It will get done, just don't push for a 2.5 release within a month. =) > > I think the branch is in an awkward state, because of the new features > added to Python 2.4 after the AST branch work ceased. The ast branch > doesn't handle generator expressions or decorators; extending the ast > to support them would be a good first step. > > There are also the simple logistical questions of integrating changes. > Since most of the AST branch changes are confined to a few files, I > suspect the best thing to do is to merge all the changes from the head > except for compile.c. I haven't done a major CVS branch integrate in > at least nine months; if someone feels more comfortable with that, it > would also be a good step. > > Perhaps interested parties should take up the discussion on the > compiler-sig. I think we can recover the state of last May's effort > pretty quickly, and I can help outline the remaining work even if I > can't help much. (Although I hope I can help, too.) > > Jeremy > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/seojiwon%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
Hi Guido, On Tue, Jan 04, 2005 at 10:28:03AM -0800, Guido van Rossum wrote: > Let's get rid of unbound methods. Is there any other use case for 'C.x' not returning the same as 'appropriate_super_class_of_C.__dict__["x"]' ? I guess it's too late now but it would have been nice if user-defined __get__() methods had the more obvious signature (self, instance) instead of (self, instance_or_None, cls=None). Given the amount of potential breakage people already pointed out I guess it is not reasonable to change that. Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please help complete the AST branch
On Thu, 6 Jan 2005 01:32:44 +0900, Jiwon Seo <[EMAIL PROTECTED]> wrote: > I'd like to help here on the AST branch, if it's not too late. > (Especially I'm interested with the generator expression part.) Great! It's not too late. > If I want to volunteer, do I just begin to work with it? Or do I need > to read something or discuss with someone? The file Python/compile.txt on the ast-branch has a brief overview of the project: http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/Attic/compile.txt?rev=1.1.2.8&only_with_tag=ast-branch&view=auto Jeremy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
Alex Martelli wrote: def f(*a): pass class C(object): pass class D(object): pass C.f = D.f = f If now we want C.f.im_class to differ from D.f.im_class then we need f to get copied implicitly when it's assigned to C.f (or, of course, when C.f is accessed... but THAT might be substantial overhead). OK, I guess, as long as we don't expect any further attribute setting on f to affect C.f or D.f (and I don't know of any real use case where that would be needed). You'd have to do a copy anyway, because f() is still a module-level callable entity. I also agree with Glyph that im_class should only really be set in the case of methods defined within the class block. Also, interestingly, removing unbound methods makes another thing possible. class A(object): def foo(self): pass class B(object): foo = A.foo class C(object): pass C.foo = A.foo I'd really like to avoid making copies of functions for the sake of reload() and edit-and-continue functionality. Currently we can track down everything that has a reference to foo, and replace it with newfoo. With copies, this would more difficult. Thanks, -Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] ast branch pragmatics
The existing ast-branch mostly works, but it does not include most of the new features of Python 2.4. There is a substantial integration effort, perhaps easy for someone who does a lot of CVS branch merges. (In particular, the head has already been merged to this branch once.) I think it would be easier to create a new branch from the current head, integrate the small number of changed files from ast-branch, and work with that branch instead. The idea is that it's an end-run around doing an automatic CVS merge and relying on someone to manually merge the changes. At the same time, since there is a groundswell of support for finishing the AST work, I'd like to propose that we stop making compiler / bytecode changes until it is done. Every change to compile.c or the bytecode ends up creating a new incompatibilty that needs to be merged. If these two plans sound good, I'll get started on the new branch. Jeremy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: super() harmful?
I'm not sure why super got dragged into this, but... On Jan 4, 2005, at 9:02 PM, Guido van Rossum wrote: I think that James Y Knight's page misrepresents the issue. Quoting: But __init__ *is* special, in that it is okay for a subclass __init__ (or __new__) to have a different signature than the base class __init__; this is not true for other methods. If you change a regular method's signature, you would break Liskov substitutability (i.e., your subclass instance wouldn't be acceptable where a base class instance would be acceptable). You're right, some issues do apply to __init__ alone. However, two important ones do not: The issue of mixing super() and explicit calls to the superclass's method occur with any method. (Thus making it difficult/impossible for a framework to convert to using super without breaking client code that subclasses). Adding optional arguments to one branch of the inheritance tree, but not another, or adding different optional args in both branches. (breaks unless you always pass optional args as keywordargs, and all methods take **kwargs and pass that on to super). Super is intended for use that are designed with method cooperation in mind, so I agree with the best practices in James's Conclusion: [[omitted]] But that's not the same as calling it harmful. :-( The 'harmfulness' comes from people being confused by, and misusing super, because it is so very very easy to do so, and so very hard to use correctly. From what I can tell, it is mostly used incorrectly. *Especially* uses in __init__ or __new__. Many people seem to use super in their __init__ methods thinking that it'll magically improve something (like perhaps making multiple inheritance trees that include their class work better), only to just cause a different set of problems for multiple inheritance trees, instead, because they don't realize they need to follow those recommendations. Here's another page that says much the same thing, but from the viewpoint of recommending the use of super and showing you all the hoops to use it right: http://wiki.osafoundation.org/bin/view/Chandler/UsingSuper James PS, I wrote that page last pycon but never got around to finishing it up and therefore never really publically announced it. But I told some people about it and then they kept asking me for the URL so I linked to it, and well, then google found it of course, so I guess it's public now. ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
On Wed, 2005-01-05 at 10:37, Glyph Lefkowitz wrote: > One approach I have taken in order to avoid copiously over-documenting > every super() using class is to decouple different phases of > initialization by making __init__ as simple as possible (setting a few > attributes, resisting the temptation to calculate things), and then > providing class methods like '.fromString' or '.forUnserialize' that > create instances that have been completely constructed for a particular > purpose. That way the signatures are much more likely to line up across > inheritance hierarchies. Perhaps this should be a suggested "best > practice" when using super() as well? Yep, I've done the same thing. It's definitely a good practice. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
On Wed, 2005-01-05 at 10:41, Glyph Lefkowitz wrote: > I think it would be reasonable to assign im_class only to functions > defined in class scope. The only serialization that would break in that > case is if your example had a 'del f' at the end. +1. If you're doing something funkier, then you can set that attribute yourself. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Let's get rid of unbound methods
> duck typing? That's the Australian pronunciation of "duct taping". ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Re: [Csv] csv module TODO list
Skip Montanaro wrote: > Andrew> There's a bunch of jobs we (CSV module > maintainers) have been > Andrew> putting off - attached is a list (in no particular order): > > ... > > In addition, it occurred to me this evening that there's > functionality in the csv module I don't think anybody uses. > ... > I'm also not aware that anyone really uses the Sniffer class, > though it does provide some useful functionality should you > need to analyze random CSV files. I used Sniffer quite heavily for my last contract. The client had multiple multigig csv's which needed deduplicating, but they were all from different sources and therefore in different formats. It would have cost me many more hours without the Sniffer. Please keep it. <:) Robert Brewer MIS Amor Ministries [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
Armin Rigo wrote: Hi Jim, On Tue, Jan 04, 2005 at 02:44:43PM -0500, Jim Fulton wrote: Actually, unbound builtin methods are a different type than bound builtin methods: Of course, but conceptually they are similar. You would still encounter the concept if you got an unbound builtin method. There are no such things as unbound builtin methods: list.append is list.__dict__['append'] True In other words 'list.append' just returns exactly the same object as stored in the list type's dict. Guido's proposal is to make Python methods behave in the same way. OK, interesting. I'm sold then. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
At 04:30 PM 1/5/05 +, Armin Rigo wrote: Hi Guido, On Tue, Jan 04, 2005 at 10:28:03AM -0800, Guido van Rossum wrote: > Let's get rid of unbound methods. Is there any other use case for 'C.x' not returning the same as 'appropriate_super_class_of_C.__dict__["x"]' ? Er, classmethod would be one; a rather important one at that. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: Please help complete the AST branch
"Jeremy Hylton" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > The file Python/compile.txt on the ast-branch has a brief overview of > the project: http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/Attic/compile.txt?rev=1.1.2.8&only_with_tag=ast-branch&view=auto Clicking on the above gave me: (502) Bad Gateway The proxy server received an invalid response from an upstream server ??? Perhaps it is a temporary glitch on SF's backend cvs server. Terry J. Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
At 12:29 PM 1/5/05 -0500, Barry Warsaw wrote: On Wed, 2005-01-05 at 10:41, Glyph Lefkowitz wrote: > I think it would be reasonable to assign im_class only to functions > defined in class scope. The only serialization that would break in that > case is if your example had a 'del f' at the end. +1. If you're doing something funkier, then you can set that attribute yourself. -Barry Um, isn't all this stuff going to be more complicated and spread out over more of the code than just leaving unbound methods in place? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
> Um, isn't all this stuff going to be more complicated and spread out over > more of the code than just leaving unbound methods in place? Well, in an early version of Python it was as simple as I'd like ot to be again: the instancemethod type was only used for bound methods (hence the name) and C.f would return same the function object as C.__dict__["f"]. Apart from backwards compatibility with all the code that has grown cruft to deal with the fact that C.f is not a function object, I still see no reason why the current state of affairs is better. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please help complete the AST branch
Jeremy Hylton <[EMAIL PROTECTED]> writes: > Does anyone want to volunteer to integrate the current head to the > branch? I think that's a pretty important near-term step. I'll take a shot at it. I see the following: 2216 changes: 1428 modifications w/o confict 399 adds 360 removes 29 conflicts Major conflict: Python/compile.c(Probably not merged during 1st merge) Lib/test/test_compile.c (ditto) Lib/test/test_os.py (AST?) Lib/test/test_re.py (AST?) Major conflict probably not AST related: Lib/test/test_bool.py Lib/test/test_urllib.py Lib/test/output/test_profile Python/pythonrun.c (check brackets!) Other issues: need local -kk to avoid another 80 conflicts due to the priceless keyword expansion, have to watch out for binary files like IDLE icons. ViewCVS is down, slows things up. I'm going to tag the trunk: mrg_to_ast-branch_05JAN05 -- KBK ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: super() harmful?
> The issue of mixing super() and explicit calls to the superclass's > method occur with any method. (Thus making it difficult/impossible for > a framework to convert to using super without breaking client code that > subclasses). Well, client classes which are leaves of the class tree can still safely use BaseClass.thisMethod(self, args) -- it's only classes that are written to be extended that must all be converted to using super(). So I'm not sure how you think your clients are breaking. > Adding optional arguments to one branch of the inheritance tree, but > not another, or adding different optional args in both branches. > (breaks unless you always pass optional args as keywordargs, and all > methods take **kwargs and pass that on to super). But that breaks anyway; I don't see how using the old Base.method(self, args) approach makes this easier, *unless* you are using single inheritance. If you're expecting single inheritance anyway, why bother with super()? > > Super is intended for use that are designed with method cooperation in > > mind, so I agree with the best practices in James's Conclusion: > > [[omitted]] > > But that's not the same as calling it harmful. :-( > > The 'harmfulness' comes from people being confused by, and misusing > super, because it is so very very easy to do so, and so very hard to > use correctly. And using multiple inheritance the old was was not confusing? Surely you are joking. > From what I can tell, it is mostly used incorrectly. *Especially* uses > in __init__ or __new__. Many people seem to use super in their __init__ > methods thinking that it'll magically improve something (like perhaps > making multiple inheritance trees that include their class work > better), only to just cause a different set of problems for multiple > inheritance trees, instead, because they don't realize they need to > follow those recommendations. If they're happy with single inheritance, let them use super() incorrectly. It works, and that's what count. Their code didn't work right with multiple inheritance before, it still doesn't. Some people just are uncomfortable with calling Base.method(self, ...) and feel super is "more correct". Let them. > Here's another page that says much the same thing, but from the > viewpoint of recommending the use of super and showing you all the > hoops to use it right: > http://wiki.osafoundation.org/bin/view/Chandler/UsingSuper The problem isn't caused by super but by multiple inheritance. > James > > PS, I wrote that page last pycon but never got around to finishing it > up and therefore never really publically announced it. But I told some > people about it and then they kept asking me for the URL so I linked > to it, and well, then google found it of course, so I guess it's public > now. ;) Doesn't mean you can't fix it. :) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ast branch pragmatics
> I think it would be easier to create a new branch from the current > head, integrate the small number of changed files from ast-branch, and > work with that branch instead. The idea is that it's an end-run > around doing an automatic CVS merge and relying on someone to manually > merge the changes. > > At the same time, since there is a groundswell of support for > finishing the AST work, I'd like to propose that we stop making > compiler / bytecode changes until it is done. Every change to > compile.c or the bytecode ends up creating a new incompatibilty that > needs to be merged. > > If these two plans sound good, I'll get started on the new branch. +1 -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Please help complete the AST branch
Would it be helpful for me to move the peepholer out of compile.c into a separate source file? Raymond Hettinger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ast branch pragmatics
Jeremy Hylton <[EMAIL PROTECTED]> writes: > The existing ast-branch mostly works, but it does not include most of > the new features of Python 2.4. There is a substantial integration > effort, perhaps easy for someone who does a lot of CVS branch merges. > (In particular, the head has already been merged to this branch once.) > > I think it would be easier to create a new branch from the current > head, integrate the small number of changed files from ast-branch, and > work with that branch instead. The idea is that it's an end-run > around doing an automatic CVS merge and relying on someone to manually > merge the changes. > > At the same time, since there is a groundswell of support for > finishing the AST work, I'd like to propose that we stop making > compiler / bytecode changes until it is done. Every change to > compile.c or the bytecode ends up creating a new incompatibilty that > needs to be merged. > > If these two plans sound good, I'll get started on the new branch. Hm, I saw this after making my previous post. Well, you can see from that post that it's a bit of work, but not overwhelming. You have a better feel for how much change was made on ast-branch and how complete the previous merge was. So, you decide: if you want me to do the merge, I can. But ast-branch-2 sounds OK, also. -- KBK ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: csv module TODO list
Quite a while ago I posted some material to the csv-list about problems using the csv module on Unix-style colon-separated files -- it just doesn't deal properly with backslash escaping and is quite useless for this kind of file. I seem to recall the general view was that it wasn't intended for this kind of thing -- only the sort of csv that Microsoft Excel outputs/inputs, but if I am mistaken about this, perhaps fixing this issue might be put on the TODO-list? I'll be happy to re-send or summarize the relevant emails, if needed. -- Magnus Lie Hetland Fallen flower I see / Returning to its branch http://hetland.org Ah! a butterfly. [Arakida Moritake] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please help complete the AST branch
On Wed, 5 Jan 2005 13:28:11 -0500, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > Would it be helpful for me to move the peepholer out of compile.c into a > separate source file? It doesn't really matter. There are two reasons. 1) We've been working on the new compiler code in newcompile.c, rather than compile.c. When it is finished, we'll replace compile.c with newcompile.c, but it was helpful to have both around at first. 2) Peephole optimizations would be done on the basic block intermediate representation rather than code objects. So we'll need to rewrite it anyway to use the new IR. Jeremy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please help complete the AST branch
On Wed, 05 Jan 2005 13:20:51 -0500, Kurt B. Kaiser <[EMAIL PROTECTED]> wrote: > Jeremy Hylton <[EMAIL PROTECTED]> writes: > > > Does anyone want to volunteer to integrate the current head to the > > branch? I think that's a pretty important near-term step. > > I'll take a shot at it. Great! I say this after reading your other message in response to my suggestion to create a new branch. If you can manage to do the integration, it's simpler for everyone to stick to a single branch. (For example, there will be no opportunity for someone to work on the wrong branch.) > 29 conflicts Oh. That's not as bad as I expected. > Major conflict: > Python/compile.c(Probably not merged during 1st merge) I think that's right. I didn't merge any of the changes, then. > Lib/test/test_compile.c (ditto) Probably. > Lib/test/test_os.py (AST?) > Lib/test/test_re.py (AST?) I wonder if these two were edited to worm around some bugs in early versions of newcompile.c. You could check the revision history. If that's the case, it's safe to drop the changes. > Major conflict probably not AST related: > Lib/test/test_bool.py > Lib/test/test_urllib.py > Lib/test/output/test_profile > Python/pythonrun.c (check brackets!) There are actually a lot of AST-related changes in pythonrun.c, because it is the gunk between files and stdin and the actual compiler and runtime. Jeremy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
Bob Ippolito wrote: Then you haven't done the appropriate research by searching pythonmac-sig. Hmm. > Do you even own a Mac? Do I have to, in order to understand the issues? But to answer your question: yes, I do. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: Please help complete the AST branch
"Terry Reedy" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/Attic/compile.txt?rev=1.1.2.8&only_with_tag=ast-branch&view=auto > > Clicking on the above gave me: > > (502) Bad Gateway > The proxy server received an invalid response from an upstream server > > ??? Perhaps it is a temporary glitch on SF's backend cvs server. Seems so, working now. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please help complete the AST branch
Jeremy Hylton <[EMAIL PROTECTED]> writes: >> 29 conflicts > > Oh. That's not as bad as I expected. Proceeding >> Major conflict: >> Python/compile.c(Probably not merged during 1st merge) > > I think that's right. I didn't merge any of the changes, then. > >> Lib/test/test_compile.c (ditto) > > Probably. So maybe it's not necessary to merge these two; just leave them behind? That would lighten the load quite a bit. -- KBK ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On Jan 5, 2005, at 16:15, Martin v. Löwis wrote: Bob Ippolito wrote: Then you haven't done the appropriate research by searching pythonmac-sig. Hmm. > Do you even own a Mac? Do I have to, in order to understand the issues? But to answer your question: yes, I do. Well, this issue has been discussed over and over again on pythonmac-sig over the past year or so (perhaps as far back as the 10.3.0 release). I do not have time at the moment to summarize, but the solution proposed is sane and there is no known better way. If you take a look at the WWDC2004 sources for Python, a similar patch is applied by Apple. However, Apple's patch breaks (at least) C++ compilation and SciPy's distutils extension for compiling Fortran due to distutils' stupidity. -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
Ronald Oussoren wrote: It gets worse when you have a user-installed python 2.3 and a user-installed python 2.4. Those will be both be installed as /Library/Frameworks/Python.framework. Yes, but one is installed in Versions/2.3, and the other in Versions/2.4. This means that you cannot use the -F flag to select which one you want to link to, '-framework Python' will only link to the python that was installed the latest. What about using -F /Library/Frameworks/Python.framework/Versions/2.3? Or, would there be a different way to specify the version of a framework when linking, in addition to -F? What about -framework Python,/Versions/2.3 I could not find a specification how the suffix in -framework is meant to work - perhaps it could be used here? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
Andrew McNamara wrote: Can you please elaborate on that? What needs to be done, and how is that going to be done? It might be possible to avoid considerable uglification. I'm not altogether sure there. The parsing state machine is all written in C, and deals with signed chars - I expect we'll need two versions of that (or one version that's compiled twice using pre-processor macros). Quite a large job. Suggestions gratefully received. I'm still trying to understand what *needs* to be done - I would move to how this is done only later. What APIs should be extended/changed, and in what way? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On Jan 5, 2005, at 16:58, Martin v. Löwis wrote: Ronald Oussoren wrote: It gets worse when you have a user-installed python 2.3 and a user-installed python 2.4. Those will be both be installed as /Library/Frameworks/Python.framework. Yes, but one is installed in Versions/2.3, and the other in Versions/2.4. This means that you cannot use the -F flag to select which one you want to link to, '-framework Python' will only link to the python that was installed the latest. What about using -F /Library/Frameworks/Python.framework/Versions/2.3? Or, would there be a different way to specify the version of a framework when linking, in addition to -F? What about -framework Python,/Versions/2.3 Nope. The only way to link to a non-current framework version is to forego any linker searching and specify the dyld file directly, i.e. /Library/Frameworks/Python.framework/Versions/2.3/Python. The gcc toolchain does not in any way whatsoever understand versioned frameworks, period. I could not find a specification how the suffix in -framework is meant to work - perhaps it could be used here? dylib suffixes are used for having separate versions of the dylib (debug, profile, etc.). It is NOT for general production use, ever. -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
Michael Hudson wrote: Martin, can you please believe that Jack, Bob, Ronald et al know what they are talking about here? I find that really hard to believe, because it contradicts to what I think Apple wants me to believe. I'm willing to follow a series of statements that I can confirm to be facts somehow (e.g. "As TechNote XY says, OSX has a bug in that it loads the Current version at run-time, no matter what version the binary says should be used"). I'm not really willing to believe a statement without any kind of proof - regardless who made that statement. "Read the mailing lists" is no proof. If I was to accept anything said without doubt, Jack would not have needed to post his message in the first place - he expressed his opinion that he believed the changes to be appropriate. It was his doubt that triggered mine. I am not going to interfere with the changes -- it's just that I want to understand them. Kind regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
Bob Ippolito wrote: Nope. The only way to link to a non-current framework version is to forego any linker searching and specify the dyld file directly, i.e. /Library/Frameworks/Python.framework/Versions/2.3/Python. The gcc toolchain does not in any way whatsoever understand versioned frameworks, period. I see. I wish you had told me right from the beginning. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's get rid of unbound methods
Hi Phillip, On Wed, Jan 05, 2005 at 01:03:42PM -0500, Phillip J. Eby wrote: > >Is there any other use case for 'C.x' not returning the same as > >'appropriate_super_class_of_C.__dict__["x"]' ? > > Er, classmethod would be one; a rather important one at that. Oups. Right, sorry. Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: super() harmful?
On Jan 5, 2005, at 1:23 PM, Guido van Rossum wrote: The issue of mixing super() and explicit calls to the superclass's method occur with any method. (Thus making it difficult/impossible for a framework to convert to using super without breaking client code that subclasses). Well, client classes which are leaves of the class tree can still safely use BaseClass.thisMethod(self, args) -- it's only classes that are written to be extended that must all be converted to using super(). So I'm not sure how you think your clients are breaking. See the section "Subclasses must use super if their superclasses do". This is particularly a big issue with __init__. Adding optional arguments to one branch of the inheritance tree, but not another, or adding different optional args in both branches. (breaks unless you always pass optional args as keywordargs, and all methods take **kwargs and pass that on to super). But that breaks anyway; I don't see how using the old Base.method(self, args) approach makes this easier, *unless* you are using single inheritance. If you're expecting single inheritance anyway, why bother with super()? There is a distinction between simple multiple inheritance, which did work in the old system vs. multiple inheritance in a diamond structure which did not work in the old system. However, consider something like the following (ignore the Interface/implements bit if you want. It's just to point out a common situation where two classes can independently implement the same method without having a common superclass): class IFrob(Interface): def frob(): """Frob the knob""" class A: implements(IFrob) def frob(self, foo=False): print "A.frob(foo=%r)"%foo class B: implements(IFrob) def frob(self, bar=False): print "B.frob(bar=%r)"%bar class C(A,B): def m(self, foo=False, bar=False): A.m(self, foo=foo) B.m(self, bar=bar) print "C.frob(foo=%r, bar=%r)"%(foo,bar) Now, how do you write that to use super? Here's what I come up with: class IFrob(Interface): def frob(): """Frob the knob""" class A(object): implements(IFrob) def frob(self, foo=False, *args, **kwargs): try: f = super(A, self).frob except AttributeError: pass else: f(foo=foo, *args, **kwargs) print "A.frob(foo=%r)"%foo class B(object): implements(IFrob) def frob(self, bar=False, *args, **kwargs): try: f = super(B, self).frob except AttributeError: pass else: f(bar=bar, *args, **kwargs) print "B.frob(bar=%r)"%bar class C(A,B): def frob(self, foo=False, bar=False, *args, **kwargs): super(C, self).frob(foo, bar, *args, **kwargs) print "C.frob(foo=%r, bar=%r)"%(foo,bar) And using multiple inheritance the old was was not confusing? Surely you are joking. It was pretty simple until you start having diamond structures. Then it's complicated. Now, don't get me wrong, I think that MRO-calculating mechanism really is "the right thing", in the abstract. I just think the way it works out as implemented in python is really confusing and it's easy to be worse off with it than without it. If they're happy with single inheritance, let them use super() incorrectly. It works, and that's what count. Their code didn't work right with multiple inheritance before, it still doesn't. Some people just are uncomfortable with calling Base.method(self, ...) and feel super is "more correct". Let them. Their code worked right in M-I without diamonds before. Now it likely doesn't work in M-I at all. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On Jan 5, 2005, at 17:38, Martin v. Löwis wrote: Bob Ippolito wrote: Nope. The only way to link to a non-current framework version is to forego any linker searching and specify the dyld file directly, i.e. /Library/Frameworks/Python.framework/Versions/2.3/Python. The gcc toolchain does not in any way whatsoever understand versioned frameworks, period. I see. I wish you had told me right from the beginning. That is only part of the reason for these changes (concurrent Python 2.3 and Python 2.4 in the same location), and is fringe enough that I wasn't even thinking of it at the time. I just dug up some information I had written on this particular topic but never published, if you're interested: http://bob.pythonmac.org/archives/2005/01/05/versioned-frameworks- considered-harmful/ -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
[Grmpf. I should check which account I use before pressing send. Here goes again] On 5-jan-05, at 1:08, Bob Ippolito wrote: The problem we're trying to solve is that due to the way Apple's framework architecture works newer versions of frameworks are preferred (at link time, and sometimes even at runtime) over older ones. Can you elaborate on that somewhat? According to http://developer.apple.com/documentation/MacOSX/Conceptual/ BPFrameworks/Concepts/VersionInformation.html there are major and minor versions of frameworks. I would think that every Python minor (2.x) release should produce a new major framework version of the Python framework. Then, there would be no problem. Why does this not work? It doesn't for reasons I care not to explain in depth, again. But I do care:-) Specifically because I trust the crowd here to come up with good ideas (even if they're not Mac users:-). Ronald already explained most of the problem, what it boils down to is that multiple versions of a framework can live in a single location. For most applications that's better than the old MacOS9 architecture (which I believe is pretty similar to the Windows dll architecture) because you can ship a single foo.framework that contains both version 1.2 and 1.3. There's also a symlink "Current" that will point to 1.3. At build time the linker will pick the version pointed at by "Current", but in the file it will record the actual version number. Hence, if you ship this framework new development will link to the newest version, but older programs will still load the older one. When I did the framework python design I overlooked the fact that an older Python would have no way to specify that an extension would have to link against its own, old, framework, because on MacOS9 this wasn't a problem (the two had different filenames). As an aside, I also overlooked the fact that a Python framework residing in /System could be overridden by one in /Library because in 2.3 we linked frameworks by relative pathname, because I simply didn't envision any Python living in /System for some time to be. The -F options could solve that problem, but not the 2.3 and 2.4 both in /Library problem. The "new" solution is basically to go back to the Unix way of building an extension: link it against nothing and sort things out at runtime. Not my personal preference, but at least we know that loading an extension into one Python won't bring in a fresh copy of a different interpreter or anything horrible like that. -- Jack Jansen, <[EMAIL PROTECTED]>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: super() harmful?
On Wed, 5 Jan 2005 18:00:38 -0500, James Y Knight <[EMAIL PROTECTED]> wrote: > On Jan 5, 2005, at 1:23 PM, Guido van Rossum wrote: > >> The issue of mixing super() and explicit calls to the superclass's > >> method occur with any method. (Thus making it difficult/impossible for > >> a framework to convert to using super without breaking client code > >> that subclasses). > > > > Well, client classes which are leaves of the class tree can still > > safely use BaseClass.thisMethod(self, args) -- it's only classes that > > are written to be extended that must all be converted to using > > super(). So I'm not sure how you think your clients are breaking. > > See the section "Subclasses must use super if their superclasses do". > This is particularly a big issue with __init__. I see. I was thinking about subclassing a single class, you are talking about subclassing multiple bases. Subclassing two or more classes is *always* very subtle. Before 2.2 and super(), the only sane way to do that was to have all except one base class be written as a mix-in class for a specific base class (or family of base classes). The idea of calling both __init__ methods doesn't work if there's a diamond; if there *is* a diamond (or could be one), using super() is the only sane solution. > >> Adding optional arguments to one branch of the inheritance tree, but > >> not another, or adding different optional args in both branches. > >> (breaks unless you always pass optional args as keywordargs, and all > >> methods take **kwargs and pass that on to super). > > > > But that breaks anyway; I don't see how using the old > > Base.method(self, args) approach makes this easier, *unless* you are > > using single inheritance. If you're expecting single inheritance > > anyway, why bother with super()? > > There is a distinction between simple multiple inheritance, which did > work in the old system Barely; see above. > vs. multiple inheritance in a diamond structure > which did not work in the old system. However, consider something like > the following (ignore the Interface/implements bit if you want. It's > just to point out a common situation where two classes can > independently implement the same method without having a common > superclass): > > class IFrob(Interface): >def frob(): > """Frob the knob""" > > class A: >implements(IFrob) >def frob(self, foo=False): > print "A.frob(foo=%r)"%foo > > class B: >implements(IFrob) >def frob(self, bar=False): > print "B.frob(bar=%r)"%bar > > class C(A,B): >def m(self, foo=False, bar=False): [I presume you meant from instead of m here] > A.m(self, foo=foo) > B.m(self, bar=bar) > print "C.frob(foo=%r, bar=%r)"%(foo,bar) > > Now, how do you write that to use super? The problem isn't in super(), the problem is that the classes A and B aren't written cooperatively, so attempting to combine them using multiple inheritance is asking for trouble. You'd be better off making C a container class that has separate A and B instances. > > And using multiple inheritance the old was was not confusing? Surely > > you are joking. > > It was pretty simple until you start having diamond structures. Then > it's complicated. Now, don't get me wrong, I think that MRO-calculating > mechanism really is "the right thing", in the abstract. I just think > the way it works out as implemented in python is really confusing and > it's easy to be worse off with it than without it. So then don't use it. You couldn't have diamonds at all before 2.2. With *care* and *understanding* you can do the right thing in 2.2 and beyond. I'm getting tired of super() being blamed for the problems inherent to cooperative multiple inheritance. super() is the tool that you need to solve a hairy problem; but don't blame super() for the problem's hairiness. > > If they're happy with single inheritance, let them use super() > > incorrectly. It works, and that's what count. Their code didn't work > > right with multiple inheritance before, it still doesn't. Some people > > just are uncomfortable with calling Base.method(self, ...) and feel > > super is "more correct". Let them. > > Their code worked right in M-I without diamonds before. Now it likely > doesn't work in M-I at all. If you have a framework with classes written using the old paradigm that a subclass must call the __init__ (or frob) method of each of its superclasses, you can't change your framework to use super() instead while maintaining backwards compatibility. If you didn't realize that before you made the change and then got bitten by it, tough. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
Bob Ippolito wrote: I just dug up some information I had written on this particular topic but never published, if you're interested: http://bob.pythonmac.org/archives/2005/01/05/versioned-frameworks- considered-harmful/ Interesting. I don't get the part why "-undefined dynamic_lookup" is a good idea (and this is indeed what bothered me most to begin with). As you say, explicitly specifying the target .dylib should work as well, and it also does not require 10.3. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
Jack Jansen wrote: But I do care:-) Specifically because I trust the crowd here to come up with good ideas (even if they're not Mac users:-). Thanks a lot. The "new" solution is basically to go back to the Unix way of building an extension: link it against nothing and sort things out at runtime. Not my personal preference, but at least we know that loading an extension into one Python won't bring in a fresh copy of a different interpreter or anything horrible like that. This sounds good, except that it only works on OS X 10.3, right? What about older versions? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv module TODO list
Can you please elaborate on that? What needs to be done, and how is that going to be done? It might be possible to avoid considerable uglification. >> >> I'm not altogether sure there. The parsing state machine is all written in >> C, and deals with signed chars - I expect we'll need two versions of that >> (or one version that's compiled twice using pre-processor macros). Quite >> a large job. Suggestions gratefully received. > >I'm still trying to understand what *needs* to be done - I would move to >how this is done only later. What APIs should be extended/changed, and >in what way? That's certainly the first step, and I have to admit that I don't have a clear idea at this time - the unicode issue has been in the "too hard" basket since we started. Marc-Andre Lemburg mentioned that he has encountered UTF-16 encoded csv files, so a reasonable starting point would be the ability to read and parse, as well as the ability to generate, one of these. The reader interface currently returns a row at a time, consuming as many lines from the supplied iterable (with the most common iterable being a file). This suggests to me that we will need an optional "encoding" argument to the reader constructor, and that the reader will need to decode the source lines. That said, I'm hardly a unicode expert, so I may be overlooking something (could a utf-16 encoded character span a line break, for example). The writer interface probably should have similar facilities. However - a number of people have complained about the "iterator" interface, wanting to supply strings (the iterable is necessary because a CSV row can span multiple lines). It's also conceiveable that the source lines could already be unicode objects. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Csv] Re: [Python-Dev] csv module TODO list
>>I'm still trying to understand what *needs* to be done - I would move to >>how this is done only later. What APIs should be extended/changed, and >>in what way? [...] >The reader interface currently returns a row at a time, consuming as many >lines from the supplied iterable (with the most common iterable being >a file). This suggests to me that we will need an optional "encoding" >argument to the reader constructor, and that the reader will need to >decode the source lines. That said, I'm hardly a unicode expert, so I >may be overlooking something (could a utf-16 encoded character span a >line break, for example). The writer interface probably should have >similar facilities. Ah - I see that the codecs module provides an EncodedFile class - better to use this than add encoding/decoding cruft to the csv module. So, do we duplicate the current reader and writer as UnicodeReader and UnicodeWriter (how else do we know to use the unicode parser)? What about the "dialects"? I guess if a dialect uses no unicode strings, it can be applied to the current parser, but if it does include unicode strings, then the parser would need to raise an exception. The DictReader and DictWriter classes will probably need matching UnicodeDictReader/UnicodeDictWriter versions (use common base class, just specify alternate parser). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] an idea for improving struct.unpack api
A problem:
The current struct.unpack api works well for unpacking C-structures where
everything is usually unpacked at once, but it
becomes inconvenient when unpacking binary files where things
often have to be unpacked field by field. Then one has to keep track
of offsets, slice the strings,call struct.calcsize(), etc...
Eg. with a current api unpacking of a record which consists of a
header followed by a variable number of items would go like this
hdr_fmt=""
item_fmt=""
item_size=calcsize(item_fmt)
hdr_size=calcsize(hdr_fmt)
hdr=unpack(hdr_fmt, rec[0:hdr_size]) #rec is the record to unpack
offset=hdr_size
for i in range(hdr[0]): #assume 1st field of header is a counter
item=unpack( item_fmt, rec[ offset: offset+item_size])
offset+=item_size
which is quite inconvenient...
A solution:
We could have an optional offset argument for
unpack(format, buffer, offset=None)
the offset argument is an object which contains a single integer field
which gets incremented inside unpack() to point to the next byte.
so with a new API the above code could be written as
offset=struct.Offset(0)
hdr=unpack("", offset)
for i in range(hdr[0]):
item=unpack( "", rec, offset)
When an offset argument is provided, unpack() should allow some bytes to
be left unpacked at the end of the buffer..
Does this suggestion make sense? Any better ideas?
Ilya
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in
On Jan 5, 2005, at 18:46, Martin v. Löwis wrote: Bob Ippolito wrote: I just dug up some information I had written on this particular topic but never published, if you're interested: http://bob.pythonmac.org/archives/2005/01/05/versioned-frameworks- considered-harmful/ Interesting. I don't get the part why "-undefined dynamic_lookup" is a good idea (and this is indeed what bothered me most to begin with). As you say, explicitly specifying the target .dylib should work as well, and it also does not require 10.3. Without -undefined dynamic_lookup, your Python extensions are bound to a specific Python installation location (i.e. the system 2.3.0 and a user-installed 2.3.4). This tends to be quite a problem. With -undefined dynamic_lookup, they are not. Just search for "version mismatch" on pythonmac-sig: http://www.google.com/search?q=%22version+mismatch%22+pythonmac- sig+site:mail.python.org&ie=UTF-8&oe=UTF-8 -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] an idea for improving struct.unpack api
[Ilya Sandler]
> A problem:
>
> The current struct.unpack api works well for unpacking C-structures
where
> everything is usually unpacked at once, but it
> becomes inconvenient when unpacking binary files where things
> often have to be unpacked field by field. Then one has to keep track
> of offsets, slice the strings,call struct.calcsize(), etc...
Yes. That bites.
> Eg. with a current api unpacking of a record which consists of a
> header followed by a variable number of items would go like this
>
> hdr_fmt=""
> item_fmt=""
> item_size=calcsize(item_fmt)
> hdr_size=calcsize(hdr_fmt)
> hdr=unpack(hdr_fmt, rec[0:hdr_size]) #rec is the record to unpack
> offset=hdr_size
> for i in range(hdr[0]): #assume 1st field of header is a counter
>item=unpack( item_fmt, rec[ offset: offset+item_size])
>offset+=item_size
>
> which is quite inconvenient...
>
>
> A solution:
>
> We could have an optional offset argument for
>
> unpack(format, buffer, offset=None)
>
> the offset argument is an object which contains a single integer field
> which gets incremented inside unpack() to point to the next byte.
>
> so with a new API the above code could be written as
>
> offset=struct.Offset(0)
> hdr=unpack("", offset)
> for i in range(hdr[0]):
> item=unpack( "", rec, offset)
>
> When an offset argument is provided, unpack() should allow some bytes
to
> be left unpacked at the end of the buffer..
>
>
> Does this suggestion make sense? Any better ideas?
Rather than alter struct.unpack(), I suggest making a separate class
that tracks the offset and encapsulates some of the logic that typically
surrounds unpacking:
r = StructReader(rec)
hdr = r('')
for item in r.getgroups('', times=rec[0]):
. . .
It would be especially nice if it handled the more complex case where
the next offset is determined in-part by the data being read (see the
example in section 11.3 of the tutorial):
r = StructReader(open('myfile.zip', 'rb'))
for i in range(3): # show the first 3 file headers
fields = r.getgroup('LLLHH', offset=14)
crc32, comp_size, uncomp_size, filenamesize, extra_size = fields
filename = g.getgroup('c', offset=16, times=filenamesize)
extra = g.getgroup('c', times=extra_size)
r.advance(comp_size)
print filename, hex(crc32), comp_size, uncomp_size
If you come up with something, I suggest posting it as an ASPN recipe
and then announcing it on comp.lang.python. That ought to generate some
good feedback based on other people's real world issues with
struct.unpack().
Raymond Hettinger
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: super() harmful?
On Jan 5, 2005, at 6:36 PM, Guido van Rossum wrote: The idea of calling both __init__ methods doesn't work if there's a diamond; if there *is* a diamond (or could be one), using super() is the only sane solution. Very true. So then don't use it. You couldn't have diamonds at all before 2.2. With *care* and *understanding* you can do the right thing in 2.2 and beyond. I'm getting tired of super() being blamed for the problems inherent to cooperative multiple inheritance. super() is the tool that you need to solve a hairy problem; but don't blame super() for the problem's hairiness. Please notice that I'm talking about concrete, real issues, not just a "super is bad!" rant. These are initially non-obvious (to me, at least) things that will actually happen in real code and that you actually do need to watch out for if you use super. Yes. It is a hard problem. However, the issues I talk about are not issues with the functionality and theory of calling the next method in an MRO, they are issues with the combination of MROs, the implementation of MRO-calling in python (via "super"), and current practices in writing python code. They are not inherent in cooperative multiple inheritance, but occur mostly because of its late addition to python, and the cumbersome way in which you have to invoke super. I wrote up the page as part of an investigation into converting Twisted to use super. I thought it would be a good idea to do the conversion, but others told me it would be a bad idea for backwards compatibility reasons. I did not believe, at first, and conducted experiments. In the end, I concluded that it is not possible, because of the issues with mixing the new and old paradigm. If you have a framework with classes written using the old paradigm that a subclass must call the __init__ (or frob) method of each of its superclasses, you can't change your framework to use super() instead while maintaining backwards compatibility. Yep, that's what I said, too. If you didn't realize that before you made the change and then got bitten by it, tough. Luckily, I didn't get bitten by it because I figured out the consequences and wrote a webpage about them before making an incorrect code change. Leaving behind the backwards compatibility issues... In order to make super really nice, it should be easier to use right. Again, the two major issues that cause problems are: 1) having to declare every method with *args, **kwargs, and having to pass those and all the arguments you take explicitly to super, and 2) that traditionally __init__ is called with positional arguments. To fix #1, it would be really nice if you could write code something like the following snippet. Notice especially here that the 'bar' argument gets passed through C.__init__ and A.__init__, into D.__init__, without the previous two having to do anything about it. However, if you ask me to detail how this could *possibly* *ever* work in python, I have no idea. Probably the answer is that it can't. class A(object): def __init__(self): print "A" next_method class B(object): def __init__(self): print "B" next_method class C(A): def __init__(self, foo): print "C","foo=",foo next_method self.foo=foo class D(B): def __init__(self, bar): print "D", "bar=",bar next_method self.bar=bar class E(C,D): def __init__(self, foo, bar): print "E" next_method class E2(C,D): """Even worse, not defining __init__ should work right too.""" E(foo=10, bar=20) E2(foo=10, bar=20) # Yet, these ought to result in a TypeError because the quaz keyword isn't recognized by # any __init__ method on any class in the hierarchy above E/E2: E(foo=10, bar=20, quaz=5) E2(foo=10, bar=20, quaz=5) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
