[Python-Dev] Cross-platform math functions?
Andreas Raab [EMAIL PROTECTED] wrote: I'm curious if there is any interest in the Python community to achieve better cross-platform math behavior. A quick test[1] shows a non-surprising difference between the platform implementations. Question: Is there any interest in changing the behavior to produce identical results across platforms (for example by utilizing fdlibm [2])? Since I have need for a set of cross-platform math functions I'll probably start with a math-compatible fdlibm module (unless somebody has done that already ;-) [1] Using Python 2.4: import math math.cos(1.0e32) WinXP:-0.39929634612021897 LinuxX86: -0.49093671143542561 Well, I hope not, but I am afraid that there is :-( The word better is emotive and inaccurate. Such calculations are numerically meaningless, and merely encourage the confusion between consistency and correctness. There is a strong sense in which giving random results between -1 and 1 would be better. Now, I am not saying that you don't have a requirement for consistency but I am saying that confusing it with correctness (as has been fostered by IEEE 754, Java etc.) is harmful. One of the great advantages of the wide variety of arithmetics available in the 1970s is that numerical testing was easier and more reliable - if you got wildly different results on two platforms, you got a strong pointer to numerical problems. That viewpoint is regarded as heresy nowadays, but used not to be! Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Adam Olsen [EMAIL PROTECTED] wrote: On 9/4/06, Gustavo Carneiro [EMAIL PROTECTED] wrote: Now, we've had this API for a long time already (at least 2.5 years). I'm pretty sure it works well enough on most *nix systems. Event if it works 99% of the times, it's way better than *failing* *100%* of the times, which is what happens now with Python. Failing 99% of the time is as bad as failing 100% of the time, if your goal is to eliminate the short timeout on poll(). 1% is quite a lot, and it would probably have an annoying tendency to trigger repeatedly when the user does certain things (not reproducible by you of course). That can make it a lot WORSE that repeated failure. At least with hard failures, you have some hope of tracking them down in a reasonable time. The problem with exception handling code that goes off very rarely, under non-reproducible circumstances, is that it is almost untestable and that bugs in it are positive nightmares. I have been inflicted with quite a large number in my time, and have a fairly good success rate, but the number of people who know the tricks is decreasing. Consider the (real) case where an unpredictable process on a large server (64 CPUs) was failing about twice a week (detectably), with no indication of how many failures were giving wrong answers. We replaced dozens of DIMMs, took days of down time and got nowhere; it then went hard (i.e. one failure a day). After a week's total down time, with me spending 100% of my time on it and the vendor allocating an expert at high priority, we cracked it. We were very lucky to find it so fast. I could give you other examples that were/are there years and decades later, because the pain threshhold never got high enough to dedicate the time (and the VERY few people with experience). I know of at least one such problem in generic TCP/IP (i.e. on Linux, IRIX, AIX and possibly Solaris) that has been there for decades and causes occasional failure in most networked applications/protocols. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cross-platform math functions?
Nick Maclaren wrote: The word better is emotive and inaccurate. Such calculations are numerically meaningless, and merely encourage the confusion between consistency and correctness. There is a strong sense in which giving random results between -1 and 1 would be better. I did, of course, mean more consistent (and yes, random consistent results would be better by this definition and indeed I would prefer that over inconsistent but more accurate results ;-) Cheers, - Andreas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
On 9/5/06, Adam Olsen [EMAIL PROTECTED] wrote: On 9/4/06, Gustavo Carneiro [EMAIL PROTECTED] wrote: Now, we've had this API for a long time already (at least 2.5 years). I'm pretty sure it works well enough on most *nix systems. Event if it works 99% of the times, it's way better than *failing* *100%* of the times, which is what happens now with Python. Failing 99% of the time is as bad as failing 100% of the time, if your goal is to eliminate the short timeout on poll(). 1% is quite a lot, and it would probably have an annoying tendency to trigger repeatedly when the user does certain things (not reproducible by you of course). That said, I do hope we can get 100%, or at least enough nines that we can increase the timeout significantly. Anyway, I was speaking hypothetically. I'm pretty sure writing to a pipe is async signal safe. It is the oldest trick in the book, everyone uses it. I don't have to see a written signed contract to know that it works. Here's a list of web sites google found me that talk about this problem: This one describes the pipe writing technique: http://www.cocoadev.com/index.pl?SignalSafety This one presents a list of The only routines that POSIX guarantees to be Async-Signal-Safe: http://docsun.cites.uiuc.edu/sun_docs/C/solaris_9/SUNWdev/MTP/p40.html#GEN-95948 Also here: http://www.cs.usyd.edu.au/cgi-bin/man.cgi?section=5topic=attributes This is all the evidence that I need. And again I reiterate that whether or not async safety can be achieved in practice for all platforms is not Python's problem. Although I believe writing to a pipe is 100% reliable for most platforms. Even if it is not, any mission critical application relying on signals for correct behaviour should be rewritten to use unix sockets instead; end of argument. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Gustavo Carneiro [EMAIL PROTECTED] wrote: Anyway, I was speaking hypothetically. I'm pretty sure writing to a pipe is async signal safe. It is the oldest trick in the book, everyone uses it. I don't have to see a written signed contract to know that it works. Ah. Well, I can assure you that it's not the oldest trick in the book, and not everyone uses it. This is all the evidence that I need. And again I reiterate that whether or not async safety can be achieved in practice for all platforms is not Python's problem. I wish you the joy of trying to report a case where it doesn't work to a large vendor and get them to accept that it is a bug. Although I believe writing to a pipe is 100% reliable for most platforms. Even if it is not, any mission critical application relying on signals for correct behaviour should be rewritten to use unix sockets instead; end of argument. Er, no. There are lots of circumstances where that isn't feasible, such as wanting to close down an application cleanly when the scheduler sends it a SIGXCPU. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] no remaining issues blocking 2.5 release
Did you still want this addressed? Anthony and I made some comments on the bug/patch, but nothing has been updated. I was waiting because I got unassigned from the bug, so I thought the maintainer was stepping up. I'll commit a fix for it today. Thanks for pinging me, -- Gustavo Niemeyer http://niemeyer.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Fwd: Problem withthe API for str.rpartition()
Reversing the order of the return tuple will break the alignment with
split/rsplit.
Why not just change which of the three strings holds the remainder in
the not-found case?
In rc1,
d.rpartition(.) -- ('d', '', '')
If that changes to
d.rpartition(.) -- ('', '', 'd')
then
(1) the loop will terminate
(2) rpartition will be more parallel to partition (and split),
(3) people who used rpartition without looping to termination (and
therefore didn't catch the problem) will still be able to use their
existing working code.
(4) the existing docstring would remain correct, though it could
still be improved. (It says returns S and two empty strings, but
doesn't specify the order.)
-jJ
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Jim Jewett wrote: Why not just change which of the three strings holds the remainder in the not-found case? That was the only change submitted. Are you happy with what was checked-in? Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Nick Maclaren wrote: Gustavo Carneiro [EMAIL PROTECTED] wrote: Anyway, I was speaking hypothetically. I'm pretty sure writing to a pipe is async signal safe. It is the oldest trick in the book, everyone uses it. I don't have to see a written signed contract to know that it works. Ah. Well, I can assure you that it's not the oldest trick in the book, and not everyone uses it. This is all the evidence that I need. And again I reiterate that whether or not async safety can be achieved in practice for all platforms is not Python's problem. I wish you the joy of trying to report a case where it doesn't work to a large vendor and get them to accept that it is a bug. Are you saying that we should let less commonly used platforms dictate features and functionality for the popular ones? I mean, who uses HP/UX, SCO and [insert your favorite flavor] as a modern desktop system where this particular bug makes a difference? Can't this just be enabled for platforms where it's known to work and let Python as it currently is for the users of these legacy systems ? Johan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Jim Jewett wrote: Why not just change which of the three strings holds the remainder in the not-found case? On 9/5/06, Raymond Hettinger [EMAIL PROTECTED] wrote: That was the only change submitted. Are you happy with what was checked-in? This change looks wrong: PyDoc_STRVAR(rpartition__doc__, -S.rpartition(sep) - (head, sep, tail)\n\ +S.rpartition(sep) - (tail, sep, head)\n\ It looks like the code itself does the right thing, but I wasn't quite confident of that. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
This change looks wrong: PyDoc_STRVAR(rpartition__doc__, -S.rpartition(sep) - (head, sep, tail)\n\ +S.rpartition(sep) - (tail, sep, head)\n\ It looks like the code itself does the right thing, but I wasn't quite confident of that. It is correct. There may be some confusion in terminology. Head and tail do not mean left-side or right-side. Instead, they refer to the small part chopped-off and the rest that is still choppable. Think of head and tail in the sense of car and cdr. A post-condition invariant for both str.partition() and str.rpartition() is: assert sep not in head For non-looping cases, users will likely to use different variable names when they unpack the tuple: left, middle, right = s.rpartition(p) But when they perform multiple partitions, the tail or rest terminology is more appropriate for the part of the string that may still contain separators. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Jim Jewett writes: This change [in docs] looks wrong: PyDoc_STRVAR(rpartition__doc__, -S.rpartition(sep) - (head, sep, tail)\n\ +S.rpartition(sep) - (tail, sep, head)\n\ Raymond Hettinger replies: It is correct. There may be some confusion in terminology. Head and tail do not mean left-side or right-side. Instead, they refer to the small part chopped-off and the rest that is still choppable. Think of head and tail in the sense of car and cdr. It is incorrect. The purpose of documentation is to explain things to users, and documentation which fails to achieve this is not correct. The level of confusion generated by using head to refer to the last part of the string and tail to refer to the beginning, is quite significant. How about something like this: S.partition(sep) - (head, sep, tail) S.rpartition(sep) - (tail, sep, rest) Perhaps someone else can find something clearer than my suggestion, but in my own head, the terms head and tail are tighly bound with the idea of beginning and end (respectively) rather than with the idea of small part chopped off and big part that is still choppable. -- Michael Chermside ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sep 5, 2006, at 1:10 PM, Raymond Hettinger wrote: This change looks wrong: PyDoc_STRVAR(rpartition__doc__, -S.rpartition(sep) - (head, sep, tail)\n\ +S.rpartition(sep) - (tail, sep, head)\n\ It looks like the code itself does the right thing, but I wasn't quite confident of that. It is correct. There may be some confusion in terminology. Head and tail do not mean left-side or right-side. Instead, they refer to the small part chopped-off and the rest that is still choppable. Think of head and tail in the sense of car and cdr. A post-condition invariant for both str.partition() and str.rpartition() is: assert sep not in head For non-looping cases, users will likely to use different variable names when they unpack the tuple: left, middle, right = s.rpartition(p) But when they perform multiple partitions, the tail or rest terminology is more appropriate for the part of the string that may still contain separators. ISTM this is just begging for newbie (and maybe not-so-newbie) confusion. Why not just document both as returning (left, sep, right) which seems the most obvious description of what the methods return? - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRP2zPHEjvBPtnXfVAQKpvQP/X1Vg9G4gZLl9R7/fnevmfeszTbqVk1Bq V7aXYm5pTFiD27cKV2e7MKZPifob6Pg8NPjsvAh6jZU5Uj0BUQhIwgDXZpcivsTM MykyPz8oVpSLRhu5xfYU1IZjbogoKfPQ04FkqWgtM2QUqKjiLcvwzPnzLNLVxx9r v2LplvrqJyc= =Tckf -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
ISTM this is just begging for newbie (and maybe not-so-newbie) confusion. Why not just document both as returning (left, sep, right) which seems the most obvious description of what the methods return? I'm fine with that (though it's a little sad that we think the rather basic concepts of head and tail are beyond the grasp of typical pythonistas). Changing to left/sep/right will certainly disambiguate questions about the ordering of the return tuple. OTOH, there is some small loss in that the head/tail terminology is highly suggestive of how to use the function when making succesive partitions. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
On Tuesday 05 September 2006 13:24, Michael Chermside wrote: How about something like this: S.partition(sep) - (head, sep, tail) S.rpartition(sep) - (tail, sep, rest) I think I prefer: S.partition(sep) - (head, sep, rest) S.rpartition(sep) - (tail, sep, rest) Here, rest is always used for what remains; head/tail are somewhat more clear here I think. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sep 5, 2006, at 1:46 PM, Raymond Hettinger wrote: ISTM this is just begging for newbie (and maybe not-so-newbie) confusion. Why not just document both as returning (left, sep, right) which seems the most obvious description of what the methods return? I'm fine with that (though it's a little sad that we think the rather basic concepts of head and tail are beyond the grasp of typical pythonistas). Changing to left/sep/right will certainly disambiguate questions about the ordering of the return tuple. OTOH, there is some small loss in that the head/tail terminology is highly suggestive of how to use the function when making succesive partitions. Personally, I'd rather the docstring be clear and concise rather than suggestive of use cases. IMO, the latter would be better served as an example in the latex documentation. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRP25cXEjvBPtnXfVAQJ4EwQAuKnVxtyabdtAv/Eu9CcZ8EkcwCJYOoAT DmgMWeml861Sn4qN6NV1vMKbXljxiKqoSBgbKdpU+FRb6TeNiCisuWA0Q9xoOfsj Jyvy3XN54WXCUBNBnfsfUROPqxjiNGnKxYUzx2a+pjkeSSSZxDzbuplU+2ijB6w4 HJWIT4JLldA= =u6iU -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
On Tuesday 05 September 2006 13:46, Raymond Hettinger wrote: Changing to left/sep/right will certainly disambiguate questions about left/right is definately not helpful. It's also ambiguous in the case of .rpartition(), where left and right in the input and result are different. the ordering of the return tuple. OTOH, there is some small loss in that the head/tail terminology is highly suggestive of how to use the function when making succesive partitions. See my previous note in this thread for another suggestion. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
On 9/5/06, Fred L. Drake, Jr. [EMAIL PROTECTED] wrote: S.partition(sep) - (head, sep, rest) S.rpartition(sep) - (tail, sep, rest) Here, rest is always used for what remains; head/tail are somewhat more clear here I think. Then shouldn't rpartition be S.rpartition(sep) - (rest, sep, tail) Another possibility is data (for head/tail) and unparsed (for rest). S.partition(sep) - (data, sep, unparsed) S.rpartition(sep) - (unparsed, sep, data) I'm not sure which is worse -- (1) distinguishing between tail and rest (2) using (overly generic) jargon like unparsed and data. Whatever the final decision, it would probably be best to add an example to the docstring. a.b.c.rpartition(.) - (a.b, ., c) -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Then shouldn't rpartition be S.rpartition(sep) - (rest, sep, tail) Gads, the cure is worse than the disease. car and cdr are starting to look pretty good ;-) Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
On Tuesday 05 September 2006 14:02, Jim Jewett wrote: Then shouldn't rpartition be S.rpartition(sep) - (rest, sep, tail) Whichever matches reality, sure. I've lost track of the rpartition() result order. --sigh-- Another possibility is data (for head/tail) and unparsed (for rest). S.partition(sep) - (data, sep, unparsed) S.rpartition(sep) - (unparsed, sep, data) It's all data, so I think that's too contrived. I'm not sure which is worse -- (1) distinguishing between tail and rest (2) using (overly generic) jargon like unparsed and data. I don't see the distinction between tail and rest as problematic. But I've not used lisp for a long time. Whatever the final decision, it would probably be best to add an example to the docstring. a.b.c.rpartition(.) - (a.b, ., c) Agreed. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sep 5, 2006, at 2:06 PM, Raymond Hettinger wrote: Then shouldn't rpartition be S.rpartition(sep) - (rest, sep, tail) Gads, the cure is worse than the disease. car and cdr are starting to look pretty good ;-) LOL, the lisper in me likes that too, but I don't think it'll work. :) Fred's disagreement notwithstanding, I still like (left, sep, right), but another alternative comes to mind after actually reading the docstring for rpartition wink: (before, sep, after). Now, that's not ambiguous is it? Seems to work for both partition and rpartition. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRP2+AHEjvBPtnXfVAQLiPAP+N80jHkoT5VNTtX1h2cqD4pONz+j2maCI QXDBoODucxLDPrig8FJ3c6IcT+Uapifu8Rrvd7Vm8gSPMUsMqAgAqhqNDbXTkHVH xLk31en2k2fdiCQKQyKJSjE1R1CaFCezByV29FK3fWvqrrxObISRnsxf/wXB6Czu pOUNSA9LLKo= =g+iz -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Johan Dahlin wrote: Nick Maclaren wrote: Gustavo Carneiro [EMAIL PROTECTED] wrote: I'm pretty sure writing to a pipe is async signal safe. It is the oldest trick in the book, everyone uses it. I ... know that it works. Ah. Well, I can assure you that it's not the oldest trick in the book, and not everyone uses it. ... Can't this just be enabled for platforms where it's known to work and let Python as it currently is for the users of these legacy systems ? Ah, but that _is_ the current state of affairs. .5 :-) -- Scott David Daniels [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
On Tue, 5 Sep 2006, Fred L. Drake, Jr. wrote: On Tuesday 05 September 2006 13:24, Michael Chermside wrote: How about something like this: S.partition(sep) - (head, sep, tail) S.rpartition(sep) - (tail, sep, rest) I think I prefer: S.partition(sep) - (head, sep, rest) S.rpartition(sep) - (tail, sep, rest) Here, rest is always used for what remains; head/tail are somewhat more clear here I think. But isn't rest is in the wrong place there, for rpartition: that's not the string that you might typically call.rpartition() on a second time. How about: S.partition(sep) - (left, sep, rest) S.rpartition(sep) - (rest, sep, right) John ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 status
On 9/4/06, Neal Norwitz [EMAIL PROTECTED] wrote: There are 3 bugs currently listed in PEP 356 as blocking:http://python.org/sf/1551432 - __unicode__ breaks on exception classesI replied on the bug report, but might as well comment here. The problem with this bug is that BaseException now defines a __unicode__() method in its PyMethodDef. That intercepts the unicode() call on the class and it complains it was not handed an instance. I guess the only way to fix this is to toss out the __unicode__() method and change the tp_str function to return Unicode as needed (unless someone else has a better idea). Or the bug can be closed as Won't Fix. http://python.org/sf/1550938 - improper exception w/relative importhttp://python.org/sf/1541697 - sgmllib regexp bug causes hangDoes anyone want to fix the sgmlib issue?If not, we should revert this week before c2 is cut.I'm hoping that we will have *no changes*in 2.5 final from c2.Should there be any bugs/patches added to orremoved from the list?The buildbots are currently humming along, but I believe all 3 versions (2.4, 2.5, and 2.6) are fine.Test out 2.5c1+ and report all bugs!n___Python-Dev mailing listPython-Dev@python.org http://mail.python.org/mailman/listinfo/python-devUnsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
On 9/5/06, Barry Warsaw [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sep 5, 2006, at 2:06 PM, Raymond Hettinger wrote: Then shouldn't rpartition be S.rpartition(sep) - (rest, sep, tail) Gads, the cure is worse than the disease. car and cdr are starting to look pretty good ;-) LOL, the lisper in me likes that too, but I don't think it'll work. :) but when it comes to cadr, cddr, cdar... ;^) I personally prefer (left, sep, right ) since it's most clear and there are many Python programmers whose first language is not English. Fred's disagreement notwithstanding, I still like (left, sep, right), but another alternative comes to mind after actually reading the docstring for rpartition wink: (before, sep, after). Now, that's not ambiguous is it? Seems to work for both partition and rpartition. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRP2+AHEjvBPtnXfVAQLiPAP+N80jHkoT5VNTtX1h2cqD4pONz+j2maCI QXDBoODucxLDPrig8FJ3c6IcT+Uapifu8Rrvd7Vm8gSPMUsMqAgAqhqNDbXTkHVH xLk31en2k2fdiCQKQyKJSjE1R1CaFCezByV29FK3fWvqrrxObISRnsxf/wXB6Czu pOUNSA9LLKo= =g+iz -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/seojiwon%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Jim Jewett wrote:
Another possibility is data (for head/tail) and unparsed (for rest).
S.partition(sep) - (data, sep, unparsed)
S.rpartition(sep) - (unparsed, sep, data)
This communicates very little about the ordering of the return tuple.
Beware of overly general terms like data that provide no hints about
the semantics of the method.
The one good part that the terms are consistent between partition and
rpartition so that the invariant can be stated:
assert sep not in datum
I recommend we just leave the existing head/tail wording and add an
example which will make the meaning instantly clear:
'www.python.org'.rpartition('.')-- ('www.python', '.', 'org')
Also, remember that this discussion is being held in abstract. An
actual user of rpartition() is already thinking in terms of parsing from
the end of the string.
Another thought is that strings don't really have a left and right.
They have a beginning and end. The left/right or top/bottom distinction
is culture specific.
Raymond
BTW, if someone chops your ankles, does it matter which way you're
facing to decide whether it was your feet or your head that had been
cut-off?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Michael Chermside wrote:
Jim Jewett writes:
This change [in docs] looks wrong:
PyDoc_STRVAR(rpartition__doc__,
-S.rpartition(sep) - (head, sep, tail)\n\
+S.rpartition(sep) - (tail, sep, head)\n\
Raymond Hettinger replies:
It is correct. There may be some confusion in terminology. Head
and tail do not mean left-side or right-side. Instead, they refer to
the small part chopped-off and the rest that is still choppable.
Think of head and tail in the sense of car and cdr.
It is incorrect. The purpose of documentation is to explain
things to users, and documentation which fails to achieve this
is not correct. The level of confusion generated by using head
to refer to the last part of the string and tail to refer to
the beginning, is quite significant.
How about something like this:
S.partition(sep) - (head, sep, tail)
S.rpartition(sep) - (tail, sep, rest)
This isn't immediately clear to me what I will get.
s.partition(sep) - (left, sep, right)
s.rpartition(sep) - (left, sep, right)
Would be clearer, along with an explanation of what left, and right are.
I hope this discussion is only about the words used and the
documentation and not about the actual order of what is received. I
would expect both the following should be true, and it is the current
behavior.
''.join(s.partition(sep)) - s
''.join(s.rpartition(sep)) - s
Perhaps someone else can find something clearer than my suggestion,
but in my own head, the terms head and tail are tighly bound
with the idea of beginning and end (respectively) rather than with
the idea of small part chopped off and big part that is still
choppable.
Maybe this?
partition(...)
S.partition(sep) - (left, sep, right)
Partition a string at the first occurrence of sep from the
left into a tuple of left, sep, and right parts.
Returns (S, '', '') if sep is not found in S.
rpartition(...)
S.rpartition(sep) - (left, sep, right)
Partition a string at the first occurrence of sep from the right
into a tuple of left, sep, and right parts.
Returns ('', '', S) if sep is not found in S.
I feel the terms head and tail, rest etc... should be used in examples
where their meaning will be clear by the context they are used in. But
not in the definition where their meanings are not obvious.
Cheers,
Ron
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Ron Adam wrote:
Correcting myself...
I hope this discussion is only about the words used and the
documentation and not about the actual order of what is received. I
would expect both the following should be true, and it is the current
behavior.
''.join(s.partition(sep)) - s
''.join(s.rpartition(sep)) - s
'abcd'.partition('x')
('abcd', '', '')
'abcd'.rpartition('x')
('abcd', '', '')
Ok, I see Raymonds point, they are not what I expected.
Although the above is still true, the returned value for the not found condition
is inconsistent.
_Ron
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 status
Brett Cannon wrote: On 9/4/06, *Neal Norwitz* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: There are 3 bugs currently listed in PEP 356 as blocking: http://python.org/sf/1551432 - __unicode__ breaks on exception classes I replied on the bug report, but might as well comment here. The problem with this bug is that BaseException now defines a __unicode__() method in its PyMethodDef. That intercepts the unicode() call on the class and it complains it was not handed an instance. I guess the only way to fix this is to toss out the __unicode__() method and change the tp_str function to return Unicode as needed (unless someone else has a better idea). Or the bug can be closed as Won't Fix. Throwing out the __unicode__ method is fine with me -- exceptions didn't have one before the NeedForSpeed rewrite, so there would be no loss in functionality. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sep 5, 2006, at 2:32 PM, Raymond Hettinger wrote: Another thought is that strings don't really have a left and right. They have a beginning and end. The left/right or top/bottom distinction is culture specific. For the target of the method, this is true, but it's not true for the results which is what we're talking about describing here. 'left' is whatever is to the left of the separator and 'right' is whatever is to the right of the separator. Seems obvious to me. I believe (left, sep, right) will be the clearest description for all users, with little chance of confusion. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRP3HIXEjvBPtnXfVAQIx5wP+MPF5tk4moX4jH0yhGvR6gKcGBusyN152 redIr0xiNqECfrIHkc756UDLn3HhB2WdEjR9pn06RzmbgePMPcGP19cjZdHGwjFK 3e4Qg8zW3cL0iCnybL4AEaoZksuHGwJpZbId9HF60GFqYdjNTKEMNIVRI7jTE9pP zbBO6Sscnl0= =HB4k -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Raymond Hettinger wrote: Another thought is that strings don't really have a left and right. They have a beginning and end. The left/right or top/bottom distinction is culture specific. Well, it should have been epartition() and not rpartition() in that case. ;-) Is python ever edited in languages that don't use left to right lines? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Ron Adam wrote:
I hope this discussion is only about the words used and the
documentation and not about the actual order of what is received. I
would expect both the following should be true, and it is the current
behavior.
''.join(s.partition(sep)) - s
''.join(s.rpartition(sep)) - s
Right. The only thing in question is wording for the documentation.
The viable options on the table are:
* Leave the current wording and add a clarifying example.
* Switch to left/sep/right and add a clarifying example.
The former tells you which part can still contain a separator and
suggests how to use the tool when successive partitions are needed. The
latter makes the left/right ordering clear and tells you nothing about
which part can still have the separators in it. That has some import
because the use cases for rpartition() all involve strings with multiple
separators --if there were only one, you would just use partition().
BTW, the last check-in fixed the return value for the sep-not-found
case, so that now:
'a'.partition('x') -- ('a', '', '')
'a'.rpartition('x') -- ('', '', 'a')
This was necessary so that looping/recursion would work and so that
rpartition() acts as a mirror-image of partition().
Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
upto, sep, rest in whatever order they apply. I think of a partition-like function as starting at some position and matching up to the first occurence of the separator (be that left or right or diagonally, up to is relative to the search direction), and leaving the rest alone. The docs should match that, since my mental model is correct ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 status
On 9/5/06, Georg Brandl [EMAIL PROTECTED] wrote: Brett Cannon wrote: On 9/4/06, *Neal Norwitz* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: There are 3 bugs currently listed in PEP 356 as blocking: http://python.org/sf/1551432 - __unicode__ breaks on exception classes I replied on the bug report, but might as well comment here. The problem with this bug is that BaseException now defines a __unicode__() method in its PyMethodDef.That intercepts the unicode() call on the class and it complains it was not handed an instance.I guess the only way to fix this is to toss out the __unicode__() method and change the tp_str function to return Unicode as needed (unless someone else has a better idea).Or the bug can be closed as Won't Fix.Throwing out the __unicode__ method is fine with me -- exceptions didn'thave one before the NeedForSpeed rewrite, so there would be no loss in functionality.If this step is done and the tp_str function is not changed to return Unicode as needed, PEP 352 will need to be updated.-Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 status
On 9/5/06, M.-A. Lemburg [EMAIL PROTECTED] wrote: Brett Cannon wrote: On 9/4/06, Neal Norwitz [EMAIL PROTECTED] wrote: There are 3 bugs currently listed in PEP 356 as blocking: http://python.org/sf/1551432 - __unicode__ breaks on exception classes I replied on the bug report, but might as well comment here. The problem with this bug is that BaseException now defines a __unicode__() method in its PyMethodDef.That intercepts the unicode() call on the class and it complains it was not handed an instance.I guess the only way to fix this is to toss out the __unicode__() method and change the tp_str function to return Unicode as needed (unless someone else has a better idea).Or the bug can be closed as Won't Fix. The proper fix would be to introduce a tp_unicode slot and letthis decide what to do, ie. call .__unicode__() methods on instancesand use the .__name__ on classes.That was my bug reaction and what I said on the bug report. Kind of surprised one doesn't already exist. I think this would be the right way to go for Python 2.6. ForPython 2.5, just dropping this .__unicode__ method on exceptions is probably the right thing to do.Neal, do you want to rip it out or should I?-Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
On 9/5/06, Tim Peters [EMAIL PROTECTED] wrote: upto, sep, rest in whatever order they apply. I think of a partition-like function as starting at some position and matching up to the first occurence of the separator (be that left or right or diagonally, up to is relative to the search direction), and leaving the rest alone. The docs should match that, since my mental model is correct ;-) +1 Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Signals, threads, blocking C functions
Johan Dahlin [EMAIL PROTECTED] wrote: Are you saying that we should let less commonly used platforms dictate features and functionality for the popular ones? I mean, who uses HP/UX, SCO and [insert your favorite flavor] as a modern desktop system where this particular bug makes a difference? You haven't been following the thread. As I posted, this problem occurs to a greater or lesser degree on all platforms. This will be my last posting on the topic, but I shall try to explain. The first problem is in the hardware and operating system. A signal interrupts the thread, and passes control to a handler with a very partial environment and (usually) information on the environment when it was interrupted. If it interrupted the thread in the middle of a system call or other library routine that uses non-Python conventions, the registers and other state may be weird. There ARE solutions to this, but they are unbelievably foul, and even Linux on x86 gas had trouble with this. And, on return, everything has to be reversed entirely transparently! It is VERY common for there to be bugs in the C run-time system and not rare for there to be ones in the kernel (that area of Linux has been rewritten MANY times, for this reason). In many cases, the run-time system simply doesn't pretend to handle interrupts in arbitrary code (which is where the C undefined behaviour is used by vendors). The second problem is that what you can do depends both on what you were doing and how your 'primitive' is implemented. For example, if you call something that takes out even a very short term lock or uses a spin loop to emulate an atomic operation, you had better not use it if you interrupted code that was doing the same. Your thread may hang, crash or otherwise go bananas. Can you guarantee that even write is free of such things? No, and certainly not if you are using a debugger, a profiling library or even tracing system calls. I have often used programs that crashed as soon as I did one of those :-( Related to this is that it is EXTREMELY hard to write synchronisation primitives (mutexes etc.) that are interrupt-safe - MUCH harder than to write thread-safe ones - and few people are even aware of the issues. There was a thread on some Linux kernel mailing list about this, and even the kernel developers were having headaches thinking about the issues. Even if write is atomic, there are gotchas. What if the interrupted code is doing something to that file at the time? Are you SURE that an unexpected operation on it (in the same thread) won't cause the library function of program to get confused? And can you be sure that the write will terminate fast enough to not cause time-critical code to fail? And have you studied the exact semantics of blocking on pipes? They are truly horrible. So this is NOT a matter of platform X is safe and platform Y isn't. Even Linux x86 isn't entirely safe - or wasn't, the last time I heard. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Tim Peters wrote: upto, sep, rest in whatever order they apply. In the rpartition case, that would be (rest, sep, upto) which seems a bit cryptic. We need some choice of words that clearly mean: * the chopped-off snippet (guaranteed to not contain the separator) * the separator if found * the unchopped remainer of the string (which may contain a separator). Of course, if a clear example is added, the choice of words becomes much less important. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sep 5, 2006, at 4:13 PM, Raymond Hettinger wrote: Tim Peters wrote: upto, sep, rest in whatever order they apply. In the rpartition case, that would be (rest, sep, upto) which seems a bit cryptic. We need some choice of words that clearly mean: * the chopped-off snippet (guaranteed to not contain the separator) * the separator if found * the unchopped remainer of the string (which may contain a separator). Of course, if a clear example is added, the choice of words becomes much less important. Ideally too, the terminology (and order) for partition and rpartition would be the same. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRP3bVXEjvBPtnXfVAQJSKwP9Ev3MPzum3kp4hNDJZyBmEShzPvL2WQv2 VThbxZX1MDfeDXupNwF22bFA5gF/9vZp3nToUqyAbOaPSd93hJSHOdeWdAhR2BdT EICkzBTGCtVkbqu3Ep1N/jb9GJUvgkgNAWtRZVuTWQtJc6AanV9ssTcF6F7ipc6p zgSWeAc0a3E= =W7LV -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
I think I finally figured out where Raymond is coming from. For Raymond, head is where he started processing -- for rpartition, this is the .endswith part. For me, head is the start of the data structure -- always the .startswith part. We won't resolve that with anything suggesting a sequential order; we need something that makes it clear which part is the large leftover. S.partition(sep) - (record, sep, remains) S.rpartition(sep) - (remains, sep, record) I do like the plural (or collective) sound of remains. I have no solid reasoning for record vs rec vs onerec. I would welcome a word that did not suggest it would have further internal structure. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sep 5, 2006, at 4:43 PM, Jim Jewett wrote: I think I finally figured out where Raymond is coming from. For Raymond, head is where he started processing -- for rpartition, this is the .endswith part. For me, head is the start of the data structure -- always the .startswith part. We won't resolve that with anything suggesting a sequential order; we need something that makes it clear which part is the large leftover. See, for me, it's all about the results of the operation, not how the results are (supposedly) used. The way I think about it is that I've got some string and I'm looking for some split point within that string. That split point is clearly the middle (but sep works too) and everything to the right of that split point gets returned in right while everything to the left gets returned in left. I'm less concerned with repeated splits because I probably have as many existing cases where I'm looking for the first split point as where I'm looking repeatedly for split points (think RFC 2822 header splitting -- partition will be awesome for this). The bias with these terms is clearly the English left-to-right order. Actually, that brings up an interesting question: what would happen if you called rpartition on a unicode string representing Hebrew, Arabic, or other RTL language? Do partition and rpartition suddenly switch directions? If not, then I think left-sep-right are fine. If so, then yeah, we probably need something else. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRP3kUHEjvBPtnXfVAQJd6wP+OBtRR22O0A+s/uHF3ACgWhrdZJdEnzEW qimKEWmDCUuK7CFIUsJKteoNNSHjIBgZIMMdnsymgI7CPgPNuB6CUAp8KFFeYvMy PVpMIqNFOFXGUVYf4VA7ED9S7QbbDzHJv32kUUZvbuTniYK9DVMi0O7GStsv1Kg6 insyP+W1EcU= =4aar -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
At 04:55 PM 9/5/2006 -0400, Barry Warsaw wrote: On Sep 5, 2006, at 4:43 PM, Jim Jewett wrote: I think I finally figured out where Raymond is coming from. For Raymond, head is where he started processing -- for rpartition, this is the .endswith part. For me, head is the start of the data structure -- always the .startswith part. We won't resolve that with anything suggesting a sequential order; we need something that makes it clear which part is the large leftover. See, for me, it's all about the results of the operation, not how the results are (supposedly) used. The way I think about it is that I've got some string and I'm looking for some split point within that string. That split point is clearly the middle (but sep works too) and everything to the right of that split point gets returned in right while everything to the left gets returned in left. +1 for left/sep/right for both operations. It's easier to remember a visual correlation (left,sep,right) than it is to try and think about an abstraction in which the order of results has something to do with what direction I found the separator in. If I'm repeating from right to left, then of course the left is the part I'll want to repeat on. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
Barry Warsaw wrote: The bias with these terms is clearly the English left-to-right order. Actually, that brings up an interesting question: what would happen if you called rpartition on a unicode string representing Hebrew, Arabic, or other RTL language? Do partition and rpartition suddenly switch directions? What happens is that rpartition searches the string backwards in logical order (i.e. left to right as the text is written, assuming it only contains Hebrew or Arabic letters, and not numbers or a mixture of scripts). But this is not switching directions; it's still searching backwards. You really don't want to think of bidirectional text in terms of presentation, when you're doing processing that should be independent of presentation. If not, then I think left-sep-right are fine. If so, then yeah, we probably need something else. +1 for (upto, sep, rest) -- and I think it should be in that order for both partition and rpartition. -- David Hopwood [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Problem withthe API for str.rpartition()
At 02:08 AM 9/6/2006 +0100, David Hopwood wrote: Barry Warsaw wrote: The bias with these terms is clearly the English left-to-right order. Actually, that brings up an interesting question: what would happen if you called rpartition on a unicode string representing Hebrew, Arabic, or other RTL language? Do partition and rpartition suddenly switch directions? What happens is that rpartition searches the string backwards in logical order (i.e. left to right as the text is written, assuming it only contains Hebrew or Arabic letters, and not numbers or a mixture of scripts). But this is not switching directions; it's still searching backwards. You really don't want to think of bidirectional text in terms of presentation, when you're doing processing that should be independent of presentation. If not, then I think left-sep-right are fine. If so, then yeah, we probably need something else. +1 for (upto, sep, rest) -- and I think it should be in that order for both partition and rpartition. It appears the problem is that one group of people thinks in terms of the order of the string, and the other in terms of the order of processing. Both groups agree that both partition and rpartition should be in the same order -- but we disagree about what that means. :) Me, I want left/sep/right because I'm in the string order camp, and you want upto/sep/rest because you're in the processing order camp. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] inspect.py very slow under 2.5
Hi all, I know that the 2.5 release is extremely close, so this will probably be 2.5.1 material. I discussed it briefly with Guido at scipy'06, and he asked for some profile-based info, which I've only now had time to gather. I hope this will be of some use, as I think the problem is rather serious. For context: I am the IPython lead developer (http://ipython.scipy.org), and ipython is used as the base shell for several interactive environments, one of which is the mathematics system SAGE (http://modular.math.washington.edu/sage). It was the SAGE lead who first ran into this problem while testing SAGE with 2.5. The issue is the following: ipython provides several exception reporting modes which give a lot more information than python's default tracebacks. In order to generate this info, it makes extensive use of the inspect module. The module in ipython responsible for these fancy tracebacks is: http://projects.scipy.org/ipython/ipython/browser/ipython/trunk/IPython/ultraTB.py which is an enhanced port of Ka Ping-Yee's old cgitb module. Under 2.5, the generation of one of these detailed tracebacks is /extremely/ expensive, and the cost goes up very quickly the more modules have been imported into the current session. While in a new ipython session the slowdown is not crippling, under SAGE (which starts with a lot of loaded modules) it is bad enough to make the system nearly unusable. I'm attaching a little script which can be run to show the problem, but you need IPython to be installed to run it. If any of you run ubuntu, fedora, suse or almost any other major linux distro, it's already available via the usual channels. In case you don't want to (or can't) run the attached code, here's a summary of what I see on my machine (ubuntu dapper). Using ipython under python 2.4.3, I get: 2268 function calls (2225 primitive calls) in 0.020 CPU seconds Ordered by: call count List reduced from 127 to 32 due to restriction 0.25 ncalls tottime percall cumtime percall filename:lineno(function) 3050.0000.0000.0000.000 :0(append) 259/2530.0100.0000.0100.000 :0(len) 1770.0000.0000.0000.000 :0(isinstance) 900.0000.0000.0000.000 :0(match) 680.0000.0000.0000.000 ultraTB.py:539(tokeneater) 680.0000.0000.0000.000 tokenize.py:16 (generate_tokens) 610.0000.0000.0000.000 :0(span) 570.0000.0000.0000.000 sre_parse.py:130(__getitem__) 560.0000.0000.0000.000 string.py:220(lower) etc, while running the same script under ipython/python2.5 and no other changes gives: 230370 function calls (229754 primitive calls) in 3.340 CPU seconds Ordered by: call count List reduced from 83 to 21 due to restriction 0.25 ncalls tottime percall cumtime percall filename:lineno(function) 550030.4200.0000.4200.000 :0(startswith) 450260.2640.0000.2640.000 :0(endswith) 200130.1480.0000.1480.000 :0(append) 121380.1800.0000.6600.000 posixpath.py:156(islink) 121380.1920.0000.1920.000 :0(lstat) 121380.1800.0000.2880.000 stat.py:60(S_ISLNK) 121380.1080.0000.1080.000 stat.py:29(S_IFMT) 118380.6800.0001.2440.000 posixpath.py:56(join) 48370.0520.0000.0520.000 :0(len) 43620.0280.0000.0280.000 :0(split) 43620.0480.0000.1000.000 posixpath.py:47(isabs) 35980.0360.0000.0560.000 string.py:218(lower) 35980.0200.0000.0200.000 :0(lower) 28150.0320.0000.0320.000 :0(isinstance) 28090.0280.0000.0280.000 :0(join) 28080.2640.0000.5200.000 posixpath.py:374(normpath) 26320.0400.0000.0680.000 inspect.py:35(ismodule) 21430.0160.0000.0160.000 :0(hasattr) 18840.0280.0000.4440.000 posixpath.py:401(abspath) 15570.0160.0000.0160.000 :0(range) 10780.0080.0000.0440.000 inspect.py:342(getfile) These enormous numbers of calls are the origin of the slowdown, and the more modules have been imported, the worse it gets. I haven't had time to dive deep into inspect.py to try and fix this, but I figured it would be best to at least report it now. As far as IPython and its user projects is concerned, I'll probably hack things to overwrite inspect.py from 2.4 over the 2.5 version in the exception reporter, because the current code is simply unusable for detailed tracebacks. It would be great if this could be fixed in the trunk at some point. I'll be happy to provide further feedback or put this information elsewhere. Guido suggested initially posting here, but if you prefer it on the SF tracker (even as
