Re: [Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython)
On Thu, Jul 29, 2010 at 6:53 AM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: On 28/07/10 23:12, Antoine Pitrou wrote: It should be noted, though, that a full GC can be detrimental to real-time applications. Kristján has already explained how some of his software disabled the cyclic GC, and took care of breaking cycles manually instead. This worries me, too. I'd be upset if I could no longer write games in Python that achieve smooth animation because of unpredictable GC pauses. There are always concurrent or incremental GCs which do not stop the world. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Best practice for new namespace (from C/API)
Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. Though I must admit the question is deeper than usual. On Thu, Jul 29, 2010 at 08:11:39AM +1000, Campbell Barton wrote: - Whats the best way to manage the namespace for running multiple scripts one after another? clear and initialize __main__'s dict each time?, keep a copy and restore it after each run? - should the original __main__ namespace be restored after running a script? - pickle expects: __import__(__main__).__dict__ == ***the current namespace***, is this apart of the python spec or just something specific to pickle? - PyDict_SetItemString(d, __builtins__, PyEval_GetBuiltins()) acts differently to PyDict_SetItemString(dict, __builtins__, PyImport_AddModule(builtins)), using the PyEval_GetBuiltins() dict adds every member of __builtins__ when running: print(__import__(__main__).__dict__.keys()), rather then just showing __builtins__. Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts fresh after EuroPython
On Thu, Jul 29, 2010 at 8:57 AM, Jesse Noller jnol...@gmail.com wrote: I thought at the last two pycons, we've all discussed that we should have a system in place for marking tests *and* modules within the stdlib as will only work on FooPython. I suspect that it's waiting on the shared-stdlib effort, which is waiting on mercurial (and time). @skipIf, @cpython_only and @test_impl_detail have been getting sprinkled fairly liberally throughout the test suite, so that part of the effort is already in progress. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Best practice for new namespace (from C/API)
On Thu, Jul 29, 2010 at 8:23 PM, Oleg Broytman p...@phd.pp.ru wrote: Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. Though I must admit the question is deeper than usual. capi-sig may also be able to provide some assistance if c.l.p isn't sufficiently helpful. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Thoughts fresh after EuroPython
On Thu, Jul 29, 2010 at 8:10 AM, Nick Coghlan ncogh...@gmail.com wrote: On Thu, Jul 29, 2010 at 8:57 AM, Jesse Noller jnol...@gmail.com wrote: I thought at the last two pycons, we've all discussed that we should have a system in place for marking tests *and* modules within the stdlib as will only work on FooPython. I suspect that it's waiting on the shared-stdlib effort, which is waiting on mercurial (and time). @skipIf, @cpython_only and @test_impl_detail have been getting sprinkled fairly liberally throughout the test suite, so that part of the effort is already in progress. mr. burnsExcellent/mr. burns I have some sprinkling of my own to do then, I guess. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] unexpected import behaviour
Hi, I'm not sure if this is a bug or not, I certainly didn't expect it. If you create a file called test.py with the following contents, class Test: pass def test_1(): import test print Test == test.Test if __name__ == '__main__': test_1() and then run it ($ python test.py), it'll print False. Now try: $python import test test.test_1() and it'll print True. Is this behaviour expected? What was the rationale for it if is? Thanks, Daniel -- active-thought.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unexpected import behaviour
Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. On Thu, Jul 29, 2010 at 07:32:28AM +0100, Daniel Waterworth wrote: class Test: pass def test_1(): import test print Test == test.Test if __name__ == '__main__': test_1() and then run it ($ python test.py), it'll print False. The problem is that when you run the code as a script it is imported as module __main__; when you import it as 'test' you get the second copy of the module. Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unexpected import behaviour
On Thu, Jul 29, 2010 at 4:32 PM, Daniel Waterworth da.waterwo...@gmail.com wrote: Hi, I'm not sure if this is a bug or not, I certainly didn't expect it. If you create a file called test.py with the following contents, class Test: pass def test_1(): import test print Test == test.Test if __name__ == '__main__': test_1() and then run it ($ python test.py), it'll print False. Now try: $python import test test.test_1() and it'll print True. Is this behaviour expected? What was the rationale for it if is? The behaviour is expected, but there's no particularly deep rationale for it - the interpreter just doesn't go out of its way to try and figure out what __main__ *would* have been called if it had been imported rather than executed (your script will still print False even if you run it via python -m test). We certainly *could* put __main__ into sys.modules under both names (i.e. __main__ and test in your example), but the backwards compatibility implications of doing so aren't particularly clear. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unexpected import behaviour
On 29/07/2010 07:32, Daniel Waterworth wrote: Hi, I'm not sure if this is a bug or not, I certainly didn't expect it. If you create a file called test.py with the following contents, The issue is that when your code is executed as a script it is run as the __main__ module and not as the test module. When you import test you then import the same module with a different name - and your Test class is recreated (so __main__.Test is then different from test.Test). When you import your code as test and it then reimports itself it is only created once. This *is* expected behaviour (not a bug), but it frequently confuses even relatively experienced programmers (it can happen by accident and cause hard to track down bugs) and I personally think that Python would be improved by issuing a warning if a __main__ script reimports itself. All the best, Michael Foord class Test: pass def test_1(): import test print Test == test.Test if __name__ == '__main__': test_1() and then run it ($ python test.py), it'll print False. Now try: $python import test test.test_1() and it'll print True. Is this behaviour expected? What was the rationale for it if is? Thanks, Daniel -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [Concurrency / parallelism] Re: Thoughts fresh after EuroPython
On 25 July 2010 19:26, Jesse Noller jnol...@gmail.com wrote: On Sat, Jul 24, 2010 at 10:08 AM, Guido van Rossum gu...@python.org wrote: - Concurrency and parallelism: Russel Winder and Sarah Mount pushed the idea of CSP (http://en.wikipedia.org/wiki/Communicating_sequential_processes) in several talks at the conference. They (at least Russell) emphasized the difference between concurrency (interleaved event streams) and parallelism (using many processors to speed things up). Their prediction is that as machines with many processing cores become more prevalent, the relevant architecture will change from cores sharing a single coherent memory (the model on which threads are based) to one where each core has a limited amount of private memory, and communication is done via message passing between the cores. This gives them (and me :-) hope that the GIL won't be a problem as long as we adopt a parallel processing model. Two competing models are the Actor model, which is based on asynchronous communication, and CSP, which is synchronous (when a writer writes to a channel, it blocks until a reader reads that value -- a rendezvous). At least Sarah suggested that both models are important. She also mentioned that a merger is under consideration between the two major CSP-for-Python packages, Py-CSP and Python-CSP. I also believe that the merger will be based on the stdlib multiprocessing package, but I'm not sure. I do expect that we may get some suggestions from that corner to make some minor changes to details of multiprocessing (and perhaps threading), and I think we should be open to those (I expect these will be good suggestions for small tweaks, not major overhauls). I'm open to changes; but remain skeptical the CSP will suddenly take over the world :) There's other, competing philosophies and toolkits out there as well. Additionally; the patches would have to be relicensed - python-csp is under GPLv2 AFAICT. Thanks for this write-up. Just a few things to follow-up from the comments here... * There is a discussion taking place, as we speak on the python-csp list, about merging PyCSP and python-csp. Like I said in the talk, it's really a matter of details more than anything else and will likely go ahead, hopefully soon. The issue of licensing is one thing we are talking about. * We do currently use threading and multiprocessing. I think we could achieve a big performance boost by loosing mp and moving to a C implementation of much of our own work, but only because so much of mp is pure Python and implementing a message-passing library requires some rather idiosyncratic programming. That's really an implementation detail for us though, so please don't read it as a comment about mp or threading in general. * In terms of the Python standard library, python-csp was very much written with our own purposes in mind, and in order to provide a platform for the CSP paradigm of programming to be used in a Pythonic style. Multiprocessing as it currently stands is great, and provides good support for the sort of design patterns that are currently very common in concurrent and parallel programming, like process-safe queues and so on. That's brilliant, but I would like to see the Python stdlib support a wider range of programming styles, including but not exclusive to message-passing concurrency. java.util.concurrent does this very nicely and has a wide range of constructs such as barriers, futures and so on. Python has are a wide range of packages outside the standard library (trellis, pycells, python-csp, STM implementations, and so on) but not so much that comes inside the stdlib. I think that's a shame. I won't bang on about this too much here, we'll write up some code over the next few months and when there's something more concrete and fully-baked to discuss I think it would a really useful discussion to have. Thanks, Sarah -- Sarah Mount, Senior Lecturer, University of Wolverhampton website: http://www.snim2.org/ twitter: @snim2 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). Apart from simplifying the internals a little bit, one nice feature of this change is that it removes the differences in formatting between printing a float and printing a container of floats: l = [1/3, 1/5, 1/7] print(l) [0., 0.2, 0.14285714285714285] print(l[0], l[1], l[2]) 0. 0.2 0.142857142857 Any thoughts or comments on this? There's a working patch at http://bugs.python.org/issue9337 Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
On 29/07/2010 19:47, Mark Dickinson wrote: Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). Apart from simplifying the internals a little bit, one nice feature of this change is that it removes the differences in formatting between printing a float and printing a container of floats: l = [1/3, 1/5, 1/7] print(l) [0., 0.2, 0.14285714285714285] print(l[0], l[1], l[2]) 0. 0.2 0.142857142857 Any thoughts or comments on this? There's a working patch at http://bugs.python.org/issue9337 +1 Michael Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
Wiadomość napisana przez Mark Dickinson w dniu 2010-07-29, o godz. 20:47: Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). Any thoughts or comments on this? +1 -- Best regards, Łukasz Langa tel. +48 791 080 144 WWW http://lukasz.langa.pl/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
On Jul 29, 2010, at 11:47 AM, Mark Dickinson wrote: Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). When you proposed the idea at EuroPython, it seemed reasonable but we didn't go into the pros and cons. The downsides include breaking tests, changing the output of report generating scripts that aren't using string formatting, and it introduces another inter-version incompatibility. The only obvious advantage is that it makes float.__repr__ and float.__str__ the same, making one less thing to explain. Can you elaborate on other advantages? Is there something wrong with the current way? IIRC, some other tools like matlab have a relatively compact default display size for floats, perhaps because formatting matrices becomes awkward when there are too many digits shown and because many of those digits are insignificant. Also, I think those tools have a way to globally change the default number of digits. Am curious about your thoughts on the problem we're trying to solve and the implications of changing the default. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
On Thu, Jul 29, 2010 at 8:16 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On Jul 29, 2010, at 11:47 AM, Mark Dickinson wrote: Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). When you proposed the idea at EuroPython, it seemed reasonable but we didn't go into the pros and cons. The downsides include breaking tests, changing the output of report generating scripts that aren't using string formatting, and it introduces another inter-version incompatibility. Yes, I agree that the change has potential for breakage; it's a change that probably would have been unacceptable for Python 2.7; for Python 3.2 I think there's a little more scope, since 3.x has fewer users. And those users it does have at the moment are the early adopters, who with any luck may be more tolerant of this level of breakage. (By the time we get to 3.2 - 3.3 that's probably not going to be true any more.) Really, this change should have gone into 3.1. FWIW, the change broke very few of the standard library tests (as Eric Smith verified): there was a (somewhat buggy) doctest in test_tokenize that needed fixing, and test_unicodedata computes a checksum that depends on the str() of various numeric values. Apart from those, only test_float and test_complex needed fixing to reflect the __str__ method changes. The only obvious advantage is that it makes float.__repr__ and float.__str__ the same, making one less thing to explain. Can you elaborate on other advantages? Is there something wrong with the current way? That's one advantage; as mentioned earlier the difference between str and repr causes confusion for floats in containers, where users don't realize that two different operations are being used. This is a genuine problem: I've answered questions about this a couple of times on the #python IRC channel. Another advantage is that is makes 'str' faithful: that is, if x and y are distinct floats then str(x) and str(y) are guaranteed distinct. I know I should know better, but I've been bitten by the lack of faithfulness a couple of times when debugging floating-point problems: I insert a print(x, y) line into the code for debugging purposes and still wonder why my 'assertEqual(x, y)' test is failing even though x and y look the same; only then do I remember that I need to use repr instead. As you say, it's just one less surprise, and one less thing to explain: a small shrinkage of the mental footprint of the language. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
On Thu, Jul 29, 2010 at 1:30 PM, Mark Dickinson dicki...@gmail.com wrote: On Thu, Jul 29, 2010 at 8:16 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On Jul 29, 2010, at 11:47 AM, Mark Dickinson wrote: Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). When you proposed the idea at EuroPython, it seemed reasonable but we didn't go into the pros and cons. The downsides include breaking tests, changing the output of report generating scripts that aren't using string formatting, and it introduces another inter-version incompatibility. Yes, I agree that the change has potential for breakage; it's a change that probably would have been unacceptable for Python 2.7; for Python 3.2 I think there's a little more scope, since 3.x has fewer users. And those users it does have at the moment are the early adopters, who with any luck may be more tolerant of this level of breakage. (By the time we get to 3.2 - 3.3 that's probably not going to be true any more.) Really, this change should have gone into 3.1. FWIW, the change broke very few of the standard library tests (as Eric Smith verified): there was a (somewhat buggy) doctest in test_tokenize that needed fixing, and test_unicodedata computes a checksum that depends on the str() of various numeric values. Apart from those, only test_float and test_complex needed fixing to reflect the __str__ method changes. The only obvious advantage is that it makes float.__repr__ and float.__str__ the same, making one less thing to explain. Can you elaborate on other advantages? Is there something wrong with the current way? That's one advantage; as mentioned earlier the difference between str and repr causes confusion for floats in containers, where users don't realize that two different operations are being used. This is a genuine problem: I've answered questions about this a couple of times on the #python IRC channel. Another advantage is that is makes 'str' faithful: that is, if x and y are distinct floats then str(x) and str(y) are guaranteed distinct. I know I should know better, but I've been bitten by the lack of faithfulness a couple of times when debugging floating-point problems: I insert a print(x, y) line into the code for debugging purposes and still wonder why my 'assertEqual(x, y)' test is failing even though x and y look the same; only then do I remember that I need to use repr instead. As you say, it's just one less surprise, and one less thing to explain: a small shrinkage of the mental footprint of the language. +1 from me for all the reasons Mark mentioned. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions
FWIW, I am +1 on dropping tkinter interface. Tkinter window looks foreign next to browser and server-side GUI that opens a new client window with each search topic does not strike me as most usable design. Furthermore, I just tried to use it on my OSX laptop and it crashed after I searched for pydoc and clicked on the first entry. (Another issue is that search window pops under the terminal window.) I think Tkinter interface to pydoc may make sense in IDLE, but not in the main pydoc GUI. If the equivalent functionality is available in the browser (preferably in the style familiar to docs.python.org users, I don't see why we need to keep old GUI and hide new behind a new option. I agree. What do you think of having a -i command line option to enter an interactive help session directly from the command line. This is easy to do. The instruction when entering and leaving need to change a bit, but that isn't hard to do. Cheers, Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
On 7/29/2010 4:30 PM, Mark Dickinson wrote: As you say, it's just one less surprise, and one less thing to explain: a small shrinkage of the mental footprint of the language. With this change, I believe the only difference between str(ob) and repr(ob) will be the addition of quotes. If so, perhaps this could be noted in the repr entry. Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical tofloat__repr__ in Python 3.2
Mark Dickinson wrote: Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). Apart from simplifying the internals a little bit, one nice feature of this change is that it removes the differences in formatting between printing a float and printing a container of floats: l = [1/3, 1/5, 1/7] print(l) [0., 0.2, 0.14285714285714285] print(l[0], l[1], l[2]) 0. 0.2 0.142857142857 Any thoughts or comments on this? There's a working patch at http://bugs.python.org/issue9337 Python 2.5.4 (r254:67916, Jan 20 2010, 21:44:03) float(0.142857142857) * 7 0.99981 float(0.14285714285714285) * 7 1.0 I've made a number of tools in the past that needed to round-trip a float through a string and back. I was under the impression that floats needed 17 decimal digits to avoid losing precision. How does one do that efficiently if neither str nor repr return 17 digits? Robert Brewer fuman...@aminus.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical tofloat__repr__ in Python 3.2
On 29/07/2010 22:37, Robert Brewer wrote: Mark Dickinson wrote: Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). Apart from simplifying the internals a little bit, one nice feature of this change is that it removes the differences in formatting between printing a float and printing a container of floats: l = [1/3, 1/5, 1/7] print(l) [0., 0.2, 0.14285714285714285] print(l[0], l[1], l[2]) 0. 0.2 0.142857142857 Any thoughts or comments on this? There's a working patch at http://bugs.python.org/issue9337 Python 2.5.4 (r254:67916, Jan 20 2010, 21:44:03) float(0.142857142857) * 7 0.99981 float(0.14285714285714285) * 7 1.0 I've made a number of tools in the past that needed to round-trip a float through a string and back. I was under the impression that floats needed 17 decimal digits to avoid losing precision. How does one do that efficiently if neither str nor repr return 17 digits? Because every floating point number represents a range of values - and the new algorithm uses the shortest possible representation within that range of values that will return the same floating point number. Mark did an excellent presentation on this at EuroPython and the slides are online: http://www.slideshare.net/dickinsm/magical-float-repr Michael Foord Robert Brewer fuman...@aminus.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
When you proposed the idea at EuroPython, it seemed reasonable but we didn't go into the pros and cons. The downsides include breaking tests, changing the output of report generating scripts that aren't using string formatting, and it introduces another inter-version incompatibility. Yes, I agree that the change has potential for breakage; it's a change that probably would have been unacceptable for Python 2.7; for Python 3.2 I think there's a little more scope, since 3.x has fewer users. +1 for making the change to 3.2 +0 for 2.7 The only obvious advantage is that it makes float.__repr__ and float.__str__ the same, making one less thing to explain. Can you elaborate on other advantages? Is there something wrong with the current way? That's one advantage; as mentioned earlier the difference between str and repr causes confusion for floats in containers, where users don't realize that two different operations are being used. This is a genuine problem: I've answered questions about this a couple of times on the #python IRC channel. Another advantage is that is makes 'str' faithful: that is, if x and y are distinct floats then str(x) and str(y) are guaranteed distinct. I know I should know better, but I've been bitten by the lack of faithfulness a couple of times when debugging floating-point problems: I insert a print(x, y) line into the code for debugging purposes and still wonder why my 'assertEqual(x, y)' test is failing even though x and y look the same; only then do I remember that I need to use repr instead. As you say, it's just one less surprise, and one less thing to explain: a small shrinkage of the mental footprint of the language. Thanks for listing the advantages. Sounds like it is worth the cost. It also really calls into question whether there are good reasons for other types to have a __str__ that is different than their __repr__. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] proto-pep: plugin proposal (for unittest)
Hello all, My apologies in advance if email mangles whitespace in the code examples. I can reformulate as a PEP if that is deemed useful and this document can be found online at: http://hg.python.org/unittest2/file/tip/description.txt (Please excuse errors and omissions - but do feel free to point them out.) This is a description, and request for feedback, of the unittest plugin system that I am currently prototyping in the plugins branch of unittest2_. My goal is to merge the plugin system back into unittest itself in Python 3.2. .. _unittest2: http://hg.python.org/unittest2 As part of the prototype I have been implementing some example plugins (in unittest2/plugins/) so I can develop the mechanism against real rather than imagined use cases. Jason Pellerin, creator of nose, has been providing me with feedback and has been trying it out by porting some of the nose plugins to unittest [#]_. He comments on the system it looks very flexible and clean. ;-) Example plugins available and included: * a pep8 and pyflakes checker * a debugger plugin that drops you into pdb on test fail / error * a doctest loader (looks for doctests in all text files in the project) * use a regex for matching files in test discovery instead of a glob * growl notifications on test run start and stop * filter individual *test methods* using a regex * load test functions from modules as well as TestCases * integration with the coverage module for coverage reporting In addition I intend to create a plugin that outputs junit compatible xml from a test run (for integration with tools like the hudson continuous integration server) and a test runner that runs tests in parallel using multiprocessing. Not all of these will be included in the merge to unittest. Which ones will is still an open question. I'd like feedback on the proposal, and hopefully approval to port it into unittest after discussion / amendment / completion. In particular I'd like feedback on the basic system, plus which events should be available and what information should be available in them. Note that the system is *not* complete in the prototype. Enough is implemented to get the general idea and to formalise the full system. It still needs extensive tests and the extra work in TestProgram makes it abundantly clear that refactoring there is well overdue... In the details below open questions and todos are noted. I *really* value feedback (but will ignore bikeshedding ;-) .. note:: Throughout this document I refer to the prototype implementation using names like ``unittest2.events.hooks``. Should this proposal be accepted then the names will live in the unittest package instead of unittest2. The core classes for the event system live in the current implementation in the ``unittest2.events`` namespace. Abstract unittest lacks a standard way of extending it to provide commonly requested functionality, other than subclassing and overriding (and reimplementing) parts of its behaviour. This document describes a plugin system already partially prototyped in unittest2. Aspects of the plugin system include: * an events mechanism where handlers can be registered and called during a test run * a Plugin class built over the top of this for easy creation of plugins * a configuration system for specifying which plugins should be loaded and for configuring individual plugins * command line integration * the specific set of events and the information / actions available to them As the plugin system essentially just adds event calls to key places it has few backwards compatibility issues. Unfortunately existing extensions that override the parts of unittest that call these events will not be compatible with plugins that use them. Framework authors who re-implement parts of unittest, for example custom test runners, may want to add calling these events in appropriate places. Rationale = Why a plugin system for unittest? unittest is the standard library test framework for Python but in recent years has been eclipsed in functionality by frameworks like nose and py.test. Among the reasons for this is that these frameworks are easier to extend with plugins than unittest. unittest makes itself particularly difficult to extend by using subclassing as its basic extension mechanism. You subclass and override behaviour in its core classes like the loader, runner and result classes. This means that where you have more than one extension working in the same area it is very hard for them to work together. Whilst various extensions to unittest do exist (e.g. testtools, zope.testrunner etc) they don't tend to work well together. In contrast the plugin system makes creating extensions to unittest much simpler and less likely that extensions will clash with each other. nose itself exists as a large system built over the top of unittest. Extending
Re: [Python-Dev] proto-pep: plugin proposal (for unittest)
Damn, the email was truncated. Probably my fault. The part missed off is: Not Yet Implemented === Except where noted, everything in this document is already working in the prototype. There are a few open issues and things still to be implemented. Certain event attributes should be read only (like extraTests, extraNames etc) because they should only be extended or modified in place instead of being replaced. Add an epilogue to the optparse help messages. A messaging API for plugins that respects verbosity. The ``startTestRun`` event needs an alternative way of replacing the default test run machinery without stopping other plugins from executing. It should only be possible for *one* plugin to use this. Should the ``onTestFailEvent`` be able to supress raised exceptions? It should also be able to modify the traceback so that bare asserts could be used but still provide useful diagnostic information. Should this event be fired for test skips? Command line options to unittest, like verbosity, should be put into the config data structure so that they are accessible to plugins. TestFailEvent needs to be fired on failures in setUpClass and setUpModule etc Plugin.register and Plugin.unregister aren't precisely complementary. This means that unregister can only be called once. This may not be an issue. Plugin.unregister will fail if any events have already been unhooked. This can easily be fixed. If the merge to unittest in Python 3.2 happens we need to decide which of the example plugins will be included in unittest. (They will probably all remain in unittest2.) Custom test outcomes are *not* supported; making interoperating tools or plugins in the presence of custom outcomes can be very hard. Should unittest2 have a different config file from unittest, so they can be configured side-by-side? (``unittest2.cfg`` perhaps.) Alternatively the same config file could be used with a '[unittest2]' section instead of '[unittest]'. Should ``StopTestEvent.timeTaken`` (plus startTime / stopTime etc) include the time for setUp, tearDown and cleanUps? Should this information be available separately perhaps? The discovery command line interface has changed a lot and the tests need reviewing to ensure they still provide full coverage. Should there be an excluded-plugins section in the config files, so projects can prevent incompatible plugins from being loaded? Should multiline values in config files be merged instead of overriding each other. (This is effectively what happens for the plugins list.) Should ``handleFile`` be fired when a test name is specified at the command line? (This would be 'tricky' as ``handleFile`` will have to be called for the containing module and then the correct name pulled out of the module.) Additional Changes == Alongside, and in support of, the plugin system a few changes have been made to unittest2. These either have no, or very minor, backwards compatibility issues. Changes so far are: TestLoader has a new attribute ``DEFAULT_PATTERN``. This is so that the regex matching plugin can change the default pattern used for test discovery when no pattern is explicitly provided. Command line parsing is all done by optparse, removing the use of getopt. This makes the help messages more consistent but makes the usage messages less useful in some situations. This can be fixed with the use of the optparse epilogue. unit2 (the default test runner) runs test discovery if invoked without any arguments. unit2 can execute tests in filenames as well as module names - so long as the module pointed to by the filename is importable from the current directory. FunctionTestCase.id() returns 'module.funcname' instead of just funcname. Added util.formatTraceback, the default way of formatting tracebacks. TestCase has a new formatTraceback method (delegating to util.formatTraceback). TestCase instances can implement formatTraceback to control how the traceback for errors and failures are represented. Useful for test items that don't represent Python tests, for example the pep8 / pyflakes checker and theoretical javascript test runners such as exist for py.test and nose. If you specify test names (modules, classes etc) at the command line they will be loaded individually using ``loader.loadTestsFromName`` instead of collectively with ``loader.loadTestsFromNames``. This enables individual names to be checked to see if they refer to filenames. References == .. [#] See http://bitbucket.org/jpellerin/unittest2/src/tip/unittest2/plugins/attrib.py and http://bitbucket.org/jpellerin/unittest2/src/tip/unittest2/plugins/errormaster.py .. [#] http://lists.idyll.org/pipermail/testing-in-python/2010-March/002799.html On 29/07/2010 23:55, Michael Foord wrote: Hello all, My apologies in advance if email mangles whitespace in the code examples. I can reformulate as a PEP if that is deemed useful and this
Re: [Python-Dev] Thoughts fresh after EuroPython
2010/7/29 Nick Coghlan ncogh...@gmail.com: On Thu, Jul 29, 2010 at 8:57 AM, Jesse Noller jnol...@gmail.com wrote: I thought at the last two pycons, we've all discussed that we should have a system in place for marking tests *and* modules within the stdlib as will only work on FooPython. I suspect that it's waiting on the shared-stdlib effort, which is waiting on mercurial (and time). @skipIf, @cpython_only and @test_impl_detail have been getting sprinkled fairly liberally throughout the test suite, so that part of the effort is already in progress. Note that as I port PyPy 2.7, I'm marking tests. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
2010/7/29 Raymond Hettinger raymond.hettin...@gmail.com: When you proposed the idea at EuroPython, it seemed reasonable but we didn't go into the pros and cons. The downsides include breaking tests, changing the output of report generating scripts that aren't using string formatting, and it introduces another inter-version incompatibility. Yes, I agree that the change has potential for breakage; it's a change that probably would have been unacceptable for Python 2.7; for Python 3.2 I think there's a little more scope, since 3.x has fewer users. +1 for making the change to 3.2 +0 for 2.7 -1 for 2.7. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Full unicode support for the import machinery
Le vendredi 09 juillet 2010 02:11:35, Victor Stinner a écrit : I'm trying to fix Python to support undecodable bytes in the Python path (...) My work is mostly done. I posted a patch on Rietveld and opened an issue. http://bugs.python.org/issue9425 http://codereview.appspot.com/1874048 -- Victor Stinner http://www.haypocalc.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
On Thu, Jul 29, 2010 at 3:30 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: It also really calls into question whether there are good reasons for other types to have a __str__ that is different than their __repr__. Maybe, but there is tons of 3rd party code that uses this distinction. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2
On Thu, Jul 29, 2010 at 6:30 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: .. It also really calls into question whether there are good reasons for other types to have a __str__ that is different than their __repr__. For strings, the distinction is very useful. In this and many other cases unifying str and repr would mean making a choice between readability and parseability. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
