Re: [pypy-dev] docs on doc.pypy.org
On Thu, May 5, 2011 at 11:00 PM, Daniel Collins dacja...@gmail.com wrote: Hello! After following the project for a long time, I just joined the pypy mailing list looking to help out. I would love to help on the pypy.org website; admittedly, I am fairly new to web design but I think I could help. I have never contributed to an open source project before so I need a little guidance about how things work here. Thanks, let me know! --Daniel It's very simple. You come up with an idea what do you want to work on (this is done, pypy website) so you need to check out an appropriate repository. http://bitbucket.org/pypy/pypy.org then you either create a branch and issue a pull request or create a patch or whatever way is really convinient. Feel free to join #pypy on irc.freenode.net if you have any more questions On Thu, May 5, 2011 at 1:39 PM, Joe qbpro...@gmail.com wrote: If you do a blog post asking for help someone would probably help out. Especially since they'd have pypy.org in their portfolio. It may not turn up anything, but worth a shot. Joe On Tue, May 3, 2011 at 1:08 PM, Leonardo Santagada santag...@gmail.com wrote: On Tue, May 3, 2011 at 1:50 PM, Laura Creighton l...@openend.se wrote: In a message of Tue, 03 May 2011 10:12:17 -, holger krekel writes: I'm not a sphinx expert, but I don't think that the default layout (i.e ., with a sidebar on the left) is the best for the home page of pypy.org. I think sphinx can be tuned to support pypy.org layout but i don't consider myself a sphinx expert either. Maybe someone feels like helping and trying with https://bitbucket.org/pypy/pypy.org/overview to put it into a sphinx format? holger Georg Brandl (and you don't get more expert than this) showed up in the channel to volunteer to give us a custom look. Now we only have to decide what it is that we want. I found this site themes to be really cheap and good looking: http://themeforest.net/category/site-templates/technology/software http://themeforest.net/category/site-templates/technology Either this or hiring someone I think is the best idea, pypy deserves a great looking website :) -- Leonardo Santagada ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] speed of sympy tests
On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik ond...@certik.cz wrote: Hi, I have downloaded the pypy 1.5 binary (with jit) and run tests (only sympy/core, so that it's fast) on Ubuntu Natty, 64bit: ondrej@eagle:~/repos/sympy(master)$ bin/test sympy/core/ = test process starts == executable: /usr/bin/python (2.7.1-final-0) ground types: python sympy/core/tests/test_arit.py[48] ...f.. .. [OK] sympy/core/tests/test_assumptions.py[28] f... [OK] sympy/core/tests/test_basic.py[10] .. [OK] sympy/core/tests/test_cache.py[1] . [OK] sympy/core/tests/test_complex.py[13] . [OK] sympy/core/tests/test_containers.py[5] . [OK] sympy/core/tests/test_count_ops.py[2] .. [OK] sympy/core/tests/test_diff.py[6] .. [OK] sympy/core/tests/test_equal.py[5] . [OK] sympy/core/tests/test_eval.py[8] .f.. [OK] sympy/core/tests/test_eval_power.py[12] [OK] sympy/core/tests/test_evalf.py[23] ... [OK] sympy/core/tests/test_expand.py[6] .. [OK] sympy/core/tests/test_expr.py[60] .. .. [OK] sympy/core/tests/test_exprtools.py[4] [OK] sympy/core/tests/test_facts.py[11] ... [OK] sympy/core/tests/test_functions.py[23] .f. [OK] sympy/core/tests/test_logic.py[11] ... [OK] sympy/core/tests/test_match.py[26] ...f.. [OK] sympy/core/tests/test_numbers.py[47] ... [OK] sympy/core/tests/test_operations.py[4] [OK] sympy/core/tests/test_priority.py[5] . [OK] sympy/core/tests/test_relational.py[7] ... [OK] sympy/core/tests/test_sets.py[18] .. [OK] sympy/core/tests/test_subs.py[30] .. [OK] sympy/core/tests/test_symbol.py[9] . [OK] sympy/core/tests/test_sympify.py[23] ... [OK] sympy/core/tests/test_truediv.py[3] ... [OK] sympy/core/tests/test_var.py[5] . [OK] === tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ondrej@eagle:~/repos/sympy(master)$ ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test sympy/core/ = test process starts == executable: /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy (2.7.1-final-42) ground types: python sympy/core/tests/test_arit.py[48] ...f.. .. [OK] sympy/core/tests/test_assumptions.py[28] f... [OK] sympy/core/tests/test_basic.py[10] .. [OK] sympy/core/tests/test_cache.py[1] . [OK] sympy/core/tests/test_complex.py[13] . [OK] sympy/core/tests/test_containers.py[5] . [OK] sympy/core/tests/test_count_ops.py[2] .. [OK] sympy/core/tests/test_diff.py[6] .. [OK] sympy/core/tests/test_equal.py[5] . [OK] sympy/core/tests/test_eval.py[8] .f.. [OK] sympy/core/tests/test_eval_power.py[12] [OK] sympy/core/tests/test_evalf.py[23] ... [OK] sympy/core/tests/test_expand.py[6] .. [OK] sympy/core/tests/test_expr.py[60] ..F... .. [FAIL] sympy/core/tests/test_exprtools.py[4] [OK] sympy/core/tests/test_facts.py[11] ...
Re: [pypy-dev] PyPy Assembler SQRT Patch
Hi. Sorry it took so long to review, long weekend and whatnot. The test for x86 backend actually fails for -1.0 on 64bit. Can you reproduce/deny? On Tue, Apr 26, 2011 at 11:51 PM, Joe qbpro...@gmail.com wrote: Attached is a patch to allow pypy to use SQRTSD rather than calling out to libc. This resulted in a 2x speedup, that scaled as the benchmark took longer. When i added another 0 to the end of the benchmark it was still a 2x speedup. benchmark results: http://paste.pocoo.org/show/378122/ I'll be happy to answer any questions about the patch. Joe ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] speed of sympy tests
On Tue, May 3, 2011 at 9:44 AM, Ondrej Certik ond...@certik.cz wrote: On Tue, May 3, 2011 at 12:31 AM, Maciej Fijalkowski fij...@gmail.com wrote: On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik ond...@certik.cz wrote: Hi, I have downloaded the pypy 1.5 binary (with jit) and run tests (only sympy/core, so that it's fast) on Ubuntu Natty, 64bit: ondrej@eagle:~/repos/sympy(master)$ bin/test sympy/core/ = test process starts == executable: /usr/bin/python (2.7.1-final-0) ground types: python sympy/core/tests/test_arit.py[48] ...f.. .. [OK] sympy/core/tests/test_assumptions.py[28] f... [OK] sympy/core/tests/test_basic.py[10] .. [OK] sympy/core/tests/test_cache.py[1] . [OK] sympy/core/tests/test_complex.py[13] . [OK] sympy/core/tests/test_containers.py[5] . [OK] sympy/core/tests/test_count_ops.py[2] .. [OK] sympy/core/tests/test_diff.py[6] .. [OK] sympy/core/tests/test_equal.py[5] . [OK] sympy/core/tests/test_eval.py[8] .f.. [OK] sympy/core/tests/test_eval_power.py[12] [OK] sympy/core/tests/test_evalf.py[23] ... [OK] sympy/core/tests/test_expand.py[6] .. [OK] sympy/core/tests/test_expr.py[60] .. .. [OK] sympy/core/tests/test_exprtools.py[4] [OK] sympy/core/tests/test_facts.py[11] ... [OK] sympy/core/tests/test_functions.py[23] .f. [OK] sympy/core/tests/test_logic.py[11] ... [OK] sympy/core/tests/test_match.py[26] ...f.. [OK] sympy/core/tests/test_numbers.py[47] ... [OK] sympy/core/tests/test_operations.py[4] [OK] sympy/core/tests/test_priority.py[5] . [OK] sympy/core/tests/test_relational.py[7] ... [OK] sympy/core/tests/test_sets.py[18] .. [OK] sympy/core/tests/test_subs.py[30] .. [OK] sympy/core/tests/test_symbol.py[9] . [OK] sympy/core/tests/test_sympify.py[23] ... [OK] sympy/core/tests/test_truediv.py[3] ... [OK] sympy/core/tests/test_var.py[5] . [OK] === tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ondrej@eagle:~/repos/sympy(master)$ ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test sympy/core/ = test process starts == executable: /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy (2.7.1-final-42) ground types: python sympy/core/tests/test_arit.py[48] ...f.. .. [OK] sympy/core/tests/test_assumptions.py[28] f... [OK] sympy/core/tests/test_basic.py[10] .. [OK] sympy/core/tests/test_cache.py[1] . [OK] sympy/core/tests/test_complex.py[13] . [OK] sympy/core/tests/test_containers.py[5] . [OK] sympy/core/tests/test_count_ops.py[2] .. [OK] sympy/core/tests/test_diff.py[6] .. [OK] sympy/core/tests/test_equal.py[5] . [OK] sympy/core/tests/test_eval.py[8] .f.. [OK] sympy/core/tests/test_eval_power.py[12] [OK] sympy/core/tests/test_evalf.py[23] ... [OK] sympy/core/tests/test_expand.py[6] .. [OK] sympy/core/tests/test_expr.py[60] ..F
Re: [pypy-dev] speed of sympy tests
On Tue, May 3, 2011 at 3:18 PM, Maciej Fijalkowski fij...@gmail.com wrote: On Tue, May 3, 2011 at 9:44 AM, Ondrej Certik ond...@certik.cz wrote: On Tue, May 3, 2011 at 12:31 AM, Maciej Fijalkowski fij...@gmail.com wrote: On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik ond...@certik.cz wrote: Hi, I have downloaded the pypy 1.5 binary (with jit) and run tests (only sympy/core, so that it's fast) on Ubuntu Natty, 64bit: ondrej@eagle:~/repos/sympy(master)$ bin/test sympy/core/ = test process starts == executable: /usr/bin/python (2.7.1-final-0) ground types: python sympy/core/tests/test_arit.py[48] ...f.. .. [OK] sympy/core/tests/test_assumptions.py[28] f... [OK] sympy/core/tests/test_basic.py[10] .. [OK] sympy/core/tests/test_cache.py[1] . [OK] sympy/core/tests/test_complex.py[13] . [OK] sympy/core/tests/test_containers.py[5] . [OK] sympy/core/tests/test_count_ops.py[2] .. [OK] sympy/core/tests/test_diff.py[6] .. [OK] sympy/core/tests/test_equal.py[5] . [OK] sympy/core/tests/test_eval.py[8] .f.. [OK] sympy/core/tests/test_eval_power.py[12] [OK] sympy/core/tests/test_evalf.py[23] ... [OK] sympy/core/tests/test_expand.py[6] .. [OK] sympy/core/tests/test_expr.py[60] .. .. [OK] sympy/core/tests/test_exprtools.py[4] [OK] sympy/core/tests/test_facts.py[11] ... [OK] sympy/core/tests/test_functions.py[23] .f. [OK] sympy/core/tests/test_logic.py[11] ... [OK] sympy/core/tests/test_match.py[26] ...f.. [OK] sympy/core/tests/test_numbers.py[47] ... [OK] sympy/core/tests/test_operations.py[4] [OK] sympy/core/tests/test_priority.py[5] . [OK] sympy/core/tests/test_relational.py[7] ... [OK] sympy/core/tests/test_sets.py[18] .. [OK] sympy/core/tests/test_subs.py[30] .. [OK] sympy/core/tests/test_symbol.py[9] . [OK] sympy/core/tests/test_sympify.py[23] ... [OK] sympy/core/tests/test_truediv.py[3] ... [OK] sympy/core/tests/test_var.py[5] . [OK] === tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ondrej@eagle:~/repos/sympy(master)$ ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test sympy/core/ = test process starts == executable: /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy (2.7.1-final-42) ground types: python sympy/core/tests/test_arit.py[48] ...f.. .. [OK] sympy/core/tests/test_assumptions.py[28] f... [OK] sympy/core/tests/test_basic.py[10] .. [OK] sympy/core/tests/test_cache.py[1] . [OK] sympy/core/tests/test_complex.py[13] . [OK] sympy/core/tests/test_containers.py[5] . [OK] sympy/core/tests/test_count_ops.py[2] .. [OK] sympy/core/tests/test_diff.py[6] .. [OK] sympy/core/tests/test_equal.py[5] . [OK] sympy/core/tests/test_eval.py[8] .f.. [OK] sympy/core/tests/test_eval_power.py[12] [OK] sympy/core/tests/test_evalf.py[23] ... [OK] sympy/core/tests/test_expand.py[6] .. [OK] sympy/core/tests/test_expr.py[60] ..F
Re: [pypy-dev] Problem with large allocation in test
On Tue, Apr 19, 2011 at 12:59 AM, p...@pocketnix.org wrote: On Mon, Apr 18, 2011 at 06:25:44PM -0400, Joe wrote: I was trying to run the test file: pypy/jit/backend/x86/test/test_rx86_64_auto_encoding.py and was getting the following traceback: http://paste.pocoo.org/show/374129/ If you look at the comment on line 17, it's trying to allocate much more memory than I have. I think it's a total of 21GB, while I only have 4GB. I'm using 64bit OpenSuSE 11.4 for my operating system. I had the kernel setting overcommit_memory set to 0 (which may be part of the problem). Anyway, after I went into ll2ctypes.py and set far_regions to True, I was able to successfully run the original test. I don't think setting far_regions to True is the correct solution to the problem, but fiddling with kernel settings on my system is not ideal either. What would be a better overall solution? If any clarification is needed let me know, Joe your vm.overcommit_ratio should be set to 50 or 50% by default, as per http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/vm/overcommit-accounting;hb=HEAD this means that any allocations of 6GB will automatically be rejected as not sane and you should receive the ENOMEM error indicating the kernel cannot satisfy the supplied range there are a couple of ways to fix this * don't allocate so much ram (did anyone test this before on a 64bit host on linux) The reason for this is that we want to test far jumps (exceeding 4G or 2^32 in address space). How can you do it otherwise? * change the vm overcommit policy to 1 (allow everything, don't perform sanity checks) * change the overcommit ratio to something that will satisfy the allocation (20GB/4GB ~= 5x, so a value of 600% or 600 should do it) * Make the mapping a rmmap.MAP_PRIVATE and rmmap.PROT_READ only, depending on what you are testing this may not be useful as indicated in the linked kernel documentation YYMV, first option is safe, 2nd option you may want to double check the documentation and the values you are passing, the 3rd option is also safe in that it will allow badly behaved apps to run, not prevent apps from running to change either of these values use the following: * to adjust the allocation policy: sysctl vm.overcommit_memory=val * to adjust the ratio: sysctl vm.overcommit_ratio=val to print the current values (and save them for restoring them after you have done tweaking them: sysctl vm.overcommit_memory or sysctl vm.overcommit_ratio Hope this is whats causing the issue Da_Blitz ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Ignore 'pinsrb/w/d' instructions in trackgcroot
thanks, commited! On Sun, Apr 17, 2011 at 10:03 PM, Matthew Woodcraft matt...@woodcraft.me.uk wrote: I found I needed the following patch in order to run translation with gcc 4.6 and -march=corei7. -M- --- a/pypy/translator/c/gcc/trackgcroot.py +++ b/pypy/translator/c/gcc/trackgcroot.py @@ -456,7 +456,7 @@ class FunctionGcRootTracker(object): 'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc', 'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv', 'bswap', 'bt', 'rdtsc', - 'punpck', 'pshufd', 'pcmp', 'pand', 'psllw', 'pslld', 'psllq', + 'punpck', 'pshufd', 'pcmp', 'pand', 'psllw', 'pslld', 'psllq', 'pinsr', # zero-extending moves should not produce GC pointers 'movz', ]) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] pypy default: fixed test_circular
On Fri, Apr 15, 2011 at 10:38 AM, Antonio Cuni anto.c...@gmail.com wrote: Right. My point was that since we dont care if they are there or not the test should not test that they are there and fail if they are not. So if there is an easy way to ignore them in this new test_pypy_c framework (which is very cool by the way), we should. If it's not easy I'm fine with keeping the test as it is. My main motivation here is to learn about the new framework :) ah, I understand now. No, ignoring all force_tokens at once is not possible at the moment, but I agree that it would be a nice feature, I think I'll implement it later. Btw, I fear I need more of your help with test_silly_max and test_iter_max (see 2e5bd737be0c): what do we want to check there? ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev Note that force_token operation is really cheap in the backend. It also does not use a whole lot of space. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] project infrastructure issues
On Fri, Apr 15, 2011 at 9:42 PM, holger krekel hol...@merlinux.eu wrote: On Thu, Apr 14, 2011 at 21:01 -0400, Alex Gaynor wrote: On Thu, Apr 14, 2011 at 1:47 PM, holger krekel hol...@merlinux.eu wrote: Hey all, now that pypy's codespeak subversion usage is basically gone i'd like to push for remaining issues related to the pypy infrastructure: - apache/website - buildbot/master - roundup/issue tracker - mailman/mailing lists pypy-dev/commits/z Which of them shall we try to move elsewhere? My preliminary suggestion: - website - readthedocs? or other site - buildbot - python.org? or other site - issue tracker - bitbucket issue tracker - mailing lists - google groups or python.org or other site The other site could be a host that i anyway need to have for remaining codespeak and merlinux stuff and which thus is somewhat guaranteed to work mail- and otherwise. Other people could get admin access as well, of course. any suggestions or comments? best, holger ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev readthedocs seems like the right solution for docs, should just be a matter of setting up the post-commit hook and adding a cname for docs.pypy.org That would still leave open the question of pypy.org itself i guess. besides, make in pypy/doc spews out a lot of errors and warnings for me. Do you know if anybody is caring for completing the transition to sphinx? holger I think laura does. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Possible sprint in Genova before/after Europython
On Wed, Apr 13, 2011 at 12:20 PM, Antonio Cuni anto.c...@gmail.com wrote: Hi all, as we were discussing yesterday in another thread, the post-europython sprint will be only two days long, and so we might want to have a longer one either before or after europython. I am considering organizing one in my place. It could be either in Genova or more preferably in some other town in the nearby of the italian riviera. The place is ~3hr away from Florence by train. Googling images for pegli or arenzano (i.e., two of the towns we could go to) shows pictures which (I think) should encourage people to come :-) http://tinyurl.com/655p2at http://tinyurl.com/67q67r3 My idea is to find a hotel that gives us a sprinting room and internet in exchange of N people sleeping there, as we do in leysin. But before asking them, I need to have a rough idea about which value of N we are talking about (of course the higher the more interesting for them it is). Thus, I ask everybody who is potentially/likely interested in coming to let me know. Also, would you prefer to do it before or after europython? ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev Not that I can't google myself, but those links are broken ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] Waf benchmark
Hello. I propose the waf benchmark removal. Originally, the idea was that we're slower than CPython for no good reason. Now that this benchmark measures some obscure piece of stdlib time (subprocesses) I don't think it's that necessary. Besides: * the variation between runs is too big, so we don't care * noone was ever remotely interested in speeding this up any opinions? Cheers, fijal ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] An interesting article on chip design
For those interested in hardware/assembler. http://www.lighterra.com/papers/modernmicroprocessors/ It's a good read and fills some of our gaps :) Cheers, fijal ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Fri, Apr 8, 2011 at 7:43 PM, Isaac Gouy igo...@yahoo.com wrote: The benchmarks game web pages now only show one language implementation for each programming language. Java -Xint, Tracemonkey JavaScript, LuaJIT, CPython, Iron Python, PyPy, Ruby 1.8.7 and JRuby 1.6 are no longer shown. As I understand this is CPython 3.2 for Python right? anyway, what's the point of the above discussion then? ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Fri, Apr 8, 2011 at 8:30 PM, Isaac Gouy igo...@yahoo.com wrote: --- On Fri, 4/8/11, Maciej Fijalkowski fij...@gmail.com wrote: The benchmarks game web pages now only show one language implementation for each programming language. Java -Xint, Tracemonkey JavaScript, LuaJIT, CPython, Iron Python, PyPy, Ruby 1.8.7 and JRuby 1.6 are no longer shown. As I understand this is CPython 3.2 for Python right? CPython 2.7 is no longer shown - 3.2 is still shown. anyway, what's the point of the above discussion then? The point of the discussion was to hear the views of pypy-dev, and I have. I guess a lot of discussions are about getting some sort of consensus. I see this one is so you can know what we think and that's it. Well, that comes as a bit of surprise. I think it's super stupid to remove Tracemonkey, LuaJIT and PyPy from it, but that's as you pointed out *your* website. On the other hand it's good, because people won't cite the computer language shootout anymore and those benchmarks are more silly than they have to be. Farewell. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Fri, Apr 8, 2011 at 9:03 PM, Isaac Gouy igo...@yahoo.com wrote: --- On Fri, 4/8/11, Maciej Fijalkowski fij...@gmail.com wrote: -snip- I guess a lot of discussions are about getting some sort of consensus. I see this one is so you can know what we think and that's it. Well, that comes as a bit of surprise. I don't know why that would come as any surprise - Of course, I'll make up my own mind but at least I'll be able to take your wishes into account. http://permalink.gmane.org/gmane.comp.python.pypy/7303 I think it's super stupid to remove Tracemonkey, LuaJIT and PyPy from it, but that's as you pointed out *your* website. On the other hand it's good, because people won't cite the computer language shootout anymore and those benchmarks are more silly than they have to be. You express both of the contrary wishes that I've heard here this week - you seem to want yes and no :-) No. I think it's bad for you good for us. fwiw someone did write - While a comparison between languages may be interesting, maybe having 1 implementation per language in the shootout would work better. - let's hope at least they are happy now. I sure hope so. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Wed, Apr 6, 2011 at 1:22 AM, Aaron DeVore aaron.dev...@gmail.com wrote: On Tue, Apr 5, 2011 at 2:49 PM, Joe qbpro...@gmail.com wrote: While I spent my Saturday trying to make PyPy look better in the language shootout, I'm leaning towards taking it out. While a comparison between languages may be interesting, maybe having 1 implementation per language in the shootout would work better. Then there is one target to optimize for. PyPy and CPython have very different performance characteristics. I feel as though speed.python.org may be a better venue for comparing python implementations. Since the pypy developers have closer ties to the Python Core developers and it's been stated they'll have influence. It can be made to be fair for all parties involved. Since all parties will likely be python implementations they can all agree on one implementation and use that. Joe I heavily recommend keeping PyPy in the Shootout in stome form. Even with the Shootout's flaws, it is nice to have a general idea of how PyPy compares to both CPython and other languages. We don't get that information now at least, since those benchmarks are badly skewed towards CPython. I know how hard is to find out a reasonable set of benchmarks and how to keep them balanced. I have another issue with ctypes numpy. This is that C implementations are allowed to use gcc-specific hacks and non-standard libraries (apache malloc for gcbench). Why wouldn't we be allowed to do the same then? I would like to have PyPy included, but I would also like the benchmark game to be fair or as close to fair as we can get. Having benchmarks with different rules for different languages doesn't seem to be quite fair in my opinion. Note that this is up to discussions - I'm fine with saying numpy ctypes are disallowed or having a separate category python + numpy with both CPython and PyPy (once we get numpy). -Aaron DeVore ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Wed, Apr 6, 2011 at 3:15 PM, Isaac Gouy igo...@yahoo.com wrote: --- On Wed, 4/6/11, Maciej Fijalkowski fij...@gmail.com wrote: -snip- I have another issue with ctypes numpy. This is that C implementations are allowed to use gcc-specific hacks and non-standard libraries (apache malloc for gcbench). Why wouldn't we be allowed to do the same then? Would you describe C as batteries included ? nope, I would not. However, ctypes come included in standard python distribution (unlike numpy or gmpy) so this is not really a valid argument. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Wed, Apr 6, 2011 at 3:08 PM, Isaac Gouy igo...@yahoo.com wrote: --- On Wed, 4/6/11, Maciej Fijalkowski fij...@gmail.com wrote: -snip- We don't get that information now at least, since those benchmarks are badly skewed towards CPython. I know how hard is to find out a reasonable set of benchmarks and how to keep them balanced. Do you mean the program you contributed is badly skewed towards CPython? http://shootout.alioth.debian.org/u32/program.php?test=nbodylang=pypyid=1 No Do you mean that the n-body problem is badly skewed towards CPython? No, that would be nonsense. I would never discuss whether those benchmarks does represent typical workflow in language X because it's impossible to find such a set that's true for every X. I never did discuss the choice of problems. Your PyPy program is shown as so much faster - how is that badly skewed towards CPython? That's true, but that's one that got through. For example reverse complement (the current version) is skewed towards CPython. I'm fine with saying that ctypes (or numpy) are not allowed, with a good explanation (and maybe an explanation why custom malloc library is allowed for C and gcbench). Another question which was raised - are programs that only work on PyPy allowed? (Due to pypy's extensions or cpython bugs). Since programs that only compile on GCC clearly are. Cheers, fijal ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Wed, Apr 6, 2011 at 5:18 PM, Isaac Gouy igo...@yahoo.com wrote: --- On Wed, 4/6/11, Maciej Fijalkowski fij...@gmail.com wrote: -snip- Do you mean the program you contributed is badly skewed towards CPython? http://shootout.alioth.debian.org/u32/program.php?test=nbody〈=pypyid=1 No Do you mean that the n-body problem is badly skewed towards CPython? No, that would be nonsense. I would never discuss whether those benchmarks does represent typical workflow in language X because it's impossible to find such a set that's true for every X. I never did discuss the choice of problems. Your PyPy program is shown as so much faster - how is that badly skewed towards CPython? That's true, but that's one that got through. I don't see how the program you contributed could be described as one that got through. Here's what happened - I noticed the n-body program failed with PyPy, I asked you guys about the problem and was told we have nbody_modified in our benchmarks and then I asked you guys to contribute your modified program. http://morepypy.blogspot.com/2010/03/introducing-pypy-12-release.html Once you contributed the program modified for PyPy it was displayed on the website within 3 hours. Cool, thank you. For example reverse complement (the current version) is skewed towards CPython. If only someone could manage to write a reverse complement skewed towards PyPy without using libc.write ;-) Deal, we'll do that. I'm fine with saying that ctypes (or numpy) are not allowed, with a good explanation (and maybe an explanation why custom malloc library is allowed for C and gcbench). Another question which was raised - are programs that only work on PyPy allowed? (Due to pypy's extensions or cpython bugs). PyPy extensions - No. CPython bugs - How strange that the CPython bug was never mentioned! - maybe. Ok. The bug was not mentioned because it takes time to decide it's a bug. Since programs that only compile on GCC clearly are. How many C language implementations are shown? How many Python language implementations are shown? If only one Python language implementation was shown do you think it would be PyPy ? I can't really read your mind, but from my opinion if the question is how fast Python programs can be run then the answer is Yes. So the position is that GCC is allowed to use extensions because it's the only C implementation shown and PyPy is not, because all Python programs should run on each runtime, is that correct? I'm not stating an opinion about it, I just want to know what the rules are. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Wed, Apr 6, 2011 at 7:33 PM, Isaac Gouy igo...@yahoo.com wrote: --- On Wed, 4/6/11, Maciej Fijalkowski fij...@gmail.com wrote: -snip- CPython bugs - How strange that the CPython bug was never mentioned! - maybe. Ok. The bug was not mentioned because it takes time to decide it's a bug. I know someone decided it's a bug because someone said so in a blog post they pushed out across the blogosphere and proggit and Hacker News and ... How strange that CPython bug was never mentioned to me! Since programs that only compile on GCC clearly are. How many C language implementations are shown? How many Python language implementations are shown? If only one Python language implementation was shown do you think it would be PyPy ? I can't really read your mind, but from my opinion if the question is how fast Python programs can be run then the answer is Yes. The Help page says something about showing working programs written in less familiar programming languages. Ok, will look it up later. So the position is that GCC is allowed to use extensions because it's the only C implementation shown and PyPy is not, because all Python programs should run on each runtime, is that correct? I don't see a way to compare CPython and PyPy unless the comparison programs do at least run on both CPython and PyPy (and x86 and x64). Well I can always write a program that runs on both (and uses more efficient data structure for pypy for example). The question is a bit academic, because I don't have any particular implementation now in mind. But would be cool to be able to do that. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy in the benchmarks game - yes or no?
On Thu, Apr 7, 2011 at 4:31 AM, Leonardo Santagada santag...@gmail.com wrote: On Wed, Apr 6, 2011 at 2:33 PM, Isaac Gouy igo...@yahoo.com wrote: So the position is that GCC is allowed to use extensions because it's the only C implementation shown and PyPy is not, because all Python programs should run on each runtime, is that correct? I don't see a way to compare CPython and PyPy unless the comparison programs do at least run on both CPython and PyPy (and x86 and x64). I don't see why this is, it is the same as comparing python to ruby, I want to see how fast can you make a program in said vm that does the same task. If the description of the problem doesn't limit what you can use I really don't see why can't you use a PyPy (or cpython) extension for it. For me a shootout without extensions (at least without numpy) is just comparing how fast a language can do without anything, which is not interesting at all. One where c programs can use libraries but python cannot is even more meaningless. Leonardo please calm down a bit. I can see reasons and why I might not agree with that, they do make sense and can be justified. It does make sense to compare CPython and PyPy on the same set of benchmarks (actually that's what we do with speed.pypy.org, we deliberately tried to avoid modifying benchmarks). -- Leonardo Santagada ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] The JVM backend and Jython
On Wed, Mar 30, 2011 at 12:11 PM, Antonio Cuni anto.c...@gmail.com wrote: On 30/03/11 19:37, fwierzbi...@gmail.com wrote: My thoughts here are taking a very primitive step - that is run the JVM translation and look at the generated Java - then see what needs to be modified so that I could use the generated Java parser from Jython. At this stage I would be using PyPy exactly the way I use ANTLR now - as a parser generator. There wouldn't be any need at all for calling into Java code (as far as I can think of). yes, I think it makes sense. Actually, as Leonardo says we don't generate java code but assembler which is converted to .class by jasmin. However, it should not change anything. I think if we Jython developers get some experience with PyPY - we might be able to help with the task of calling into Java from PyPy - since we know a bit about that :) that would be extremely cool :-) Ok, so if Ademan tells me that he's not going to work on the ootype-virtualref branch, I'll try to finish the work so you can start playing with it. Note to frank: this is kind of cool but only needed for the JIT, otherwise it's a normal reference. ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] The JVM backend and Jython
On Thu, Mar 31, 2011 at 4:02 PM, Antonio Cuni anto.c...@gmail.com wrote: On 31/03/11 21:57, Maciej Fijalkowski wrote: Ok, so if Ademan tells me that he's not going to work on the ootype-virtualref branch, I'll try to finish the work so you can start playing with it. Note to frank: this is kind of cool but only needed for the JIT, otherwise it's a normal reference. well, no. Virtualrefs were introduced for the JIT, but they also need to be supported by normal backends. This is why translation is broken at the moment. It is true that the implementation is straightforward, though (I suppose this is what you meant originally :-)) Sure. I was mostly saying the complex part of the implementation for ootype can be ommited if we skip the JIT part. ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Google Summer of Code Idea Proposal
On Sun, Mar 27, 2011 at 2:07 PM, Romain Guillebert romain...@gmail.com wrote: Hi I'm interested in doing the Summer of code on PyPy, from what I saw on the mailing list and on irc, I would like to work on 2 things which might interest you (there is no order of preference). * Python backend for Cython (to interface with C code, this could use use ctypes or an API exposed by PyPy), the backend would either produce python code or python bytecode and will allow PyPy to JIT the Cython extension (I talked about it with Alex Gaynor yesterday). * Improve the JVM backend, making it translate on the trunk and interface (R)Python code with Java. (As proposed by Antonio Cuni). I'm available from the end of May (around the 25th) to the beginning of September so this shouldn't be a problem. If this is OK, which one would you prefer ? Hey. Personally I would vastly prefer the first one. It's more immediately usable. We do require people submitting proposals to contribute some code (like fixing a small issue) first before being accepted. Anywhere you would like to contribute something? Feel free to pop up on IRC and ask people around what's interesting. Cheers, fijal Regards Romain ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Ctypes module written in RPython
On Thu, Mar 24, 2011 at 3:14 PM, Alex Perry alex.pe...@pamurray.com wrote: I can't find it in the docs, but it has been alluded to in the past. How far is the project from being able to compile a rpython module? I'd expect that to emit a wrapper that imports ctypes and declares calls into a shared library, and a directory of C code which implements the internals and can compile into the expected shared library. Hi. I'm a bit confused what you're trying to achieve. re compiler is not RPython, however the sre module (which runs the regular expression is). Regular expressions are kind of fast and they can be improved in places where JIT don't run, but in general re module is faster than the equivalent written in RPython, because it's jitted. I completely don't follow the second part of your mail. Again: what are you trying to achieve? ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] status of the graphviz server on codespeak?
On Wed, Mar 23, 2011 at 2:36 AM, Laura Creighton l...@openend.se wrote: In a message of Wed, 23 Mar 2011 09:02:25 +0100, Antonio Cuni writes: On 23/03/11 02:36, Maciej Fijalkowski wrote: On Tue, Mar 22, 2011 at 7:17 PM, Laura Creighton l...@openend.se wrote : getting-started-dev says: Download and install `Dot Graphviz`_ (optional if you have an internet connection: the flowgraph viewer then connects to codespeak.net and lets it convert the flowgraph by a graphviz serve r). What's going to happen to that when codespeak goes away? This one is defunct for ages now I think. no, I think it's still up: http://codespeak.net/pypy/convertdot.cgi we can either migrate it to some other machine (tannit?) or discontinue t he service. Nowadays, it's only useful for us developers, and (I think) we a ll have graphviz installed locally. I think discontinuation is the way to go. Running on tannit is a bad idea, since it would impact our benchmarks, but running someplace else is a possibility. I don't think it's worth it. I assumed it doesn't work because it never worked for me :) Crash on dot also crashed later on trying to run on codespeak. I would say just remove Laura ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Implementing CAS
On Tue, Mar 22, 2011 at 3:14 PM, Benjamin Peterson benja...@python.org wrote: 2011/3/22 Timothy Baldridge tbaldri...@gmail.com: After my last discussion about getting some multi-threading primitives, I took a look at how locks are implemented in PyPy. PyPy currently uses OS level mutexes...which is okay for most uses, but in my case, I need super fast spinlocks, and for that I need a CAS operation. What I'm looking for is a bit of direction on how to go about implementing this function (shown in C syntax): int cas(*expected, *new, **location) I suggest you create a new low-level operation. Then the C backend can translate it into asm. Which requires touching multiple places, but it's relatively easy. Start with writing a test and then add it to pypy/rpython/lltypesystem/lloperation.py -- Regards, Benjamin ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] status of the graphviz server on codespeak?
On Tue, Mar 22, 2011 at 7:17 PM, Laura Creighton l...@openend.se wrote: getting-started-dev says: Download and install `Dot Graphviz`_ (optional if you have an internet connection: the flowgraph viewer then connects to codespeak.net and lets it convert the flowgraph by a graphviz server). What's going to happen to that when codespeak goes away? This one is defunct for ages now I think. Laura ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] ideas for Google Summer of code
On Thu, Mar 17, 2011 at 12:00 PM, Danilo Freitas dsurvi...@gmail.com wrote: Hi, all. I'm interested in applying for GSoC this year. I'm talking to Miquel Torres about some stuff in Codespeed, but I don't know if it could be considered as PyPy project for GSoC. We're trying to allow Codespeed branch comparing, to check if a feature branch is getting faster than trunk. So, we'll see if a feature is really evolving. This would also affect speed.pypy.org. After that, we shall work on more stuff. So, could Codespeed improvements be considered as PyPy GSoC projects? Laura, I'm from Brazil and was a GSoC student in 2009 for Cython. I had about only 1 month free of college (June~July), but I completed what I promised without problems caused by college. So, I guess people from south hemisphere can handle with it, if they dedicate themselves :) Hi. That would definitely be considered PyPy project. One ideas that I have in mind is to create speed.python.org - a place where a whole lot of different implementations can be run. This requires improvements to both our benchmark infrastructure and codespeed itself. 2011/3/17 Baiju M baiju.m.m...@gmail.com: On Thu, Mar 17, 2011 at 1:23 PM, Leonardo Santagada santag...@gmail.com wrote: My ideas, take the ones you guys like and don't bother if some are too hard or the pypy team is not interested: 1 - Some pypy compatibility site, like the one brett made for python 3 I am interested to set up a site for PyPY. I have already created a similar site for Python 3: http://getpython3.net/ If this idea sound good, you can add one DNS A record pointing to this IP: 184.106.69.139 A sub-domain like http://compatibility.pypy.org/ would be fine. Otherwise you can provide me a hosting place also. BTW, the code is here: https://github.com/baijum/getpython3 (Flask app) Regards, Baiju M ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev -- - Danilo Freitas ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] translation with --stackless option
On Sat, Mar 12, 2011 at 9:11 PM, Hervé Coatanhay herve.coatan...@gmail.com wrote: After a clone of the repository, whereas i can do the following translation with just some warnings: python translate.py --opt=jit targetpypystandalone.py I got an exception with that one: python translate.py --stackless targetpypystandalone.py As the --stackless option is not detailed in documentation maybe I should had something along with it ? I tried this on both Mac OS X and a CentOS Linux box, the attach file contains the traceback of the translation. Any idea on what i am doing wrong ? It's broken. We should fix it some time, preferably soon. That said, there is a feature missing which is stackless working with JIT. We're not actively working on it but this is something that would be a useful contribution to the project and not too much work. But yes, we should fix the failure first,. -- Hervé Coatanhay ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Change to the frontpage of speed.pypy.org
On Wed, Mar 9, 2011 at 8:19 AM, Massa, Harald Armin c...@ghum.de wrote: I really, really like the new display! And it motivated me to dig into the data ... which is a great result on its own. The first question for myself was hey, why is it slow on slowspitfire, and, btw, what is slowspitfire? Could that be something that my application does, too? But I was unable to find out what slowspitfire is doing ... I found spitfire, which does some HTML templating stuff, and deducted, that slowspitfire will do some slow HTML templating stuff. Where did I click wrong? Is there a path down to the slowspitfire.py file or an explanation what slowspitfire is doing? Harald https://bitbucket.org/pypy/benchmarks/src/b93caae762a0/unladen_swallow/performance/bm_spitfire.py It's creating a very large template table (1000x1000 elements I think) The explanation why it's slow is a bit longish. It's a combination of factors, including very long lists with GC objects in it, using ''.join(list) instead of cStringIO (the latter is faster and yes, it is a bug) and a bit of other factors. -- GHUM GmbH Harald Armin Massa Spielberger Straße 49 70435 Stuttgart 0173/9409607 Amtsgericht Stuttgart, HRB 734971 - persuadere. et programmare ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Change to the frontpage of speed.pypy.org
Hey Miquel. A small feature request ;-) Can we get favicon? Cheers, fijal ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Change to the frontpage of speed.pypy.org
On Tue, Mar 8, 2011 at 8:14 AM, Laura Creighton l...@openend.se wrote: In a message of Tue, 08 Mar 2011 09:10:32 +0100, Miquel Torres writes: Hi, I finished the changes to the speed.pypy.org home page last night, but alas!, I didn't have time to deploy. I will do it later today and will then ping you back. The extra info provided is really nice as an overview, you will see ;-) Ah good. Thank you very much. We spent yesterday afternoon with the Mozilla engineers, and I got to talk to the person who maintains the benchmarks for tracemonkey. He had timelines very much like ours. There is one feature he has that I would like to have. Take a look at the timeline for spectral.norm. There are two spikes there. Mozilla has lines like that too, though mostly it is because their jit decides that the whole benchmark is bogus and optimises out all the code. So it takes 0 time. oops. At any rate, aside from knowing that something went horribly wrong with that rev, you don't really need to know how wrong. And by making the graph display up to that point means that the dots where things really do matter get crammed closer together than would otherwise be the case. So he had a mode where things wehre displayed with an arbitrary value at the bottom (in our coase it would be the top) which he could specify. Then the graph would be replotted, with the outliers off the graph, but making it easier to read the dots for the more normal cases. Any chance we could do that too? Link maybe? Laura ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] GIL in non-Python languages
On Mon, Mar 7, 2011 at 9:08 AM, Timothy Baldridge tbaldri...@gmail.com wrote: Right, that would be an issue. In this case, I could probably start with a reference counting GCas I'm not exactly sure that it's possible to get cyclic references with immutable objects. At least not without allot of work...yeah, I'm pretty sure they are impossible to code in Clojure. But someone may correct me there. Hey. Refcounting GC would be generated for you, have you chosen to use it, but it'll be very inefficient and probably not thread safe. What you can do *right now* is to use boehm GC which is thread safe, but not very fast (much faster and much better than refcounting though). I don't think there are any other issues that are not thread safe in RPython than GC. Obviously it's like C - if you do something wrong you'll segfault. However, the true answer would be to work on a real GC that thread-friendly and thread-safe. This is work, it's however fun :-) Anyway, reference counting would get rid of most of the parallel issues with the GC, I could simply use a CAS or atomic instructions to increment/decrement reference counts. Timothy One major problem is that RPython its self is not really thread-safe. For example, the gc is very non-concurrent. So, that would have to be fixed. -- Regards, Benjamin -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] extradoc moved / pypy-z moved / new pypy groups
On Tue, Mar 1, 2011 at 8:42 PM, holger krekel hol...@merlinux.eu wrote: Hi all, I just turned http://codespeak.net/svn/pypy/extradoc as well as the internal pypy-z repository READ-ONLY. Thanks to Ronny's conversion work we now have these repositories at bitbucket.org: http://bitbucket.org/pypy/extradoc talks, sprintinfo, the source of pypy.org website and http://bitbucket.org/pypy/z (PRIVATE) internal contracts/docs, available to long-time contributors I created a new pypy group on bitbucket, the pypy:committers group which has general read/write access to all public pypy repositories. It contains the people formerly listed individually for the pypy repo. New committers should be added to that pypy:committers group so they get access to all pypy repos. There also is a pypy:z group which also gets write/admin acccess to all repos but additionally can read/write the pypy/z one. Next in line for conversion are: pyrepl - pypy/pyrepl (discussed with Michael Hudson, the primary author already) testrunner - to be flatly inlined into pypy repo and then the remaining parts of pypy/lang What about sqlite3? After that there should be nothing left from PyPy that would require codespeak's svn. best, holger ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Change to the frontpage of speed.pypy.org
Hey. Just my 5 cents. It would be cool if default view has a down-scaled version of comparison against CPython. I can look anywhere for recent changes. Also the recent changes as they're now are not very informative and I don't use them at all. They stick around, so I don't know if they're new or old. I'm also as interested in good as in bad changes. Simply this: http://speed.pypy.org/changes/ is way more informative. Can we either just remove the red recent changes for now or simply put a vs cpython, scaled down graph there? At least for pycon this seems like a better way to go. Cheers, fijal PS. Miquel, don't get me wrong, I think you're doing an awesome job, the speed website itself was a huge step forward for us. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] implementing the additional repo migrations
On Sat, Feb 26, 2011 at 11:51 AM, Laura Creighton l...@openend.se wrote: In a message of Sat, 26 Feb 2011 10:25:45 +0100, Jacob Hallén writes: snip While I am fine with dropping older revisions of just about everything in extradoc, I wonder if it wouldn't be better better for the future to keep this repository in svn format. That way you will only get one copy of everything when cloning the repository. Jacob I'm not fine with the dropping of older revisions. One of the chief benefits for me of moving to a version control system was that I could feel comfortable ruthlessly condensing my writing, knowing that if I ever wanted this stuff later -- say to use in a different document, I could always go back and get the old revision that contained the wonderful words or diagrams I now propose to cut. And this has happened in the past, where early versions of things I wrote ended up raised out of the grave of the repository to live on as part of completely different documents. I'm not going to be comfortable deleting stuff this if I think that the grim reaper is out there, just waiting to purge all my earlier attempts once some document is deemed to be 'final'. So I either won't delete stuff, or I will go back to my old practice of having dozens of versions around 'just in case'. I'm fine with continuing to have the extradoc managed by svn, though I really want a script that runs nightly looking for things in extradoc that have a mimetype of binary and complains about this. Laura Isn't the debate mostly about older revisions of binary files, since the rest is fine? Or you want to keep those as well? (say .doc) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] pypy jit-virtual_state: never retrace a loop more than 5 times
On Tue, Feb 8, 2011 at 8:42 PM, hakanardo commits-nore...@bitbucket.org wrote: Author: Hakan Ardo ha...@debian.org Branch: jit-virtual_state Changeset: r41714:91586ef4b9c5 Date: 2011-02-08 19:42 +0100 http://bitbucket.org/pypy/pypy/changeset/91586ef4b9c5/ Log: never retrace a loop more than 5 times Is this always a good idea? Maybe we should have a configurable parameter for that (like threshold). diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py --- a/pypy/jit/metainterp/optimizeopt/unroll.py +++ b/pypy/jit/metainterp/optimizeopt/unroll.py @@ -671,7 +671,10 @@ jumping to preamble instead) self.emit_operation(op) return - if not self.retraced: + retraced_count = len(short) + if descr.failed_states: + retraced_count += len(descr.failed_states) + if not self.retraced and retraced_count5: if not descr.failed_states: raise RetraceLoop for failed in descr.failed_states: ___ pypy-svn mailing list pypy-...@codespeak.net http://codespeak.net/mailman/listinfo/pypy-svn ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson fredrik.johans...@gmail.com wrote: On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski fij...@gmail.com wrote: On Thu, Feb 3, 2011 at 1:43 AM, Fredrik Johansson fredrik.johans...@gmail.com wrote: On Thu, Feb 3, 2011 at 12:32 AM, Alex Gaynor alex.gay...@gmail.com wrote: If you have any benchmarks where we are slower that don't involve longs that'd be great, since for float operations we really should be able to beat up on CPython. A simple example: mpmath.fp.fsum(mpmath.fp.sqrt(k) for k in xrange(100)) A brief follow up. * PyPy trunk is faster (by quite a bit). * I noticed that you happily use mixture of old and new style classes. As of now this is a really bad case for PyPy. Example: [isinstance(i, type) for i in mpmath.fp.__class__.__mro__] [True, True, True, True, False, False, True, True, False, True, True, True, True, True, True] when I replace it with newstyle classes it runs much faster Interesting. The mixture of old and new style classes is a mistake, of course. I'm going to add a test to make sure this doesn't happen. Thanks for pointing it out. In fact this speeds up another benchmark I did -- [fp.lambertw(k) for k in xrange(5)]-- by 10x, which is quite a ridiculous ratio! Mixture of old and new style classes is not only preventing us from doing optimizations but also hits a bad case of tradeoffs. However, we decided we don't care that much. You should use new style classes anyway :) Other things that speed up both CPython and PyPy: * Put this things into function instead of at global scope Do you mean in the benchmark or did you have some other code you saw in mind? The benchmark. * Use list comprehension instead of generator expression. I hope PyPy can do more in the future to speed up generator expressions. It doesn't speed up things by much. Yeah, I can imagine we can improve on this, but it's also a bit hard. Fredrik ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 12:23 PM, Fredrik Johansson fredrik.johans...@gmail.com wrote: On Thu, Feb 3, 2011 at 11:14 AM, Maciej Fijalkowski fij...@gmail.com wrote: Mixture of old and new style classes is not only preventing us from doing optimizations but also hits a bad case of tradeoffs. However, we decided we don't care that much. You should use new style classes anyway :) Yes, this is perfectly reasonable. You should make PyPy print warnings when it encounters mixed-type classes :) Fredrik That's not a bad idea actually. Maybe with something like some -X option? ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 2:10 PM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2011/2/3 Maciej Fijalkowski fij...@gmail.com: You should make PyPy print warnings when it encounters mixed-type classes :) That's not a bad idea actually. Maybe with something like some -X option? If we use the warnings module to emit JitWarnings, the option could be -Wd::JitWarning I'm not sure about warnings module. How about --jit warnings=1 ? That would fit with other jit options. I was more thinking about this being optional -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 2:21 PM, Antonio Cuni anto.c...@gmail.com wrote: On 03/02/11 13:13, Maciej Fijalkowski wrote: I'm not sure about warnings module. How about --jit warnings=1 ? That would fit with other jit options. not really. The other jit options really belongs to the JIT engine, while this one is dependent on the Python interpreter true. ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 6:15 PM, Stefan Behnel stefan...@behnel.de wrote: Maciej Fijalkowski, 03.02.2011 11:14: On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson wrote: On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski wrote: * Use list comprehension instead of generator expression. I hope PyPy can do more in the future to speed up generator expressions. It doesn't speed up things by much. Yeah, I can imagine we can improve on this, but it's also a bit hard. Does PyPy generate inlined looping code for them if applicable? (e.g. for any(), all(), sum(), sorted(), ...) We don't inline generator expr. The reason is that it's hard to judge when generator frame escapes. We can probably do something with it, but it's not done yet. Stefan ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] pypy jit
2011/2/2 Łukasz Ligowski orangewarr...@gmail.com: Hi, First of all I'd like to thank all developers for good work on PyPy ;) I'd like to ask are there any rules of thumb to write code that PyPy JIT can easily optimize? Or maybe which constructs are hardly optimizable by JIT so it's better to avoid them? I should probably right it down somewhere. As of now there is no such guide. However, the rule of thumb is simple is better than complex. Best regards, Lukasz Ligowski ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] testing floating point
On Thu, Feb 3, 2011 at 1:43 AM, Fredrik Johansson fredrik.johans...@gmail.com wrote: On Thu, Feb 3, 2011 at 12:32 AM, Alex Gaynor alex.gay...@gmail.com wrote: If you have any benchmarks where we are slower that don't involve longs that'd be great, since for float operations we really should be able to beat up on CPython. A simple example: mpmath.fp.fsum(mpmath.fp.sqrt(k) for k in xrange(100)) A brief follow up. * PyPy trunk is faster (by quite a bit). * I noticed that you happily use mixture of old and new style classes. As of now this is a really bad case for PyPy. Example: [isinstance(i, type) for i in mpmath.fp.__class__.__mro__] [True, True, True, True, False, False, True, True, False, True, True, True, True, True, True] when I replace it with newstyle classes it runs much faster Other things that speed up both CPython and PyPy: * Put this things into function instead of at global scope * Use list comprehension instead of generator expression. That all should make PyPy 3x faster than CPython. PyPy 1.4 runs this about 2.4 times slower than CPython. Fredrik ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Link errors while translating with VS2010 and 64bit
On Tue, Feb 1, 2011 at 10:11 AM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: Hi, 2011/2/1 Tasos Vogiatzoglou tvog...@gmail.com: Amaury, It seems that there is a general issue with the compiler/link . I did the translations without the _hashlib and ssl and after a while I got the following errors. [...] [platform:ERROR] implement_52.obj : error LNK2019: unresolved external symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr We have never tried the win64 platform, and I don't have access to a Windows 64bit machine. But I suspect that even if you removed all external dependencies, the result would not work; pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) Before running a translation, could you run the tests in pypy/translator/c ? in the pypy directory, run: python test_all.py translator/c I'd like at least the files test_genc and test_newgc to pass without errors. Surprisingly enough, I have -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] Worrying set of checkins
Anyone feeling like looking into this: http://speed.pypy.org/changes/?tre=10rev=41510:77f94f44989aexe=3env=tannit ? ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Worrying set of checkins
On Tue, Feb 1, 2011 at 12:17 PM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2011/2/1 Maciej Fijalkowski fij...@gmail.com: Anyone feeling like looking into this: http://speed.pypy.org/changes/?tre=10rev=41510:77f94f44989aexe=3env=tannit This show benchmarks for pypy *without* JIT. the pypy-c-jit figures are more positive. -- Amaury Forgeot d'Arc Yes. Still, worth looking into. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Worrying set of checkins
On Tue, Feb 1, 2011 at 12:19 PM, Maciej Fijalkowski fij...@gmail.com wrote: On Tue, Feb 1, 2011 at 12:17 PM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2011/2/1 Maciej Fijalkowski fij...@gmail.com: Anyone feeling like looking into this: http://speed.pypy.org/changes/?tre=10rev=41510:77f94f44989aexe=3env=tannit This show benchmarks for pypy *without* JIT. the pypy-c-jit figures are more positive. -- Amaury Forgeot d'Arc Yes. Still, worth looking into. Well, false alarms. Next run nulled out those changes. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyObject_AsCharBuffer: compiler warning
On Mon, Jan 24, 2011 at 10:38 AM, Arnd Rechenburg arnd.rechenb...@tomtom.com wrote: Hi, When I try to use the function PyObject_AsCharBuffer I get the following compiler warning: warning: passing argument 2 of 'PyObject_AsCharBuffer' from incompatible pointer type By using Python it works without problems. In Python: int PyObject_AsCharBuffer(PyObject *obj, const char **buffer, Py_ssize_t *buffer_len) Is there something different in pypy? We don't have const char**, we use char** instead. It's a small deficiency of how we do stuff now, probably fixable in the future, but don't worry too much, it works the same way. Cheers, fijal ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] Fwd: [issue10994] implementation details in sys module
I think this explanation what sys.getsizeof does is interesting. We might want to have it. -- Forwarded message -- From: Antoine Pitrou rep...@bugs.python.org Date: Mon, Jan 24, 2011 at 4:13 PM Subject: [issue10994] implementation details in sys module To: fij...@gmail.com Antoine Pitrou pit...@free.fr added the comment: I suppose wrt getsizeof it's more of if you provide us with a reasonable expectations, we can implement this other than anything else. The expectation is that it returns the memory footprint of the given object, and only it (not taking into account sharing, caching, dependencies or anything else). For example, an instance will not count its attribute __dict__. But a str object will count its object header plus the string payload, if the payload is private. Of course, you are free to tweak these semantics for the PyPy implementation. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10994 ___ ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] rPython
On Fri, Jan 21, 2011 at 5:03 PM, Eva Maia eva_m...@sapo.pt wrote: Hi, It is possible through the PyPy toolchain check if a program is a rPython program? I know the tool Pylint (RPylint) gives an idea, but it seems to me that this tool gives some errors that result from being a little outdated. Thanks, Eva Maia. ___ Hi. This tool is completely broken. Primary reason is that this tool operates on the level of source code, while RPython operates on live python objects and that's a huge difference. That said, I don't think there is any other way of finding what's RPython without compiling it. In fact, RPython is defined as a thing that can be accepted by our translation toolchain :-) PS. Why you want to write RPython in the first place? Cheers, fijal ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] New version of Codespeed (0.7) for speed.pypy.org
Great! On Thu, Jan 20, 2011 at 11:00 PM, Miquel Torres tob...@googlemail.com wrote: Hi all, I want to announce the release of Codespeed 0.7, the version which is now powering speed.pypy.org The announcement has been made on http://groups.google.com/group/codespeed/browse_thread/thread/3dd2d9491b26d044, but I will paste it here: -- I will briefly describe the most significant improvements. Result Reports -- While the previous Codespeed version allowed a project to analyse and visualize performance changes and compare different versions or even competitors, it didn't address well a use-case: a core developer that is interested in the daily performance variations. You had to go to the Changes view, click on the desired revision, maybe another executable, and again for every revision you wanted to check. That use case is now addressed by Reports. They are a summary of the most significant changes on every run. It's a ultra-condensed Changes table. It is shown on the main page, and you can even subscribe to a RSS feed. That way you only visit the page when you are interested in some particular revision or change. After a few weeks of use, you may find out that a different algorithm is better (for example dropping trends or trends triggering the red state...), or that the thresholds need to be tweaked. And there are surely other interesting run statistics that could be gathered and shown. I eagerly await the feedback and ensuing discussion ;-) Support for small screen sizes -- Using CSS media queries, the site layout adapts to smaller screens. It is optimized for 3 sizes: - Desktop: normal layout for big screens - Netbook, tablets (~1000px): Similar layout but smaller margins, smaller plots. Coincidentally, it is about the same width of Maciej's laptop screen ;-) - Smartphones: iPhones and Androids in landscape mode should be now able to visit the site without problems. Though it can be tested just by resizing the browser window, it would be great to get some feedback of anyone actually using the above mentioned devices! Performance improvements Exarkun made the timeline view faster by pre-fetching all related data. The timeline grid took 1.46 seconds to load, and a single plot 0.45 seconds. They now need 1.26 and 0.26 seconds respectively. A big improvement! There is yet another improvement waiting in the wings, but we still need to figure out some regression it introduces. The Changes view is also much faster (from 480ms to 210ms), because it now uses the cached table stored in the corresponding report (only when viewing with the default trend=10 ). A last detail --- Another change you may notice is that the default checked executables in the Comparison view are now configurable (I didn't forget Maciej! :). So they are not all selected per default any more, so the plot shouldn't look any longer like the Flying Spaghetti Monster ;-) I'll leave it there. I hope you enjoy the changes! Miquel ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] OSError: [Errno 10] No child processes
On Thu, Jan 13, 2011 at 2:12 PM, Armin Rigo ar...@tunes.org wrote: Hi Amaury, On Wed, Jan 12, 2011 at 2:49 AM, Paolo Giarrusso p.giarru...@gmail.com wrote: I propose that PyPy keeps reporting the error for files opened in any write mode I would also think that it's better to keep reporting errors (and for all files instead of just write mode files). In my opinion, it would, if nothing else, give users the message: we got an error when close()ing this file automatically, but you should really close it explicitly yourself in the first place. Maybe it can even be written in that sense in the error message. How about a link to differencies between pypy and cpython, especially about closing files? Cheers, fijal A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy is now available in Fedora
Good job! On Tue, Jan 4, 2011 at 12:13 AM, David Malcolm dmalc...@redhat.com wrote: I've packaged pypy in RPM form for the Fedora distribution [1] - RPM packages are now built in the development branch targeting the next major release (Fedora 15). So it should now be possible for Fedora users to type # yum install pypy and obtain a precompiled /usr/bin/pypy executable via an rpm package, consisting of the JIT-enabled pypy, with all standard settings, without needing to do the full build themselves. I'll do my best to keep these downstream packages promptly updated as further upstream releases of PyPy occur. Many thanks to everyone who helped with this, especially fijal, for all his great feedback on the packaging review [2] (Caveat: actually, the build won't be available yet via yum until a cronjob runs tonight; for now, the build can be downloaded from: http://koji.fedoraproject.org/koji/buildinfo?buildID=212473 ) Some other links, for the curious, can be seen here: https://admin.fedoraproject.org/pkgdb/acls/name/pypy e.g. to the rpm packaging sources. Hope this is helpful Dave [1] http://fedoraproject.org/ [2] https://bugzilla.redhat.com/show_bug.cgi?id=588941 ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Mercurial support for speed.pypy.org
On Sat, Dec 25, 2010 at 1:10 AM, Miquel Torres tob...@googlemail.com wrote: Hi all, I finally added Mercurial support to Codespeed, following Antonio's suggestion of parsing the (templated) command line output.Thanks to everyone that helped! I waited for today to give the awesome PyPy team a present. Merry Christmas! Miquel That's a great present, thanks! Merry Christmas! Cheers, fijal ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] dlopen NotImplementedError
CDLL(None) (or LoadLibrary(None)) is supposed to return to you the whole namespace (all loaded libraries). It's unsupported as far as I can tell. On Thu, Dec 23, 2010 at 12:45 AM, Alex Gaynor alex.gay...@gmail.com wrote: On Wed, Dec 22, 2010 at 4:41 PM, exar...@twistedmatrix.com wrote: On 09:54 pm, alex.gay...@gmail.com wrote: On Wed, Dec 22, 2010 at 3:52 PM, Gary Robinson gary...@me.com wrote: Hmm, I'm thinking the dlopen error might be my problem. I think I built python at /root/pypy, and then moved it to a more proper location. But the error message is referring to /root/pypy, so I'm guessing it has that location hardcoded because that's where I built it? I'm going to rebuild but I wanted to post this message so that you guys don't worry about it (unless rebuilding doesn't help). Nope, it's a known thing, anything trying to import ctypes on fast- forward blows up ATM. If anyone knows what dlopen(None) is supposed to mean, can they let me know and I'll try to take a pass at fixing this? From the dlopen man page: If filename is NULL, then the returned handle is for the main program. I imagine that's the case someone is trying to trigger with ctypes.LoadLibrary(None) (which is what the name `dlopen` is bound to in this context). On the other hand, maybe it's just a screw up somewhere else that causes None to show up instead of the right library name, I haven't looked at the code in question much. Jean-Paul ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev Ok I see the issue I think. It's coming from PyDLL(None). Not sure what the right solution is there, I know Amaury said he was working on the cpythonapi stuff. Alex -- I disapprove of what you say, but I will defend to the death your right to say it. -- Evelyn Beatrice Hall (summarizing Voltaire) The people's good is the highest law. -- Cicero Code can always be simpler than you think, but never as simple as you want -- Me ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] long.__itemsize__
On Tue, Dec 21, 2010 at 1:02 PM, René Dudfield ren...@gmail.com wrote: Hi, __itemsize__ - in bytes, corresponds to item size field in the types definition structure. It's a field for types. See: http://docs.python.org/c-api/typeobj.html#tp_itemsize Well... Those are docs for C API. It doesn't say it's exposed at applevel nor since which version. (Also, to be precise, C API is known to be implementation specific) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] long.__itemsize__
On Tue, Dec 21, 2010 at 1:31 PM, Antonio Cuni anto.c...@gmail.com wrote: On 21/12/10 12:05, Maciej Fijalkowski wrote: __itemsize__ - in bytes, corresponds to item size field in the types definition structure. It's a field for types. See: http://docs.python.org/c-api/typeobj.html#tp_itemsize Well... Those are docs for C API. It doesn't say it's exposed at applevel nor since which version. (Also, to be precise, C API is known to be implementation specific) Moreover, I don't think we could give it a sane semantics on PyPy, given that the same applevel type can be potentially implemented by many different low level structures with different sizes. Not even potentially, it actually is in some places. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: [IronPython] SciPy
Oh wow, that's really cool. On Tue, Dec 21, 2010 at 6:15 PM, Gary Robinson gary...@me.com wrote: I thought this email could be relevant for those interested in SciPy / Numpy on pypy. With enthought implementing a smaller core and using compatibility layers for alternative platforms, it would seem to be a good basis for a pypy port. All I can do is give a BIG +1 to anything that can get numpy/scipy up more quickly. PyPy is starting to give us the speed we need for statistical/scientific work on python (the speed it still lacks compared to C or Java is made up for, for my purposes, by the ease of writing python compared to those languages). The recent 64-bit functionality lets us process a lot of data. The fast-forward branch is giving us the multiprocessing we need. (I recognize that there are other solutions, but for simple things we need to write quickly, the multiprocessing module is really sweet.) The main thing missing now is numpy/scipy. The addition of that will make PyPy a huge win in the scientific community, IMO. Anyway, I mention it in case the opinion of one person who is using Python in production for statistical processing is of interest. -- Gary Robinson CTO Emergent Discovery, LLC personal email: gary...@me.com work email: grobin...@emergentdiscovery.com Company: http://www.emergentdiscovery.com Blog: http://www.garyrobinson.net ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Interpretor for vectorized langugage
On Fri, Dec 17, 2010 at 1:12 AM, Paolo Giarrusso p.giarru...@gmail.com wrote: On Thu, Dec 16, 2010 at 10:16, Armin Rigo ar...@tunes.org wrote: Hi, have you seen numpy/scipy? Of course you are then going to hit the same problems that Ademan tries to solve for numpy/scipy, notably how to implement at least the basic linear algebra operations in such a way that the JIT can improve them. There are various goals there, e.g. to turn Python (or Matlab) code like A+B+C, adding three matrices together, into one matrix operation instead of two (as it is now: (A+B)+C). This is all a bit experimental so far. [More about numpy/scipy]: How about the ideas from C++ expression templates? Or even better, some sort of lazy evaluation? With C++ expression templates [1], (A+B) returns something like PlusMatrix(A, B). When finally assigning to a variable, the copy constructor converts this into a real matrix; in the following line: D = (A+B) + C PlusMatrix(PlusMatrix(A, B), C) is built, and D gets the result. What they could further do, but they don't (see FAQ of uBlas [1]), is to cache the result of evaluating a template when requested; that could be cached (as in lazy evaluation - expressions like PlusMatrix are not very different from thunks, in that case) or maybe JIT compilation can recognize the constantness of this. Best regards [1] One library using them is Boost::uBlas, but the idea is much older: http://www.boost.org/doc/libs/1_45_0/libs/numeric/ublas/doc/index.htm -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev Hi. I did that (it's not published anywhere though) and it gave quite huge speedups compared to numexpr or numpy. I'll look into implementing it properly at some point (instead of doing that in pure python). ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Problem building C Extension
Not to comment on issue, but it's actually pretty good that zope.interface C speedups don't work. They won't be speedups in pypy case even if they did work. On Tue, Dec 14, 2010 at 12:13 PM, Kevin Gill ke...@movieextras.ie wrote: Hi, I am new to PyPy. I built pypy using... $ python translate.py --opt=jit targetpypystandalone.py When I try to build eggs with C extensions it doesn't work. I am using pip in a virtualenv. The build uses the wrong compiler cc instead of gcc and does not use CFLAGS. For example: $ pip -v install zope.interface==3.6.1 shows this output cc -I/home/kevin/pypp/env/include -c src/zope/interface/_zope_interface_coptimizations.c -o build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o src/zope/interface/_zope_interface_coptimizations.c: In function ‘verifying_changed’: src/zope/interface/_zope_interface_coptimizations.c:1349: warning: assignment makes pointer from integer without a cast cc -shared build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o -o build/lib.linux-x86_64-2.5/zope/interface/_zope_interface_coptimizations.pypy-14.so /usr/bin/ld: build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC build/temp.linux-x86_64-2.5/src/zope/interface/_zope_interface_coptimizations.o: could not read symbols: Bad value collect2: ld returned 1 exit status I did some digging through the source. I found that the compiler configuration is normally loaded via distutils/sysconfig.py, but that module was effectively disabled in pypy. I cannot override using environment variables for example because the customisation function is as follows: def customize_compiler(options): Dummy method to let some easy_install packages that have optional C speedup components. pass I see from the mailing list that people are compiling C extensions. What am I doing wrong? Thanks Kevin Version of pypy that I am using. $ svn info Path: . URL: http://codespeak.net/svn/pypy/trunk Repository Root: http://codespeak.net/svn Repository UUID: fd0d7bf2-dfb6-0310-8d31-b7ecfe96aada Revision: 79796 Node Kind: directory Schedule: normal Last Changed Author: afa Last Changed Rev: 79794 Last Changed Date: 2010-12-03 21:52:49 + (Fri, 03 Dec 2010) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] pypy commit 7b8aa74da1fb: Handle output on standard error when invoking Mercurial.
This commit contains no tests whatsoever, it would be cooler if it did.
On Tue, Dec 14, 2010 at 8:19 PM, Bitbucket
commits-nore...@bitbucket.org wrote:
# HG changeset patch -- Bitbucket.org
# Project pypy
# URL http://bitbucket.org/pypy/pypy/overview
# User Dan Villiom Podlaski Christiansen dan...@gmail.com
# Date 1292349796 -3600
# Node ID 7b8aa74da1fb06bd5af8b37e086538e4e7d5d0a1
# Parent 1f81ed99bfbc17bbb3cc0920ac612218b4194f8c
Handle output on standard error when invoking Mercurial.
Instead of just piped to the process standard error, we use a logger
to warn if 'hg' wrote anything on stderr.
--- a/pypy/tool/version.py
+++ b/pypy/tool/version.py
@@ -13,24 +13,39 @@ def get_mercurial_info():
hgexe = py.path.local.sysfind('hg')
if hgexe and os.path.isdir(os.path.join(pypyroot, '.hg')):
+ def maywarn(err):
+ if not err:
+ return
+
+ from pypy.tool.ansi_print import ansi_log
+ log = py.log.Producer(version)
+ py.log.setconsumer(version, ansi_log)
+ log.WARNING('Errors getting Mercurial information:' + err)
+
env = dict(os.environ)
# get Mercurial into scripting mode
env['HGPLAIN'] = '1'
# disable user configuration, extensions, etc.
env['HGRCPATH'] = os.devnull
- p = Popen([str(hgexe), 'id', '-i', pypyroot], stdout=PIPE, env=env)
+ p = Popen([str(hgexe), 'id', '-i', pypyroot],
+ stdout=PIPE, stderr=PIPE, env=env)
hgid = p.stdout.read().strip()
+ maywarn(p.stderr.read())
- p = Popen([str(hgexe), 'id', '-t', pypyroot], stdout=PIPE, env=env)
+ p = Popen([str(hgexe), 'id', '-t', pypyroot],
+ stdout=PIPE, stderr=PIPE, env=env)
hgtags = [t for t in p.stdout.read().strip().split() if t != 'tip']
+ maywarn(p.stderr.read())
if hgtags:
return 'PyPy', hgtags[0], hgid
else:
# use the branch instead
- p = Popen([str(hgexe), 'id', '-b', pypyroot], stdout=PIPE,
env=env)
+ p = Popen([str(hgexe), 'id', '-b', pypyroot],
+ stdout=PIPE, stderr=PIPE, env=env)
hgbranch = p.stdout.read().strip()
+ maywarn(p.stderr.read())
return 'PyPy', hgbranch, hgid
else:
___
pypy-svn mailing list
pypy-...@codespeak.net
http://codespeak.net/mailman/listinfo/pypy-svn
___
pypy-dev@codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Idea for speed.pypy.org
On Mon, Dec 13, 2010 at 10:51 PM, Miquel Torres tob...@googlemail.com wrote: @Maciej: it doesn't make a lot of sense. Looking at this graph: http://speed.pypy.org/comparison/?exe=2%2B35,4%2B35,1%2B172,3%2B172ben=11,14,15env=1hor=falsebas=nonechart=normal+bars slowspitfire is much faster than the other two. Is that because it performs more iterations? I think it's apples to oranges (they have different table sizes and different number of iterations) Also, how come pypy-c-jit is faster than cpython or psyco precisely in cstringio, where performance should be dependent on cstringIO and thus be more similar across interpreters? because having a list of small strings means you have a large (old) object referencing a lot of young objects, hence GC cost. It's not the case with cstringio where you have a single chunk of memory which does not contain GC pointers. 2010/12/13 Leonardo Santagada santag...@gmail.com: why not have only 2 versions, both with the same size table and name one spitfire_cstringio and the other spitfire_strjoin? I think it would make things clearer. On Mon, Dec 13, 2010 at 2:43 PM, Maciej Fijalkowski fij...@gmail.com wrote: Hi. spitfires are confusing. slowspitfire and spitfire use ''.join(list-of-strings) where spitfire_cstringio uses cStringIO instead. spitfire and spitfire_cstringio use smaller table to render (100x100 I think) which was the default on original benchmarks slowspitfire uses 1000x1000 (which is why it used to be slower than spitfire) and was chosen by US guys to let the JIT warm up. We should remove _slow these days. On Mon, Dec 13, 2010 at 5:08 PM, Miquel Torres tob...@googlemail.com wrote: sorry, I meant the opposite. To recap, according to http://code.google.com/p/unladen-swallow/wiki/Benchmarks, spitfire: psyco slowspitfire: pure python in addition we have spitfire_cstringio, which uses a c module (so it is even faster). what is vanilla spitfire in our case? 2010/12/13 Miquel Torres tob...@googlemail.com: @Carl Friedrich exarkun: thanks, I've added those. only spectral-norm, slowspitfire and ai to go. slowspitfire is described at the Unladen page as using psyco, but it doesn't make sense in our case? 2010/12/13 exar...@twistedmatrix.com: On 08:20 am, tob...@googlemail.com wrote: Thanks all for the input. I've compiled a list based on your mails, the Unladen benchmarks page (http://code.google.com/p/unladen-swallow/wiki/Benchmarks), and the alioth descriptions. Here is an extract of the current speed.pypy.org admin: [snip] twisted_iteration Iterates a Twisted reactor as quickly as possible without doing any work. twisted_names Runs a DNS server with Twisted Names and then issues requests to it over loopback UDP. twisted_pb Runs a Perspective Broker server with a no-op method and invokes that method over loopback TCP with some strings, dictionaries, and tuples as arguments. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev -- Leonardo Santagada ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Migration to mercurial
On Tue, Dec 14, 2010 at 9:24 AM, Seung Soo, sungs...@gmail.com wrote: Antonio Cuni anto.cuni at gmail.com writes: Hi all, finally, it's happening . Thanks to Ronny's work, we are going to complete the migration to mercurial very soon. +1 Cool. Providing binaries through bitbucket.org/pypy/pypy/downloads would be nice too. Are the bugtracking/code review facilities of bitbucket going to be used too? Or is the current https://codespeak.net/issue/pypy-dev/ here to stay? All of this will happen, it's just a one-by-one process (since they don't depend on each other). Don't worry, we'll get there :) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Idea for speed.pypy.org
Hey miquel, didn't we loose colors somehow? On Tue, Dec 14, 2010 at 8:32 AM, Maciej Fijalkowski fij...@gmail.com wrote: On Mon, Dec 13, 2010 at 10:51 PM, Miquel Torres tob...@googlemail.com wrote: @Maciej: it doesn't make a lot of sense. Looking at this graph: http://speed.pypy.org/comparison/?exe=2%2B35,4%2B35,1%2B172,3%2B172ben=11,14,15env=1hor=falsebas=nonechart=normal+bars slowspitfire is much faster than the other two. Is that because it performs more iterations? I think it's apples to oranges (they have different table sizes and different number of iterations) Also, how come pypy-c-jit is faster than cpython or psyco precisely in cstringio, where performance should be dependent on cstringIO and thus be more similar across interpreters? because having a list of small strings means you have a large (old) object referencing a lot of young objects, hence GC cost. It's not the case with cstringio where you have a single chunk of memory which does not contain GC pointers. 2010/12/13 Leonardo Santagada santag...@gmail.com: why not have only 2 versions, both with the same size table and name one spitfire_cstringio and the other spitfire_strjoin? I think it would make things clearer. On Mon, Dec 13, 2010 at 2:43 PM, Maciej Fijalkowski fij...@gmail.com wrote: Hi. spitfires are confusing. slowspitfire and spitfire use ''.join(list-of-strings) where spitfire_cstringio uses cStringIO instead. spitfire and spitfire_cstringio use smaller table to render (100x100 I think) which was the default on original benchmarks slowspitfire uses 1000x1000 (which is why it used to be slower than spitfire) and was chosen by US guys to let the JIT warm up. We should remove _slow these days. On Mon, Dec 13, 2010 at 5:08 PM, Miquel Torres tob...@googlemail.com wrote: sorry, I meant the opposite. To recap, according to http://code.google.com/p/unladen-swallow/wiki/Benchmarks, spitfire: psyco slowspitfire: pure python in addition we have spitfire_cstringio, which uses a c module (so it is even faster). what is vanilla spitfire in our case? 2010/12/13 Miquel Torres tob...@googlemail.com: @Carl Friedrich exarkun: thanks, I've added those. only spectral-norm, slowspitfire and ai to go. slowspitfire is described at the Unladen page as using psyco, but it doesn't make sense in our case? 2010/12/13 exar...@twistedmatrix.com: On 08:20 am, tob...@googlemail.com wrote: Thanks all for the input. I've compiled a list based on your mails, the Unladen benchmarks page (http://code.google.com/p/unladen-swallow/wiki/Benchmarks), and the alioth descriptions. Here is an extract of the current speed.pypy.org admin: [snip] twisted_iteration Iterates a Twisted reactor as quickly as possible without doing any work. twisted_names Runs a DNS server with Twisted Names and then issues requests to it over loopback UDP. twisted_pb Runs a Perspective Broker server with a no-op method and invokes that method over loopback TCP with some strings, dictionaries, and tuples as arguments. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev -- Leonardo Santagada ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] r79938 - in pypy/trunk/pypy: module/posix module/posix/test rpython/module translator/c/test
Do we *really* need those functions available on RPython level?
On Thu, Dec 9, 2010 at 7:30 PM, ar...@codespeak.net wrote:
Author: arigo
Date: Thu Dec 9 18:30:33 2010
New Revision: 79938
Modified:
pypy/trunk/pypy/module/posix/__init__.py
pypy/trunk/pypy/module/posix/interp_posix.py
pypy/trunk/pypy/module/posix/test/test_posix2.py
pypy/trunk/pypy/rpython/module/ll_os.py
pypy/trunk/pypy/translator/c/test/test_extfunc.py
Log:
Implement os.getloadavg(). (Phew, we really need to edit files in all
corners of the world for this...)
Modified: pypy/trunk/pypy/module/posix/__init__.py
==
--- pypy/trunk/pypy/module/posix/__init__.py (original)
+++ pypy/trunk/pypy/module/posix/__init__.py Thu Dec 9 18:30:33 2010
@@ -111,6 +111,8 @@
interpleveldefs['sysconf_names'] = 'space.wrap(os.sysconf_names)'
if hasattr(os, 'ttyname'):
interpleveldefs['ttyname'] = 'interp_posix.ttyname'
+ if hasattr(os, 'getloadavg'):
+ interpleveldefs['getloadavg'] = 'interp_posix.getloadavg'
for name in ['setsid', 'getuid', 'geteuid', 'getgid', 'getegid', 'setuid',
'seteuid', 'setgid', 'setegid', 'getpgrp', 'setpgrp',
Modified: pypy/trunk/pypy/module/posix/interp_posix.py
==
--- pypy/trunk/pypy/module/posix/interp_posix.py (original)
+++ pypy/trunk/pypy/module/posix/interp_posix.py Thu Dec 9 18:30:33
2010
@@ -963,6 +963,17 @@
return space.w_None
chown.unwrap_spec = [ObjSpace, str, c_nonnegint, c_nonnegint]
+def getloadavg(space):
+ try:
+ load = os.getloadavg()
+ except OSError, e:
+ raise OperationError(space.w_OSError,
+ space.wrap(Load averages are unobtainable))
+ return space.newtuple([space.wrap(load[0]),
+ space.wrap(load[1]),
+ space.wrap(load[2])])
+getloadavg.unwrap_spec = [ObjSpace]
+
if _WIN:
from pypy.rlib import rwin32
Modified: pypy/trunk/pypy/module/posix/test/test_posix2.py
==
--- pypy/trunk/pypy/module/posix/test/test_posix2.py (original)
+++ pypy/trunk/pypy/module/posix/test/test_posix2.py Thu Dec 9 18:30:33
2010
@@ -521,6 +521,14 @@
assert os.WIFEXITED(status)
assert os.WEXITSTATUS(status) == exit_status
+ if hasattr(os, 'getloadavg'):
+ def test_os_getloadavg(self):
+ os = self.posix
+ l0, l1, l2 = os.getloadavg()
+ assert type(l0) is float and l0 = 0.0
+ assert type(l1) is float and l0 = 0.0
+ assert type(l2) is float and l0 = 0.0
+
if hasattr(os, 'fsync'):
def test_fsync(self):
os = self.posix
Modified: pypy/trunk/pypy/rpython/module/ll_os.py
==
--- pypy/trunk/pypy/rpython/module/ll_os.py (original)
+++ pypy/trunk/pypy/rpython/module/ll_os.py Thu Dec 9 18:30:33 2010
@@ -730,6 +730,22 @@
return extdef([traits.str, int, int], int, traits.ll_os_name('open'),
llimpl=os_open_llimpl, oofakeimpl=os_open_oofakeimpl)
+ �...@registering_if(os, 'getloadavg')
+ def register_os_getloadavg(self):
+ AP = rffi.CArrayPtr(lltype.Float)
+ c_getloadavg = self.llexternal('getloadavg', [AP, rffi.INT],
rffi.INT)
+
+ def getloadavg_llimpl():
+ load = lltype.malloc(AP.TO, 3, flavor='raw')
+ r = c_getloadavg(load, 3)
+ result_tuple = load[0], load[1], load[2]
+ lltype.free(load, flavor='raw')
+ if r != 3:
+ raise OSError
+ return result_tuple
+ return extdef([], (float, float, float),
+ ll_os.ll_getloadavg, llimpl=getloadavg_llimpl)
+
# --- os.read ---
@registering(os.read)
Modified: pypy/trunk/pypy/translator/c/test/test_extfunc.py
==
--- pypy/trunk/pypy/translator/c/test/test_extfunc.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_extfunc.py Thu Dec 9 18:30:33
2010
@@ -755,3 +755,13 @@
for i in range(5):
res = func(i)
assert res == os.uname()[i]
+
+if hasattr(os, 'getloadavg'):
+ def test_os_getloadavg():
+ def does_stuff():
+ a, b, c = os.getloadavg()
+ print a, b, c
+ return a + b + c
+ f = compile(does_stuff, [])
+ res = f()
+ assert type(res) is float and res = 0.0
___
pypy-svn mailing list
pypy-...@codespeak.net
Re: [pypy-dev] Idea for speed.pypy.org
On Wed, Dec 8, 2010 at 8:41 PM, Miquel Torres tob...@googlemail.com wrote: Hey, I was serious when I said I would improve on benchmarks info! Anyone want to help in creating that benchmark description list? Just reply to this mail so that everyone can review what the benchmarks do. Hey. I can do some stuff. I was serious about documenting why we're slow/fast on benchmarks - that maybe we should bring down our docs to a manageable number first :) Benchmarks descriptions are however unlikely to change 2010/12/4 Maciej Fijalkowski fij...@gmail.com: On Sat, Dec 4, 2010 at 1:05 PM, Laura Creighton l...@openend.se wrote: re: keeping the 'why we are slower/what we could do to fix it' info up to date -- one possibility is to make a 'why we were/what we are for release 1.4.' Then every time you make a major release, you update those fields as needed. And if major changes happen between 'what was in the last major release' vs 'what's on trunk now' you can even make a note of it -- 'fixed in rev whatever it was, when we merged in whoever it was' brank, see blog post over here'. And if you forget, well, you will catch it when the next major release comes out. Just an idea. Laura I think the general idea we know what's wrong vs we have no clue is good. However, I would like to maybe decide what we do with tons of docs that we already have first :) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Idea for speed.pypy.org
On Sat, Dec 4, 2010 at 1:05 PM, Laura Creighton l...@openend.se wrote: re: keeping the 'why we are slower/what we could do to fix it' info up to date -- one possibility is to make a 'why we were/what we are for release 1.4.' Then every time you make a major release, you update those fields as needed. And if major changes happen between 'what was in the last major release' vs 'what's on trunk now' you can even make a note of it -- 'fixed in rev whatever it was, when we merged in whoever it was' brank, see blog post over here'. And if you forget, well, you will catch it when the next major release comes out. Just an idea. Laura I think the general idea we know what's wrong vs we have no clue is good. However, I would like to maybe decide what we do with tons of docs that we already have first :) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Testing if a function exists?
Use: import __builtin__ and treat it as module. __builtins__ is an ugly hack that is sometimes a dict and sometimes a module and pypy has different corner cases. __builtin__ will always work On Fri, Dec 3, 2010 at 9:13 PM, Leonardo Santagada santag...@gmail.com wrote: (I'm guessing but) In python 2.5 modules are not iterable, you can use getattr for the same effect. On Fri, Dec 3, 2010 at 5:04 PM, Dan Stromberg drsali...@gmail.com wrote: How does one test if a function exists in pypy? In CPython 2.x and 3.x, it appears to be sufficient to use: 'funcname' in __bultins__ ...but that doesn't appear to work in pypy 1.4: print 'platform_version' in platform Traceback (most recent call last): File console, line 1, in module TypeError: 'module' object is not iterable print 'bytes' in __builtins__ Traceback (most recent call last): File console, line 1, in module TypeError: 'module' object is not iterable -- Leonardo Santagada ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Idea for speed.pypy.org
- Why PyPy performs better/worse than CPYthon (if known) I'm a bit worried about keeping this up to date ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Testing if a function exists?
According to this: http://docs.python.org/py3k/whatsnew/3.0.html they renamed __builtin__ to builtins (with removed underscores). So probably a bunch of ifs would do On Sat, Dec 4, 2010 at 7:42 AM, Dan Stromberg drsali...@gmail.com wrote: This sounds promising, though sadly, it doesn't appear to work in Python 3.x: $ /usr/local/cpython-3.1/bin/python3 -c 'import __builtin__' Traceback (most recent call last): File string, line 1, in module ImportError: No module named __builtin__ benchbox-dstromberg:~/src/home-svn/backshift/trunk i686-pc-linux-gnu 22275 - above cmd done 2010 Fri Dec 03 09:40 PM $ /usr/local/cpython-3.0/bin/python3.0 -c 'import __builtin__' Traceback (most recent call last): File string, line 1, in module ImportError: No module named __builtin__ benchbox-dstromberg:~/src/home-svn/backshift/trunk i686-pc-linux-gnu 22275 - above cmd done 2010 Fri Dec 03 09:40 PM $ /usr/local/cpython-2.7/bin/python -c 'import __builtin__' benchbox-dstromberg:~/src/home-svn/backshift/trunk i686-pc-linux-gnu 22275 - above cmd done 2010 Fri Dec 03 09:41 PM $ /usr/local/cpython-2.6/bin/python -c 'import __builtin__' benchbox-dstromberg:~/src/home-svn/backshift/trunk i686-pc-linux-gnu 22275 - above cmd done 2010 Fri Dec 03 09:41 PM $ /usr/local/cpython-2.5/bin/python -c 'import __builtin__' benchbox-dstromberg:~/src/home-svn/backshift/trunk i686-pc-linux-gnu 22275 - above cmd done 2010 Fri Dec 03 09:41 PM $ On Fri, Dec 3, 2010 at 11:37 AM, Maciej Fijalkowski fij...@gmail.com wrote: Use: import __builtin__ and treat it as module. __builtins__ is an ugly hack that is sometimes a dict and sometimes a module and pypy has different corner cases. __builtin__ will always work On Fri, Dec 3, 2010 at 9:13 PM, Leonardo Santagada santag...@gmail.com wrote: (I'm guessing but) In python 2.5 modules are not iterable, you can use getattr for the same effect. On Fri, Dec 3, 2010 at 5:04 PM, Dan Stromberg drsali...@gmail.com wrote: How does one test if a function exists in pypy? In CPython 2.x and 3.x, it appears to be sufficient to use: 'funcname' in __bultins__ ...but that doesn't appear to work in pypy 1.4: print 'platform_version' in platform Traceback (most recent call last): File console, line 1, in module TypeError: 'module' object is not iterable print 'bytes' in __builtins__ Traceback (most recent call last): File console, line 1, in module TypeError: 'module' object is not iterable -- Leonardo Santagada ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
On Wed, Dec 1, 2010 at 9:48 AM, Stefan Behnel stefan...@behnel.de wrote: Paolo Giarrusso, 01.12.2010 00:34: Anyway, this does not interact with benchmarks above - Stefan, I still don't get why you complained that pyexpat is slow by showing benchmarks for another module, I guess I do not understand your email, but it asks reasonable? after Amaury talks about pyexpat. Well, in CPython, I can see little to no reason why anyone would go as low-level as pyexpat when you can use cElementTree. So IMHO the level to compare is what people would normally use rather than what people could potentially use if they were beaten hard enough. Stefan Hey. Sure, makes sense :-) I think one of the reasons for some slowdown is that calls from C are not jitted if they don't contain loops themselves. This doesn't explain the whole thing obviously, because there is something really wrong going on looking at numbers. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] (no subject)
Hey. I would suggest using array module. It allocates memory in a non-moving location and it's address can be found using buffer_info method. Cheers, fijal On Wed, Dec 1, 2010 at 3:18 PM, renaud blanch rndbl...@gmail.com wrote: hi, i'm trying to make some pyopengl [0] -based code [1] run on top of pypy. this is partially successful, but i need some advice to progress further. pyopengl 3.x makes use of ctypes to provide the opengl binding, and it works out of the box for simple functions (those that do not takes c-pointer to buffers of data as arguments). for the rest, the first issue is that pyopengl use two functions from the ctypes.pythonapi lib, namely PyString_AsString and PyBuffer_FromMemory. any advice on how to replace those functions to make them compatible with pypy? Mike Fletcher (pyopengl author) gave me some hints about that point: For the first issue, those are going to require some reworking, in essence those are C implemented code that happens to use Python/ctypes as the implementation language and makes assumptions about the data-storage for the objects (e.g. that a string is internally a contiguous series of bytes, which is *not necessarily* true in PyPy). We'd need to find a mechanism in PyPy that would give us that direct memory-pointer access to be able to use it. Note: a compacting garbage collector (or anything else that can move memory locations) will cause problems there, so we may need to find a way to signal PyPy not to move a given object, and to use contiguous data-arrays for their storage. thanks a lot for any advice, renaud 0. PyOpenGL 3.x / The Python OpenGL® Binding http://pyopengl.sourceforge.net/ 1. opengl-programmable / a short step by step tutorial to OpenGL programmable pipeline http://bitbucket.org/rndblnch/opengl-programmable/ ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
On Tue, Nov 30, 2010 at 1:45 AM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2010/11/30 Paolo Giarrusso p.giarru...@gmail.com As a matter of fact, however, pyexpat is not involved here for PyPy, and here (v1.4) it is still implemented through ctypes (in lib_pypy/pyexpat.py), and not in RPython in pypy/rlib/. It's also module/pyexpat and not rlib (rlib is for RPython libraries) Did you compile pypy yourself? if the expat development files are present, the translation should build the pyexpat module: Python 2.5.2 (79656, Nov 29 2010, 21:05:28) [PyPy 1.4.0] on linux2 import pyexpat pyexpat module 'pyexpat' (built-in) -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy 1.4 released
Hey. On Sun, Nov 28, 2010 at 10:57 AM, Phyo Arkar phyo.arkarl...@gmail.com wrote: i got python-magic working , after i installed without easy_install (easy_install fail because it tried to install ctypes). great Now what is not working is python-lxml , which is very important for my project. lxml won't work out of the box. if you think it's important enough, you can try to port cython to generate something saner (right now what it generates won't work on cpyext). Cheers, fijal here are the errors: Running lxml-2.3beta1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Gg3GRA/lxml-2.3beta1/egg-dist-tmp-bwUkM2 Building lxml version 2.3.beta1. NOTE: Trying to build without Cython, pre-generated 'src/lxml/lxml.etree.c' needs to be available. Using build configuration of libxslt 1.1.26 Building against libxml2/libxslt in the following directory: /usr/lib src/lxml/lxml.etree.c:75: error: conflicting types for ‘Py_buffer’ /home/v3ss/pypy-1.4/include/object.h:19: note: previous declaration of ‘Py_buffer’ was here Had Anyone got lxml working in pypy successfully? On 11/27/10, Antonio Cuni anto.c...@gmail.com wrote: On 27/11/10 03:09, Phyo Arkar wrote: libmagic python fails to work on pypy (python-magic) it uses ctypes and easy-install try to download and instaill it , but it fails. how to enable ctypes on pypy? Hi Phyo, ctypes *is* enabled on pypy by default. If python-magic does not work, it can either: 1) be a pypy bug: please report it as an issue (possibly with a simple failing test and the full traceack) 2) a python-magic issue, e.g. if it plays dirty ctypes trick that cannot really be supported by pypy due to the internal differences with CPython ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
On Sun, Nov 28, 2010 at 11:58 AM, René Dudfield ren...@gmail.com wrote: Hi, what xml libraries are people using with pypy? What is working well? cu, PyExpat works, although it's slow (ctypes-based implementation). I know genshi has some troubles with it, someone is debugging now. Besides I don't think there are any working (unless someone wrote a pure-python one) Cheers, fijal On Sun, Nov 28, 2010 at 9:48 AM, Maciej Fijalkowski fij...@gmail.com wrote: Hey. On Sun, Nov 28, 2010 at 10:57 AM, Phyo Arkar phyo.arkarl...@gmail.com wrote: i got python-magic working , after i installed without easy_install (easy_install fail because it tried to install ctypes). great Now what is not working is python-lxml , which is very important for my project. lxml won't work out of the box. if you think it's important enough, you can try to port cython to generate something saner (right now what it generates won't work on cpyext). Cheers, fijal here are the errors: Running lxml-2.3beta1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Gg3GRA/lxml-2.3beta1/egg-dist-tmp-bwUkM2 Building lxml version 2.3.beta1. NOTE: Trying to build without Cython, pre-generated 'src/lxml/lxml.etree.c' needs to be available. Using build configuration of libxslt 1.1.26 Building against libxml2/libxslt in the following directory: /usr/lib src/lxml/lxml.etree.c:75: error: conflicting types for ‘Py_buffer’ /home/v3ss/pypy-1.4/include/object.h:19: note: previous declaration of ‘Py_buffer’ was here Had Anyone got lxml working in pypy successfully? On 11/27/10, Antonio Cuni anto.c...@gmail.com wrote: On 27/11/10 03:09, Phyo Arkar wrote: libmagic python fails to work on pypy (python-magic) it uses ctypes and easy-install try to download and instaill it , but it fails. how to enable ctypes on pypy? Hi Phyo, ctypes *is* enabled on pypy by default. If python-magic does not work, it can either: 1) be a pypy bug: please report it as an issue (possibly with a simple failing test and the full traceack) 2) a python-magic issue, e.g. if it plays dirty ctypes trick that cannot really be supported by pypy due to the internal differences with CPython ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] PyPy 1.4 released
=== PyPy 1.4: Ouroboros in practice === We're pleased to announce the 1.4 release of PyPy. This is a major breakthrough in our long journey, as PyPy 1.4 is the first PyPy release that can translate itself faster than CPython. Starting today, we are using PyPy more for our every-day development. So may you :) You can download it here: http://pypy.org/download.html What is PyPy PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython. It's fast (`pypy 1.4 and cpython 2.6`_ comparison) Among its new features, this release includes numerous performance improvements (which made fast self-hosting possible), a 64-bit JIT backend, as well as serious stabilization. As of now, we can consider the 32-bit and 64-bit linux versions of PyPy stable enough to run `in production`_. Numerous speed achievements are described on `our blog`_. Normalized speed charts comparing `pypy 1.4 and pypy 1.3`_ as well as `pypy 1.4 and cpython 2.6`_ are available on benchmark website. For the impatient: yes, we got a lot faster! More highlights === * PyPy's built-in Just-in-Time compiler is fully transparent and automatically generated; it now also has very reasonable memory requirements. The total memory used by a very complex and long-running process (translating PyPy itself) is within 1.5x to at most 2x the memory needed by CPython, for a speed-up of 2x. * More compact instances. All instances are as compact as if they had ``__slots__``. This can give programs a big gain in memory. (In the example of translation above, we already have carefully placed ``__slots__``, so there is no extra win.) * `Virtualenv support`_: now PyPy is fully compatible with virtualenv_: note that to use it, you need a recent version of virtualenv (= 1.5). * Faster (and JITted) regular expressions - huge boost in speeding up the `re` module. * Other speed improvements, like JITted calls to functions like map(). .. _virtualenv: http://pypi.python.org/pypi/virtualenv .. _`Virtualenv support`: http://morepypy.blogspot.com/2010/08/using-virtualenv-with-pypy.html .. _`in production`: http://morepypy.blogspot.com/2010/11/running-large-radio-telescope-software.html .. _`our blog`: http://morepypy.blogspot.com .. _`pypy 1.4 and pypy 1.3`: http://speed.pypy.org/comparison/?exe=1%2B41,1%2B172ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20env=1hor=falsebas=1%2B41chart=normal+bars .. _`pypy 1.4 and cpython 2.6`: http://speed.pypy.org/comparison/?exe=2%2B35,1%2B172ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20env=1hor=falsebas=2%2B35chart=normal+bars Cheers, Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski, Amaury Forgeot d'Arc, Armin Rigo and the PyPy team ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] r79480 - pypy/trunk/pypy/jit/tool
On Thu, Nov 25, 2010 at 9:45 AM, Antonio Cuni anto.c...@gmail.com wrote: On 25/11/10 07:40, Maciej Fijalkowski wrote: Is pypy repository really a place for such files? Maybe we should keep it somewhere else? well, it's the memory usage of cpython when translating pypy. Why the pypy repo should not be the place for a pypy-related file? For the same reason why we don't put extradoc in checkout I guess. On Wed, Nov 24, 2010 at 6:39 PM,antoc...@codespeak.net wrote: Author: antocuni Date: Wed Nov 24 17:39:03 2010 New Revision: 79480 Added: pypy/trunk/pypy/jit/tool/cpython.vmrss Log: add the vmrss file produced by armin's memusage.py when running ./translate.py -Ojit at rev 79456 ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] r79382 - pypy/trunk/pypy/config
On Wed, Nov 24, 2010 at 2:39 PM, Armin Rigo ar...@tunes.org wrote: Hi, On Wed, Nov 24, 2010 at 1:19 AM, exar...@twistedmatrix.com wrote: I guess it should just be another log category, like jit-summary. Then you can get the effect you want by setting (in the shell) PYPYLOG to jit-summary:-. This or an env variable. The PYPYLOG solution I described *is* an environment variable. What do you mean? yes of course. nothing any more :) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] r79480 - pypy/trunk/pypy/jit/tool
Is pypy repository really a place for such files? Maybe we should keep it somewhere else? On Wed, Nov 24, 2010 at 6:39 PM, antoc...@codespeak.net wrote: Author: antocuni Date: Wed Nov 24 17:39:03 2010 New Revision: 79480 Added: pypy/trunk/pypy/jit/tool/cpython.vmrss Log: add the vmrss file produced by armin's memusage.py when running ./translate.py -Ojit at rev 79456 Added: pypy/trunk/pypy/jit/tool/cpython.vmrss == --- (empty file) +++ pypy/trunk/pypy/jit/tool/cpython.vmrss Wed Nov 24 17:39:03 2010 @@ -0,0 +1,4101 @@ +124 +16600 +20620 +20640 +22036 +94084 +94084 +94108 +94108 +94108 +94108 +94108 +94120 +94164 +94160 +94160 +94160 +94160 +94160 +94216 +110644 +123144 +135236 +143680 +148500 +153104 +157088 +160640 +164760 +167992 +163108 +163232 +163232 +163436 +163436 +163436 +163444 +163444 +163444 +163448 +163448 +163448 +163448 +163448 +163448 +167060 +170948 +174432 +177212 +177216 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176520 +176520 +176520 +176520 +176520 +176520 +176520 +176520 +176520 +176540 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +179120 +187120 +189380 +191052 +192156 +193320 +194900 +195860 +198516 +201484 +202600 +202600 +202600 +202600 +202832 +202832 +202836 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +207784 +212136 +216320 +220508 +224696 +228876 +245088 +245088 +247844 +252032 +256212 +260400 +264592 +268776 +272776 +275060 +279244 +283428 +287616 +291032 +293900 +298080 +302272 +304364 +308548 +310644 +312740 +316924 +319016 +323208 +325296 +327392 +331572 +333668 +335760 +337856 +354328 +356424 +358520 +362700 +364792 +366892 +368984 +371080 +373168 +375260 +377356 +379448 +381540 +383636 +383636 +385732 +387820 +390032 +391160 +392292 +394552 +396816 +397092 +399072 +401340 +403600 +405860 +408008 +408148 +412640 +414900 +417164 +419420 +421680 +423944 +426204 +428464 +430724 +432768 +434980 +436476 +437932 +440332 +441984 +442740 +445152 +447688 +449148 +449960 +452436 +454712 +454896 +457180 +45 +459688 +462040 +463480 +464408 +466812 +467244 +469224 +471096 +471684 +474136 +474328 +476508 +478872 +481356 +483472 +483780 +486072 +488480 +489668 +490888 +493420 +495704 +496836 +498116 +500520 +502756 +503668 +505400 +507760 +509296 +510204 +512764 +514708 +515508 +517372 +519764 +520648 +522188 +524596 +525524 +527004 +529412 +534224 +536632 +538080 +539216 +541588 +542560 +543384 +543384 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519032 +519032 +519032 +519032 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519052 +519052 +519052 +519052 +519052
Re: [pypy-dev] [pypy-svn] r79382 - pypy/trunk/pypy/config
On Tue, Nov 23, 2010 at 6:52 PM, Armin Rigo ar...@tunes.org wrote: Hi Anto, On Tue, Nov 23, 2010 at 8:31 AM, Antonio Cuni anto.c...@gmail.com wrote: Disable jit debugging by default Not sure it's a good idea. It's useful to see the statistics at the end of the run. I know that you can turn it on explicitly, but it's easy to forget. What about having an env variable that automatically turns jit debug on? I guess it should just be another log category, like jit-summary. Then you can get the effect you want by setting (in the shell) PYPYLOG to jit-summary:-. This or an env variable. I find it really disruptive for everyday usage. Like py.test -k test_tab displaying jit log is not exactly what I want. You can turn it on, put an alias or log it somewhere. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Memory error when importing optparse module
I guess the first thing would be to build pypy svn yourself instead of using 1.3 and see if it helps On Sat, Nov 20, 2010 at 9:20 PM, Roman Prykhodchenko romcheg.pri...@gmail.com wrote: I built the executable using macports (port install pypy). My python is 64 bit. I can help you to find the bug if you help me to help you. - Roman On Nov 20, 2010, at 10:09 AM, Maciej Fijalkowski wrote: Hey. Unfortunately we don't have a full-time OS X developer and it makes it harder for this platform to be fully spported. Is your python 32bit or 64bit? How did you build that executable? On Fri, Nov 19, 2010 at 9:46 PM, Roman Prykhodchenko romcheg.pri...@gmail.com wrote: Hi guys, I'm trying to port my scripts to pypy but when I import the optparse module I get the error: import optparse Traceback (most recent call last): File console, line 1, in module File /opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py, line 84, in module from gettext import gettext File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py, line 49, in module import locale, copy, os, re, struct, sys File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py, line 30, in module from _locale import * File /opt/local/share/pypy-1.3/pypy/lib/_locale.py, line 6, in module from ctypes import (Structure, POINTER, create_string_buffer, File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py, line 489, in module _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr) File /opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py, line 104, in __init__ ffiargs, ffires, self._flags_) MemoryError OS: Mac OS Snow Leopard 10.6.5 I tried to sign up your bugtracker to report a bug but the registration is broken -- I got the error message when I was trying to confirm my registration. My login is prykhodchenko --- Roman Prykhodchenko ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Memory error when importing optparse module
PyPy is known not to work on mac os x 64bit. Blame a combination of crazy licensing schemes (we can't run a buildbot in a virtual machine nor we can debug it) with awkward outdated tools available on platform. Or blame us :) Generally someone has to fix this, but as I said before we don't have an OS X developer. Precise problem is some difference in generated assembler by gcc or platform needs. On Sun, Nov 21, 2010 at 2:31 PM, Roman Prykhodchenko romcheg.pri...@gmail.com wrote: I have already built it myself. I got lots of errors and warnings during the build process and as a result I have pypy-c that doesn't work. May be that was caused by --opt=jit on my x64 platform. Will try to go with --opt=2 a bit later and report you the result. - Roman On Nov 21, 2010, at 2:20 PM, Maciej Fijalkowski wrote: I guess the first thing would be to build pypy svn yourself instead of using 1.3 and see if it helps On Sat, Nov 20, 2010 at 9:20 PM, Roman Prykhodchenko romcheg.pri...@gmail.com wrote: I built the executable using macports (port install pypy). My python is 64 bit. I can help you to find the bug if you help me to help you. - Roman On Nov 20, 2010, at 10:09 AM, Maciej Fijalkowski wrote: Hey. Unfortunately we don't have a full-time OS X developer and it makes it harder for this platform to be fully spported. Is your python 32bit or 64bit? How did you build that executable? On Fri, Nov 19, 2010 at 9:46 PM, Roman Prykhodchenko romcheg.pri...@gmail.com wrote: Hi guys, I'm trying to port my scripts to pypy but when I import the optparse module I get the error: import optparse Traceback (most recent call last): File console, line 1, in module File /opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py, line 84, in module from gettext import gettext File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py, line 49, in module import locale, copy, os, re, struct, sys File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py, line 30, in module from _locale import * File /opt/local/share/pypy-1.3/pypy/lib/_locale.py, line 6, in module from ctypes import (Structure, POINTER, create_string_buffer, File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py, line 489, in module _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr) File /opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py, line 104, in __init__ ffiargs, ffires, self._flags_) MemoryError OS: Mac OS Snow Leopard 10.6.5 I tried to sign up your bugtracker to report a bug but the registration is broken -- I got the error message when I was trying to confirm my registration. My login is prykhodchenko --- Roman Prykhodchenko ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Memory error when importing optparse module
Hey. Unfortunately we don't have a full-time OS X developer and it makes it harder for this platform to be fully spported. Is your python 32bit or 64bit? How did you build that executable? On Fri, Nov 19, 2010 at 9:46 PM, Roman Prykhodchenko romcheg.pri...@gmail.com wrote: Hi guys, I'm trying to port my scripts to pypy but when I import the optparse module I get the error: import optparse Traceback (most recent call last): File console, line 1, in module File /opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py, line 84, in module from gettext import gettext File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py, line 49, in module import locale, copy, os, re, struct, sys File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py, line 30, in module from _locale import * File /opt/local/share/pypy-1.3/pypy/lib/_locale.py, line 6, in module from ctypes import (Structure, POINTER, create_string_buffer, File /opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py, line 489, in module _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr) File /opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py, line 104, in __init__ ffiargs, ffires, self._flags_) MemoryError OS: Mac OS Snow Leopard 10.6.5 I tried to sign up your bugtracker to report a bug but the registration is broken -- I got the error message when I was trying to confirm my registration. My login is prykhodchenko --- Roman Prykhodchenko ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Needs some guidance in writing the kqueue part of select
(part response) 2) How do I declare that an argument is const in rffi? usually you don't ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] gdbm
On Wed, Nov 17, 2010 at 1:24 AM, Dan Stromberg drsali...@gmail.com wrote: On Tue, Nov 16, 2010 at 8:38 AM, exar...@twistedmatrix.com wrote: On 04:27 pm, drsali...@gmail.com wrote: On Tue, Nov 16, 2010 at 4:17 AM, Antonio Cuni anto.c...@gmail.com wrote: Sounds fine, do you feel like implementing it? :-) Moreover, I also agree with amaury that your code is very similar to the one in the current dbm.py, so we should maybe try to refactor things to share common parts between the twos. I'd be happy to. Would sharing based on inheritance or a more functional approach be preferred? No, avoid inheritance where possible. Composition is preferred. Due to performance? Or flat is better than nested - as more of a philosophical issue? I'm not arguing for inheritance, just wanting to understand the reasoning behind it. I think one of the points is that those are originally not related in CPython. And someone somewhere *will* use this and complain. People do crazy stuff with Python. As for functional, I don't really know what approach you're describing with that word. I mean something like what you'd do in Lisp, ML or Haskell (not that I know any of those that well). I suppose in this case it would mean functools.partial() as a decorator, or perhaps a wrapper around functools.partial() used as a decorator. Then again, maybe functools.partial() imposes its own performance penalty? That's part of why I'm wondering if avoiding inheritance is a performance-related goal. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
[pypy-dev] Memory consumption of pypy-c translate.py
Hello. I did a bit of measurments using various techniques and pypy-c-64bit-jit and the outcome is that after a bit of annotation, where memory consumption is 1200M, we get: * 31M of mmaped code blocks for assembler * about 150M I can account for that is jit resume data * about 150M of the rest I can account for. Where is everything else? Can GC trash some memory (like a lot). Here is output of dump.py -r http://codespeak.net/~fijal/jit64.out ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy JIT C extensions
On Mon, Oct 25, 2010 at 2:08 PM, Armin Rigo ar...@tunes.org wrote: Hi, On Mon, Oct 25, 2010 at 5:12 AM, Andrew Francis andrewfr_...@yahoo.com wrote: A few days ago, I floated this idea in the #pypy IRC channel. Why can't we take a JITed pypy, install the greenlet package, run it and see what happens? You will just get strange segfaults and no way to understand from outside where they come from. I can give some more details about them, but I guess it would not be too helpful. In summary, C extension modules have a chance to work with PyPy as long as they don't play strange tricks. Hey. I had a private email exchange with Andrew and I think what he wants to achieve is a working greenlet module for PyPy. Is a current C version a good start, or something else would be a good start? Cheers, fijal A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] r78267 - pypy/branch/arm-backend/pypy/jit/backend/arm/tools
Hey.
Can we make this directory 'tool' instead of 'tools' like every other
tool directory?
Cheers,
fijal
On Mon, Oct 25, 2010 at 4:45 PM, da...@codespeak.net wrote:
Author: david
Date: Mon Oct 25 16:45:58 2010
New Revision: 78267
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/tools/
pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py (contents,
props changed)
Log:
Wrapper for calling objdump on dumped memory
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py
==
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py Mon
Oct 25 16:45:58 2010
@@ -0,0 +1,5 @@
+#!/usr/bin/env python
+import os
+import sys
+
+os.system('objdump -D --architecture=arm --target=binary %s' % sys.argv[1])
___
pypy-svn mailing list
pypy-...@codespeak.net
http://codespeak.net/mailman/listinfo/pypy-svn
___
pypy-dev@codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] r77933 - pypy/branch/jitffi/pypy/jit/metainterp/test
The exception still can't happen though. On Thu, Oct 14, 2010 at 3:32 PM, antoc...@codespeak.net wrote: Author: antocuni Date: Thu Oct 14 15:32:22 2010 New Revision: 77933 Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Log: update this test too Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py == --- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (original) +++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Thu Oct 14 15:32:22 2010 @@ -77,7 +77,9 @@ call(0, p2, descr=libffi_prepare) call(0, p2, i0, descr=libffi_push_arg) call(0, p2, f1, descr=libffi_push_arg) - i3 = call(0, p2, 12345, descr=libffi_call) + i3 = call_may_force(0, p2, 12345, descr=libffi_call) + guard_not_forced() [] + guard_no_exception() [] jump(i3, f1, p2) expected = ops ___ pypy-svn mailing list pypy-...@codespeak.net http://codespeak.net/mailman/listinfo/pypy-svn ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] issue tracker spam issues / switching
On Sun, Oct 10, 2010 at 9:37 AM, holger krekel hol...@merlinux.eu wrote: Hi all, seems like the pypy-dev issue tracker got some spam through bot-registered users. Unless i am mistaken there is no easy way to add a captcha in addition to the e-mail confirmation registration and so i made the default role for new issue tracker users anonymous. This means new users need to find an admin who assigns the role User. Admins on the trackers are e.g. Armin, Maciej, Anto, Amaury, myself and probably others. Not a great solution, i know. It doesn't affect any of the existing users, though. it's probably a good occassion to push for switching to using new issue tracker software especially since there has been other complaints as well. Hey. Yes, I think so, especially if we can give handling of spam to someone else ;-) I did a bit of research at some point and personally I would like to give JIRA a try. It looks good, but it's not open source (it's free for open source users though). Cheers, fijal best, holger ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-svn] r77589 - pypy/trunk/pypy/jit/metainterp
On Mon, Oct 4, 2010 at 11:56 PM, Antonio Cuni anto.c...@gmail.com wrote: On 04/10/10 23:32, fi...@codespeak.net wrote: def transform(op): from pypy.jit.metainterp.history import AbstractDescr - # Rename CALL_PURE to CALL. + # Rename CALL_PURE and CALL_INVARIANT to CALL. # Simplify the VIRTUAL_REF_* so that they don't show up in the backend. - if op.getopnum() == rop.CALL_PURE: + if (op.getopnum() == rop.CALL_PURE or + op.getopnum() == rop.CALL_LOOPINVARIANT): op = ResOperation(rop.CALL, op.getarglist()[1:], op.result, op.getdescr()) elif op.getopnum() == rop.VIRTUAL_REF: Hi, I just wanted to tell that since the resoperation-refactoring, there is now a nice copy_and_change method to be used exactly for the case where you want to replace an operation with a similar one. So, the code above should be written like: op = op.copy_and_change(rop.CALL, args=op.getarglist()[1:]) ah, great! :) ciao, Anto ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Question on the future of RPython
On Tue, Sep 28, 2010 at 2:43 AM, Terrence Cole list-s...@trainedmonkeystudios.org wrote: On Tue, 2010-09-28 at 01:57 +0200, Jacob Hallén wrote: Monday 27 September 2010 you wrote: On Sun, 2010-09-26 at 23:57 -0700, Saravanan Shanmugham wrote: Well, I am happy to see that the my interest in a general purpose RPython is not as isolated as I was lead to believe :-)) Thx, What I wrote has apparently been widely misunderstood, so let me explain what I mean in more detail. What I want is _not_ RPython and it is _not_ Shedskin. What I want is not a compiler at all. What I want is a visual tool, for example, a plugin to an IDE. This tool would perform static analysis on a piece of python code. Instead of generating code with this information, it would mark up the python code in the text display with colors, weights, etc in order to show properties from the static analysis. This would be something like semantic highlighting, as opposed to syntax highlighting. I think it possible that this information would, if created and presented in the correct way, represent the sort of optimizations that pypy-c-jit -- a full python implementation, not a language subset -- would likely perform on the code if run. Given this sort of feedback, it would be much easier for a python coder to write code that works well with the jit: for example, moving a declaration inside a loop to avoid boxing, based on the information presented. Ideally, such a tool would perform instantaneous syntax highlighting while editing and do full parsing and analysis in the background to update the semantic highlighting as frequently as possible. Obviously, detailed static analysis will provide far more information than it would be possible to display on the code at once, so I see this gui as having several modes -- like predator vision -- that show different information from the analysis. Naturally, what those modes are will depend strongly on the details of how pypy-c-jit works internally, what sort of information can be sanely collected through static analysis, and, naturally, user testing. I was somewhat baffled at first as to how what I wrote before was interpreted as interest in a static python. I think the disconnect here is the assumption on many people's part that a static language will always be faster than a dynamic one. Given the existing tools that provide basically no feedback from the compiler / interpreter / jitter, this is inevitably true at the moment. I foresee a future, however, where better tools let us use the full power of a dynamic python AND let us tighten up our code for speed to get the full advantages of jit compilation as well. I believe that in the end, this combination will prove superior to any fully static compiler. -Terrence Sarvi - Original Message From: Terrence Cole list-s...@trainedmonkeystudios.org To: pypy-dev@codespeak.net Sent: Sun, September 26, 2010 2:28:12 PM Subject: Re: [pypy-dev] Question on the future of RPython On Sat, 2010-09-25 at 17:47 +0200, horace grant wrote: i just had a (probably) silly idea. :) if some people like rpython so much, how about writing a rpython interpreter in rpython? wouldn't it be much easier for the jit to optimize rpython code? couldn't jitted rpython code theoretically be as fast as a program that got compiled to c from rpython? hm... but i wonder if this would make sense at all. maybe if you ran rpython code with pypy-c-jit, it already could be jitted as well as with a special rpython interpreter? ...if there were a special rpython interpreter, would the current jit generator have to be changed to take advantage of the more simple language? An excellent question at least. A better idea, I think, would be to ask what subset of full-python will jit well. What I'd really like to see is a static analyzer that can display (e.g. by coloring names or lines) how jit friendly a piece of python code is. This would allow a programmer to get an idea of what help the jit is going to be when running their code and, hopefully, help people avoid tragic performance results. Naturally, for performance intensive code, you would still need to profile, but for a lot of uses, simply not having catastrophically bad performance is more than enough for a good user experience. With such a tool, it wouldn't really matter if the answer to what is faster is RPython -- it would be whatever python language subset happens to work well in a particular case. I've started working on something like this [1], but given that I'm doing a startup, I don't have nearly the time I would need to make this useful in the near-term. The JIT works because it has more information at runtime than what is
Re: [pypy-dev] PyPy JIT C extensions, greenlet
Hey. greenlet C module is quite incompatible with pypy and won't work. However making pypy work with jit and stackless is something that requires a bit of work only (teaching jit how to unroll the stack mostly) and I plan to look into it in the very near future. Cheers, fijal On Mon, Sep 27, 2010 at 1:49 PM, Ian P. Cooke i...@srand.net wrote: There was a recent thread with the same subject and I would like to look into this a bit more. I knew pypy-stackless wouldn't work after I built a working 64-bit pypy w/ JIT, well, now I'm intrigued. I will look at the code more closely soon. Armin, Carl Friedrich, would you answer a couple of questions in the mean-time? What is the largest roadblock to making pypy-stackless work on pypy w/ JIT? Would it be possible/easier to port the greenlet module? Having built-in support for co-routines would be very nice but my own goal is to get greenlet working in any manner. If I could build a 64-bit pypy w/ JIT and then easy_install greenlet, that would work for me. Thanks, Ian P.S. congratulations on all your recent progress! I always look forward for the next pypy blog update :) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy JIT C extensions, greenlet
PyPy stackless does support greenlets. PyPy would not work with CPython C module called greenlets. On Mon, Sep 27, 2010 at 2:29 PM, Andy angelf...@yahoo.com wrote: Why wouldn't pypy work with greenlet but would work with Stackless? greenlet calls itself a spin-off of Stackless. Isn't greenlet a subset of Stackless without the scheduling? Could you explain a bit more? Thanks. --- On Mon, 9/27/10, Maciej Fijalkowski fij...@gmail.com wrote: From: Maciej Fijalkowski fij...@gmail.com Subject: Re: [pypy-dev] PyPy JIT C extensions, greenlet To: Ian P. Cooke i...@srand.net Cc: pypy-dev@codespeak.net Date: Monday, September 27, 2010, 8:23 AM Hey. greenlet C module is quite incompatible with pypy and won't work. However making pypy work with jit and stackless is something that requires a bit of work only (teaching jit how to unroll the stack mostly) and I plan to look into it in the very near future. Cheers, fijal On Mon, Sep 27, 2010 at 1:49 PM, Ian P. Cooke i...@srand.net wrote: There was a recent thread with the same subject and I would like to look into this a bit more. I knew pypy-stackless wouldn't work after I built a working 64-bit pypy w/ JIT, well, now I'm intrigued. I will look at the code more closely soon. Armin, Carl Friedrich, would you answer a couple of questions in the mean-time? What is the largest roadblock to making pypy-stackless work on pypy w/ JIT? Would it be possible/easier to port the greenlet module? Having built-in support for co-routines would be very nice but my own goal is to get greenlet working in any manner. If I could build a 64-bit pypy w/ JIT and then easy_install greenlet, that would work for me. Thanks, Ian P.S. congratulations on all your recent progress! I always look forward for the next pypy blog update :) ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] External RPython mailing list
On Mon, Sep 13, 2010 at 10:08 AM, Armin Rigo ar...@tunes.org wrote: Hi Maciej, On Mon, Sep 13, 2010 at 10:03 AM, Maciej Fijalkowski fij...@gmail.com wrote: While we're at it, how about splitting the translation toolchain from pypy interpreter? I don't mean on technical merits, it can still be the same or mostly the same source codebase, but more on the conceptual level, to have 2 different websites names etc. I don't care too much right now. My motivation was to make RPython *less* visible, not create a second website for the translation toolchain (which would make RPython more visible). I don't think it's hideable. What we can do instead is to leave some kind of description why it is like it is and what it is. Trying to hide it means to some people that we have an awesome tool that we don't want to share. Instead it's worth explaining why we don't share this (because it's eg hard to use) A bientôt, Armin. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] PyPy JIT and Django
On Thu, Sep 9, 2010 at 10:37 PM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2010/9/9 Andy angelf...@yahoo.com: Thanks Antonio. So right now there's no way to run PyPy JIT on Linux 64 bit? Yes there is! Armin merged the asmgcc-64 branch yesterday. You have to use trunk version of PyPy, and build it yourself. Although I would mark this support as experimental for now, it works :-) -- Amaury Forgeot d'Arc ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Question on the future of RPython
On Thu, Sep 2, 2010 at 9:56 AM, Paolo Giarrusso p.giarru...@gmail.com wrote: Hi, I was curious about the interplay between type inference and separate compilation. On Thu, Sep 2, 2010 at 09:09, William Leslie william.leslie@gmail.com wrote: At the moment, the lack of separate compilation is a real issue standing in the way of using rpython as a general purpose language, or even as an extension language. Having to re-translate *everything* every time you want to install an extension module is not on. Even C doesn't require that. The other is that type inference is global and changes you make to one function can have far-reaching consequences. Is it module-global or is it performed on the whole program? I guess you'd need modular type inference before allowing separate compilation, and of course lots of implementation work. Functional languages allow separate compilation - is there any RPython-specific problem for that? I've omitted my guesses here. There is no notion of a module in RPython. RPython is compiled from live python objects (hence python is a metaprogramming language for RPython). There is a bunch of technical problems, but it's generally possible to implement separate compilation (it's work though). Cheers, fijal -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Re: [pypy-dev] Loop invaraints
On Tue, Aug 31, 2010 at 9:20 AM, Hakan Ardo ha...@debian.org wrote: On Sun, Aug 29, 2010 at 1:04 PM, Armin Rigo ar...@tunes.org wrote: I'm not saying that loop-invariant code motion could also have a negative effect on large loops; I think it's a pure win, so it's probably worth a try. I'm just giving a warning: it may not help much in the case of a general Python program doing lots of stuff, but only in the case of small numerical computation loops. Right. I write a lot of numerical computation loops these days, both small and somewhat bigger, and I am typically force to write them in C to get decent performance. So the motivation here would rater be to broaden the usability of python than to improve performance of exciting python programs. Another motivation might be to help pypy developers focus on the important instruction while staring at traces, ie by hiding the instructions that will be inserted only once :) I second hakan here - small loops are not uninteresting, since it broadens areas where you can use python, not limiting yourself to existing python programs. ___ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
