Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-07-17 Thread Mark Dickinson
On Sun, Jul 15, 2012 at 1:36 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 15 Jul 2012 18:47:38 +1000 Nick Coghlan ncogh...@gmail.com wrote: I'm not seeing the value in returning None over 0 for the don't know case - it just makes the API harder to use. The point is that 0 is a

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-07-17 Thread Antoine Pitrou
On Tue, 17 Jul 2012 13:19:55 +1000 Nick Coghlan ncogh...@gmail.com wrote: There are no provisions for infinite iterators, that is not within the scope of this proposal. I'll repeat my observation that remaining silent on this point is effectively identical to blessing the practice of

Re: [Python-Dev] io.BytesIO slower than monkey-patching io.RawIOBase

2012-07-17 Thread Antoine Pitrou
On Tue, 17 Jul 2012 06:34:14 +0300 Eli Bendersky eli...@gmail.com wrote: Is there any reason for this to be so? What does BytesIO give us that the second approach does not (I tried adding more methods to the patched RawIOBase to make it more functional, like seekable() and tell(), and it

Re: [Python-Dev] io.BytesIO slower than monkey-patching io.RawIOBase

2012-07-17 Thread Serhiy Storchaka
On 17.07.12 06:34, Eli Bendersky wrote: The second approach is consistently 10-20% faster than the first one (depending on input) for trunk Python 3.3 Is there any reason for this to be so? What does BytesIO give us that the second approach does not (I tried adding more methods to the patched

Re: [Python-Dev] io.BytesIO slower than monkey-patching io.RawIOBase

2012-07-17 Thread Eli Bendersky
On Tue, Jul 17, 2012 at 2:59 PM, Serhiy Storchaka storch...@gmail.comwrote: On 17.07.12 06:34, Eli Bendersky wrote: The second approach is consistently 10-20% faster than the first one (depending on input) for trunk Python 3.3 Is there any reason for this to be so? What does BytesIO give us

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-07-17 Thread Lennart Regebro
On Sun, Jul 15, 2012 at 1:28 AM, Alexandre Zani alexandre.z...@gmail.com wrote: I'm +1 on not having a public API for this. Ultimately the contract for a length hint will depend heavily upon what you need it for. Some applications would require a length hint to be an at least others an at most

[Python-Dev] Use function names instead of functions for os.supports_dir_fd?

2012-07-17 Thread Victor Stinner
Hi, Python 3.3 introduced os.supports_dir_fd to check if some os functions do accept a file descriptor instead of a path. The problem is that os.supports_dir_fd is a list of functions, not a list of function names. If os functions are monkey patched, you cannot test anymore if a function supports

[Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Victor Stinner
Hi, I would like to write yet another JIT compiler for CPython. Before writing anything, I would like your opinion because I don't know well other Python compilers. I also want to prepare a possible integration into CPython since the beginning of the project, or at least stay very close to the

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Stefan Behnel
Victor Stinner, 17.07.2012 20:38: -- Subset of Python -- * `pymothoa http://code.google.com/p/pymothoa/`_: use LLVM; don't support classes nor exceptions. * `unpython http://code.google.com/p/unpython/`_: Python to C * `Perthon http://perthon.sourceforge.net/`_: Python to Perl *

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Dag Sverre Seljebotn
I'll admit I didn't read through your email, but you should absolutely check out Numba which is ramping up just now to do this: https://github.com/numba (I'm CC-ing their mailing list, perhaps some of them will read this and respond.) It is probably much less ambitious but that hopefully

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Amaury Forgeot d'Arc
Hi, 2012/7/17 Victor Stinner victor.stin...@gmail.com: -- Milestone 3: JIT -- * Depending on the type seen at runtime, recompile the function to generate specialized functions * Use guard to fallback to a generic implementation if the type is not the expected type From my understanding,

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Maciej Fijalkowski
Hi Victor. I'm willing to explain to you details why having LLVM does not solve almost any issues and why PyPy is complex, or why you think it's complex. Find me on IRC if you want (fijal, can be found on #pypy on freenode for example). In our opinion something like psyco that gets brought to the

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Stefan Behnel
Maciej Fijalkowski, 17.07.2012 21:16: It would be much better to concentrate efforts rather than write yet another half-finished JIT (because reading code is hard). +1 Stefan ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Glyph
On Jul 17, 2012, at 11:38 AM, Victor Stinner victor.stin...@gmail.com wrote: IMO PyPy is complex and hard to maintain. PyPy has a design completly different than CPython and is much faster and has a better memory footprint. I don't expect to be as fast as PyPy, just faster than CPython. I

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread martin
I would like to write yet another JIT compiler for CPython. FWIW, so do I. I did not understand exactly why Unladen Swallow and psyco projects failed, so please tell me if you think that my project is going to fail too! It may well happen that your project fails, or doesn't even start. Mine

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread martin
If you disagree, you should feel no need to convince me; just go do it and prove me wrong, which I will be quite happy to be. I would just like to think about whether this is the best use of your energy first. While I follow most of your reasoning, I think this is a flaw in your logic.

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Amaury Forgeot d'Arc
2012/7/18 mar...@v.loewis.de: I would like to write yet another JIT compiler for CPython. FWIW, so do I. I don't know whether it's good news (that Martin wants to put his expertise in this area) or a bad sign (that he did not start after so many years of Python development - the problem

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Victor Stinner
Personally, I like the idea of having a JIT compiler more or less as an extension module at hand. Sort-of like a co-processor, just in software. Lets you run your code either interpreter or JITed, just as you need. Me too, so something like psyco. LLVM is written in C++ and may have license

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Victor Stinner
It's the JIT compiler of Unladen Swallow that failed; in my understanding because LLVM is crap (i.e. it is slow, memory-consuming, and buggy) - as a low-level virtual machine; it may be ok as a compiler backend (but I still think it is buggy there as well). What is the status of LLVM

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Nick Coghlan
On the cpyext front, it would be rather helpful if developers interested in a high speed Python interpreter with good C extension compatibility worked with Dave Malcolm on his static analyser for Python C extensions. One of the reasons cpyext has trouble is that many refcounting bugs in extensions

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Devin Jeanpierre
On Tue, Jul 17, 2012 at 6:20 PM, Victor Stinner victor.stin...@gmail.com wrote: What is the status of LLVM nowadays? Is it not a good solution to write a portable JIT? I don't want to write my own library to generate machine code. You don't have to, even if you don't want to use LLVM. There

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Amaury Forgeot d'Arc
2012/7/18 Nick Coghlan ncogh...@gmail.com: On the cpyext front, it would be rather helpful if developers interested in a high speed Python interpreter with good C extension compatibility worked with Dave Malcolm on his static analyser for Python C extensions. One of the reasons cpyext has

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Yury Selivanov
On 2012-07-17, at 6:38 PM, Devin Jeanpierre wrote: On Tue, Jul 17, 2012 at 6:20 PM, Victor Stinner victor.stin...@gmail.com wrote: What is the status of LLVM nowadays? Is it not a good solution to write a portable JIT? I don't want to write my own library to generate machine code. You

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Victor Stinner
As your original message shows, there has already been enough duplication of effort in this area. I didn't find yet a project reusing ceval.c: most projects implement their own eval loop and don't use CPython at all. My idea is not to write something new, but just try to optimize the existing

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Mark Lawrence
On 17/07/2012 23:20, Victor Stinner wrote: http://psyco.sourceforge.net/ says: News, 12 March 2012 Psyco is unmaintained and dead. Please look at PyPy for the state-of-the-art in JIT compilers for Python. Victor A search on pypi for JIT compilers gives no matches. -- Cheers. Mark

Re: [Python-Dev] Use function names instead of functions for os.supports_dir_fd?

2012-07-17 Thread Larry Hastings
On 07/17/2012 10:34 AM, Victor Stinner wrote: Python 3.3 introduced os.supports_dir_fd to check if some os functions do accept a file descriptor instead of a path. The problem is that os.supports_dir_fd is a list of functions, not a list of function names. If os functions are monkey patched, you

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Steven D'Aprano
Victor Stinner wrote: == Other Python VM and compilers == As far as I know, these are all still active, although possibly experimental: Pynie (Python for the Parrot VM) WPython (16-bit word-codes instead of byte-codes) HotPy (high-performance

Re: [Python-Dev] Use function names instead of functions for os.supports_dir_fd?

2012-07-17 Thread Steven D'Aprano
Victor Stinner wrote: Hi, Python 3.3 introduced os.supports_dir_fd to check if some os functions do accept a file descriptor instead of a path. The problem is that os.supports_dir_fd is a list of functions, not a list of function names. If os functions are monkey patched, you cannot test

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Larry Hastings
On 07/17/2012 04:34 PM, Steven D'Aprano wrote: As far as I know, these are all still active, although possibly experimental: Pynie (Python for the Parrot VM) WPython (16-bit word-codes instead of byte-codes) [...] WPython in particular seems to be very promising, and

Re: [Python-Dev] Use function names instead of functions for os.supports_dir_fd?

2012-07-17 Thread Victor Stinner
Monkey patching is a common practice in Python. test_os.py replaces os.exec*() functions temporary for example. Perhaps for testing, but I don't think monkey-patching is common in production code. Perhaps you are thinking of Ruby :) The gevent library does monkey-patch os.fork (and

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Alex Gaynor
Victor Stinner victor.stinner at gmail.com writes: Example: a = GETLOCAL(0); # a if (a == NULL) /* error */ b = GETLOCAL(1); # b if (b == NULL) /* error */ return PyNumber_Add(a, b); I don't expect to run a program 10x faster, but I would be happy if I can run arbitrary

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Stefan Behnel
Victor Stinner, 18.07.2012 00:15: Personally, I like the idea of having a JIT compiler more or less as an extension module at hand. Sort-of like a co-processor, just in software. Lets you run your code either interpreter or JITed, just as you need. Me too, so something like psyco. In the

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Stefan Behnel
Alex Gaynor, 18.07.2012 03:24: Victor Stinner writes: Example: a = GETLOCAL(0); # a if (a == NULL) /* error */ b = GETLOCAL(1); # b if (b == NULL) /* error */ return PyNumber_Add(a, b); I don't expect to run a program 10x faster, but I would be happy if I can run arbitrary

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread Cesare Di Mauro
2012/7/18 Victor Stinner victor.stin...@gmail.com I don't expect to run a program 10x faster, but I would be happy if I can run arbitrary Python code 25% faster. If that's your target, you don't need to resort to a bytecode-to-binary-equivalent compiler. WPython already gave similar results

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread martin
What is the status of LLVM nowadays? Is it not a good solution to write a portable JIT? I don't think it is. It is still slow and memory hungry. The fact that the version that Apple ships with Xcode still miscompiles Python 3.3 tells me that it is still buggy. I don't want to write my own

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread martin
Zitat von Mark Lawrence breamore...@yahoo.co.uk: On 17/07/2012 23:20, Victor Stinner wrote: http://psyco.sourceforge.net/ says: News, 12 March 2012 Psyco is unmaintained and dead. Please look at PyPy for the state-of-the-art in JIT compilers for Python. Victor A search on pypi for JIT

Re: [Python-Dev] A new JIT compiler for a faster CPython?

2012-07-17 Thread martin
Please, learn from Unladen Swallow and other's experiences, otherwise they're for naught, and frankly we (python-dev) waste a lot of time. Again: we (python-dev) won't waste much time (unless we chose to in discussions); Victor may lose time, but then he may not. Regards, Martin