On Tue, 2 Feb 2016 at 01:29 Victor Stinner <victor.stin...@gmail.com> wrote:
> Hi, > > I'm back for the FOSDEM event at Bruxelles, it was really cool. I gave > talk about FAT Python and I got good feedback. But friends told me > that people now have expectations on FAT Python. It looks like people > care of Python performance :-) > > FYI the slides of my talk: > https://github.com/haypo/conf/raw/master/2016-FOSDEM/fat_python.pdf > (a video was recorded, I don't know when it will be online) > > I take a first look at your patch and sorry, I'm skeptical about the > design. I have to play with it a little bit more to check if there is > no better design. > > To be clear, FAT Python with your work looks more and more like a > cheap JIT compiler :-) Guards, specializations, optimizing at runtime > after a threshold... all these things come from JIT compilers. I like > the idea of a kind-of JIT compiler without having to pay the high cost > of a large dependency like LLVM. I like baby steps in CPython, it's > faster, it's possible to implement it in a single release cycle (one > minor Python release, Python 3.6). Integrating a JIT compiler into > CPython already failed with Unladen Swallow :-/ > > PyPy has a complete different design (and has serious issues with the > Python C API), Pyston is restricted to Python 2.7, Pyjion looks > specific to Windows (CoreCLR), Numba is specific to numeric > computations (numpy). IMHO none of these projects can be easily be > merged into CPython "quickly" (again, in a single Python release > cycle). By the way, Pyjion still looks very young (I heard that they > are still working on the compatibility with CPython, not on > performance yet). > We are not ready to have a serious discussion about Pyjion yet as we are still working on compatibility (we have a talk proposal in for PyCon US 2016 and so we are hoping to have something to discuss at the language summit), but Victor's email shows there is some misconceptions about it already and a misunderstanding of our fundamental goal. First off, Pyjion is very much a work-in-progress. You can find it at https://github.com/microsoft/pyjion (where there is an FAQ), but for this audience the key thing to know is that we are still working on compatibility (see https://github.com/Microsoft/Pyjion/blob/master/Tests/python_tests.txt for the list of tests we do (not) pass from the Python test suite). Out of our roughly 400 tests, we don't pass about 18 of them. Second, we have not really started work on performance yet. We have done some very low-hanging fruit stuff, but just barely. IOW we are not really ready to discuss performance (ATM we JIT instantly for all code objects and even being that aggressive with the JIT overhead we are even/slightly slower than an unmodified Python 3.5 VM, so we are hopeful this work will pan out). Third, the over-arching goal of Pyjion is not to add a JIT into CPython, but to add a C API to CPython that will allow plugging in a JIT. If you simply JIT code objects then the API required to let someone plug in a JIT is basically three functions, maybe as little as two (you can see the exact patch against CPython that we are working with at https://github.com/Microsoft/Pyjion/blob/master/Patches/python.diff). We have no interest in shipping a JIT with CPython, just making it much easier to let others add one if they want to because it makes sense for their workload. We have no plans to suggest shipping a JIT with CPython, just to make it an option for people to add in if they want (and if Yury's caching stuff goes in with an execution counter then even the one bit of true overhead we had will be part of CPython already which makes it even more of an easy decision to consider the API we will eventually propose). Fourth, it is not Windows-only by design. CoreCLR is cross-platform on all major OSs, so that is not a restriction (and honestly we are using CoreCLR simply because Dino used to work on the CLR team so he knows the bytecode really well; we easily could have used some other JIT to prove our point). The only reason Pyjion doesn't work with other OSs is momenum/laziness on Dino and my part; Dino hacked together Pyjion at PyCon US 2015 and he is the most comfortable on Windows, and so he just did it in Windows on Visual Studio and just didn't bother to start with e.g., CMake to make it build on other OSs. Since we are still trying to work out some compatibility stuff so we would rather do that than worry about Linux or OS X support right now. Fifth, if we manage to show that a C API can easily be added to CPython to make a JIT something that can simply be plugged in and be useful, then we will also have a basic JIT framework for people to use. As I said, our use of CoreCLR is just for ease of development. There is no reason we couldn't use ChakraCore, v8, LLVM, etc. But since all of these JIT compilers would need to know how to handle CPython bytecode, we have tried to design a framework where JIT compilers just need a wrapper to handle code emission and our framework that we are building will handle driving the code emission (e.g., the wrapper needs to know how to emit add_integer(), but our framework handles when to have to do that). Anyway, as I said, Pyjion is very much a work in progress. We hope to have something more solid to propose/discuss at the language summit at PyCon US 2016. The only reason I keep mentioning it is because what Victor is calling "JIT-like" is really "minimize doing extra work that's not needed" and that benefits everyone trying to do any computational work that takes extra time to speed up CPython (which includes Pyjion). IOW Yury's work combined with Victor's work could quite easily just spill out beyond just local caches and into allowing pluggable JITs in CPython.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
