On Tue, Sep 28, 2010 at 2:43 AM, Terrence Cole <[email protected]> wrote: > On Tue, 2010-09-28 at 01:57 +0200, Jacob Hallén wrote: >> Monday 27 September 2010 you wrote: >> > On Sun, 2010-09-26 at 23:57 -0700, Saravanan Shanmugham wrote: >> > > Well, I am happy to see that the my interest in a general purpose RPython >> > > is not as isolated as I was lead to believe :-)) >> > > Thx, >> > >> > What I wrote has apparently been widely misunderstood, so let me explain >> > what I mean in more detail. What I want is _not_ RPython and it is >> > _not_ Shedskin. What I want is not a compiler at all. What I want is a >> > visual tool, for example, a plugin to an IDE. This tool would perform >> > static analysis on a piece of python code. Instead of generating code >> > with this information, it would mark up the python code in the text >> > display with colors, weights, etc in order to show properties from the >> > static analysis. This would be something like semantic highlighting, as >> > opposed to syntax highlighting. >> > >> > I think it possible that this information would, if created and >> > presented in the correct way, represent the sort of optimizations that >> > pypy-c-jit -- a full python implementation, not a language subset -- >> > would likely perform on the code if run. Given this sort of feedback, >> > it would be much easier for a python coder to write code that works well >> > with the jit: for example, moving a declaration inside a loop to avoid >> > boxing, based on the information presented. >> > >> > Ideally, such a tool would perform instantaneous syntax highlighting >> > while editing and do full parsing and analysis in the background to >> > update the semantic highlighting as frequently as possible. Obviously, >> > detailed static analysis will provide far more information than it would >> > be possible to display on the code at once, so I see this gui as having >> > several modes -- like predator vision -- that show different information >> > from the analysis. Naturally, what those modes are will depend strongly >> > on the details of how pypy-c-jit works internally, what sort of >> > information can be sanely collected through static analysis, and, >> > naturally, user testing. >> > >> > I was somewhat baffled at first as to how what I wrote before was >> > interpreted as interest in a static python. I think the disconnect here >> > is the assumption on many people's part that a static language will >> > always be faster than a dynamic one. Given the existing tools that >> > provide basically no feedback from the compiler / interpreter / jitter, >> > this is inevitably true at the moment. I foresee a future, however, >> > where better tools let us use the full power of a dynamic python AND let >> > us tighten up our code for speed to get the full advantages of jit >> > compilation as well. I believe that in the end, this combination will >> > prove superior to any fully static compiler. >> > >> > -Terrence >> > >> > > Sarvi >> > > >> > > >> > > ----- Original Message ---- >> > > >> > > > From: Terrence Cole <[email protected]> >> > > > To: [email protected] >> > > > Sent: Sun, September 26, 2010 2:28:12 PM >> > > > Subject: Re: [pypy-dev] Question on the future of RPython >> > > > >> > > > On Sat, 2010-09-25 at 17:47 +0200, horace grant wrote: >> > > > > i just had a (probably) silly idea. :) >> > > > > >> > > > > if some people like rpython so much, how about writing a rpython >> > > > > interpreter in rpython? wouldn't it be much easier for the jit to >> > > > > optimize rpython code? couldn't jitted rpython code theoretically be >> > > > > as fast as a program that got compiled to c from rpython? >> > > > > >> > > > > hm... but i wonder if this would make sense at all. maybe if you ran >> > > > > rpython code with pypy-c-jit, it already could be jitted as well as >> > > > > with a special rpython interpreter? ...if there were a special >> > > > > rpython interpreter, would the current jit generator have to be >> > > > > changed to take advantage of the more simple language? >> > > > >> > > > An excellent question at least. >> > > > >> > > > A better idea, I think, would be to ask what subset of full-python >> > > > will jit well. What I'd really like to see is a static analyzer that >> > > > can display (e.g. by coloring names or lines) how "jit friendly" a >> > > > piece of python code is. This would allow a programmer to get an >> > > > idea of what help the jit is going to be when running their code and, >> > > > hopefully, help people avoid tragic performance results. Naturally, >> > > > for performance intensive code, you would still need to profile, but >> > > > for a lot of uses, simply not having catastrophically bad performance >> > > > is more than enough for a good user experience. >> > > > >> > > > With such a tool, it wouldn't really matter if the answer to "what is >> > > > faster" is RPython -- it would be whatever python language subset >> > > > happens to work well in a particular case. I've started working on >> > > > something like this [1], but given that I'm doing a startup, I don't >> > > > have nearly the time I would need to make this useful in the >> > > > near-term. >> >> The JIT works because it has more information at runtime than what is >> available at compile time. If the information was available at compile time >> we >> could do the optimizations then and not have to invoke the extra complexity >> required by the JIT. Examples of the extra information include things like >> knowing that introspection will not be used in the current evaluation of a >> loop, specific argument types will be used in calls and that some arguments >> will be known to be constant over part of the program execution.. Knowing >> these bits allows you to optimize away large chunks o f the code that >> otherwise would have been executed. >> >> Static analysis assumes that none of the above mentioned possibilities can >> actually take place. It is impossible to make such assumptions at compile >> time >> in a dynamic language. Therefore PyPy is a bad match for people wanting to >> staically compile subsets of Python. Applying the JIT to RPython code > > Yes, that idea is just dumb. It's also not what I suggested at all. I > can see now that what I said would be easy to misinterpret, but on > re-reading it, it clearly doesn't say what you think it does. > >> is not >> workable, because the JIT is optimized to remove bits of generated assembler >> code that never shows up in the compilation of RPython code. >> >> These are very basic first principle concepts, and it is a mystery to me why >> people can't work them out for themselves. > > You are quite right that static analysis will be able to do little to > help an optimal jit. However, I doubt that in the near term pypy's jit > will cover all the dark corners of python equally well -- C has been > around for 38 years and its still got room for optimization. > > -Terrence > >> Jacob Hallén > > > _______________________________________________ > [email protected] > http://codespeak.net/mailman/listinfo/pypy-dev
Hey. I'm really interested in having jit feedback displayed as text info (say for profiling purposes). Do you have any particular ideas in mind or just a general one? _______________________________________________ [email protected] http://codespeak.net/mailman/listinfo/pypy-dev
