Hello Charles and Thomas, On 31 August 2014 00:04, Thomas Wuerthinger <thomas.wuerthin...@oracle.com> wrote:
> Thanks a lot, Charlie, for this very detailed feedback! In fact, this is > probably the most comprehensive feedback we’ve received for Truffle so far > :). > > There are some valid points, some points where I’d like to add a comment, > and some where Graal and Truffle may have been misunderstood. I’ll try to > address them in a similarly structured form: > > Regarding disadvantage 1 “AST is not enough”: > A. Yes, you need to provide specialisations for your operations for better > performance. This makes it harder to implement a Truffle AST interpreter > than a simple AST interpreter. It has however the advantage that it gives > you more predictable performance for the different usages of your dynamic > language operation. We are currently working on source code visualisations > for Truffle ASTs so that users can see what nodes are specialised and to > what types. > B. Yes, it is necessary to store your local variables in a Truffle frame > object. This object can however contain pointers to arbitrary extra data > necessary for your guest language. > C. We did a lot of improvements to Truffle, Graal, and also TruffleRuby > since January. Inlining works without any problems and independent of the > guest language, also in TruffleRuby. > > Regarding disadvantage 2 “Long startup and warmup times”: > The benefit of the system is absolutely *not* lost when the compiler > (Graal) and the Truffle guest language interpreter are AOT compiled. It > gives you decent startup and high peak. The interpreter is immediately > available in highly optimized machine code. Hot guest language methods are > dynamically compiled to machine code by the precompiled Graal. > > Regarding disadvantage 3 “Limited concurrency”: > There is no deeper reason why TruffleRuby is single threaded right now. > For sure none that has to do with the fundamentals of the Truffle approach. > We are planning to support 100% multi-threading also in TruffleRuby. One of > the explorations we are currently doing is to support guest language level > safepoints such that guest language developers themselves can easier deal > with concurrency without compromising any peak performance. > I plan to work on concurrency in Truffle during my PhD in Linz. Making Truffle thread-safe is a priority and some work is already done for AST replacements. I am also interested in supporting different concurrency primitives such as threads and fibers for the guest languages. Benoit Regarding disadvantage 4 “Limited availability”: > Yes, this is indeed a chicken and egg problem. Truffle is however not as > closely tied to Graal as you suggest here. I believe that it is fairly > straightforward to create a Truffle front-end for C2 (or any other compiler > supporting deoptimization). There are only 3k LOC in Graal that are > specific to Truffle. I think that they could be ported in a reasonable time > frame. The Truffle interpreters themselves run on any Java system even if > it has only very limited features - this is actually an advantage over a > pure bytecode generation approach. They can also be AOT compiled for > devices that do not support a full JVM and have strong footprint > requirements. This would of course be slower execution than in a full > fledged VM, but it would at least run correctly. I furthermore think that > it is possible to do the Truffle partial evaluation via bytecode generation > for backwards compatibility. > > Regarding disadvantage 5 "Unclear benefits for real-world applications”: > This kind of argument can hardly be countered before a system is 100% > finished and shipped. The term “real world” is also somewhat loosely > defined. I would very much support the development of a JRuby benchmark > suite that tries to reflect “real world” as close as possible. > There is absolutely no reason to believe that a Truffle-based Ruby > implementation would not have benefits for “real-world applications”. Or > that it would not be able to run a large application for a long time. It is > clear that the TruffleRuby prototype needs more completeness work both at > the language and the library level. We are very happy with the results we > got so far with Chris working for about a year. We are planning to increase > the number of people working on this, and would also be grateful for any > help we can get from the Ruby community. > > Regarding Graal: Did you ever try to benchmark JRuby without Truffle with > the latest Graal binaries available at > http://lafo.ssw.uni-linz.ac.at/builds/? We would be looking forward to > see the peak performance results on a couple of workloads. We are not > speculating about Graal becoming part of a particular OpenJDK release (as > experimental or regular option). This is the sovereign decision of the > OpenJDK community. All we can do is to demonstrate and inform about Graal’s > performance and stability. > > We recognise that there is a long road ahead. But in particular in this > context, I would like to emphasize that we are looking for more people to > support this effort for a new language implementation platform. I strongly > believe that Truffle is the best currently available vehicle to make Ruby > competitive in terms of performance with node.js. We are happy to try to > *prove* you wrong - even happier about support of any kind along the road > ;). I am also looking forward to continue this discussion at JavaOne (as > part of the TruffleRuby session or elsewhere). > > Regards, thomas > > On 30 Aug 2014, at 21:21, Charles Oliver Nutter <head...@headius.com> > wrote: > > > Removing all context, so it's clear this is just my opinions and > thoughts... > > > > As most of you know, we've opened up our codebase and incorporated the > > graciously-donated RubyTruffle directly into JRuby. It's available on > > JRuby master and we are planning to ship Truffle support with JRuby > > 9000, our next major version (due out in the next couple months). > > > > At the same time, we have been developing our own next-gen IR-based > > compiler, which will run unmodified on any JVM (with or without > > invokedynamic, though I still have to implement the "without" side). > > Why are we doing this when Truffle shows such promise? > > > > I'll try to enumerate the benefits and problems of Truffle here. > > > > * Benefits of using Truffle > > > > 1. Simpler implementation. > > > > From day 1, the most obvious benefit of Truffle is that you just have > > to write an AST interpreter. Anyone who has implemented a programming > > language can do this easily. This specific benefit doesn't help us > > implement JRuby, since we already have an AST interpreter, but it did > > make Chris Seaton's job easier building RubyTruffle initially. This > > also means a Truffle-based language is more approachable than one with > > a complicated compiler pipeline of its own. > > > > 2. Better communication with the JIT. > > > > Truffle, via Graal, has potential to pass much more information on to > > the JIT. Things like type shape, escaped references, frame access, > > type specialization, and so on can be communicated directly, rather > > than hoping and praying they'll be inferred by the shape of bytecodes. > > This is probably the largest benefit; much of my time optimizing JRuby > > has been spend trying to "trick" C2 into doing the right thing, since > > I don't have a direct way to communicate intent. > > > > The peak performance numbers for Truffle-based languages have been > > extremely impressive. If it's possible to get those numbers reasonably > > quickly and with predictable steady-state behavior in large, > > heterogeneous codebases, this is definitely the quickest path (on any > > runtime) to a high-performance language implementation. > > > > 3. OSS and pure Java > > > > Truffle and Graal are just OpenJDK projects under OpenJDK licenses, > > and anyone can build, hack, or distribute them. In addition, both > > Truffle and Graal are 100% Java, so for the first time a plain old > > Java developer can see (and manipulate) exactly how the JIT works > > without getting lost in a sea of plus plus. > > > > * Problems with Truffle > > > > I want to emphasize that regardless of its warts, we love Truffle and > > Graal and we see great potential here. But we need a dose of reality > > once in a while, too. > > > > 1. AST is not enough. > > > > In order to make that AST fly, you can't just implement a dumb generic > > interpreter. You need to know about (and generously annotate your AST > > for) many advanced compiler optimization techniques: > > > > A. Type specialization plus guarded fallbacks: Truffle will NOT > > specialize your code for you. You must provide every specialized path > > in your AST nodes as well as annotating "slow path", "transfer to > > interpreter", etc. > > > > B. Frame access and reification: In order to have cross-call access to > > frames or to squash frames created for multiple inlined calls, you > > must use Truffle's representation of a frame. This means loads/stores > > within your AST must be done against a Truffle object, not against an > > arbitrary object of your own creation. > > > > C. Method invocation and inlining: Up until fairly recently, if you > > wanted to inline methods you had to essentially build your own call > > site logic, profiling, deopt paths within your Truffle AST. When I did > > a little hacking on RubyTruffle around OSS time (December/January) it > > did *no* inlining of Ruby-to-Ruby calls. I hacked in inlining using > > existing classes and managed to get it to work, but I was doing all > > the plumbing myself. I know this has improved in the Truffle codebase > > since then, but I have my concerns about production readiness when the > > inlining call site parts of Truffle were just recently added and are > > still in flux. > > > > And there's plenty of other cases. Building a basic language for > > Truffle is pretty easy (I did a micro-language in about two hours at > > JVMLS last year), but building a high-performance language for Truffle > > still takes a fair investment of effort and working knowledge of > > dynamic compiler optimizations. > > > > 2. Long startup and warmup times. > > > > As Thomas pointed out in the other thread, because Truffle and Graal > > are normally run as plain Java libraries, they can actually aggravate > > startup time issues. Now, not only would all of JRuby have to warm up, > > but the eventual native code JIT has to warm up too. This is not > > surprising, really. It is possible to mitigate this by doing some form > > of AOT against Graal, but for every case I have seen the Truffle/Graal > > approach makes startup time much, much worse compared to just running > > atop JVM. > > > > Warmup time is also worsened significantly. > > > > The AST you create for Truffle must be heavily mutated while running > > in order to produce a specialized version of that AST. This must > > happen before the AST is eventually fed into Graal, which means you > > have a self-modifying interpreter spinning AST objects like mad while > > executing the early phases of your application. Compare to a dumb > > interpreter as in JRuby's old AST, where interpreting the AST produces > > no additional objects other than those necessary for execution of the > > code. > > > > The Truffle approach itself adds overhead too. Until optimized, the > > fully-reified frame objects, specialization markup (which triggers AST > > rewriting), deoptimization guards, and so on are all done manually > > against heap-level data structures. This is in addition to the > > JVM-level overhead of executing an AST (native frame-per-node, boxing > > and type-widening, poor inlining profile). > > > > Some amount of AOT *might* be applicable here, but the benefit of > > Truffle and Graal is lost in the AOT case if we're not getting > > real-world profile information. The Substrate VM has ben brought up to > > aid startup and warmup too...but that direction produces a > > closed-world executable based on optimizing all code up front...not > > exactly what we're looking for in a general-purpose language runtime. > > > > 3. Limited concurrency > > > > The RubyTruffle runtime currently has to execute code under the > > watchful eye of a global lock. Yes, you read that right...RubyTruffle > > is single-threaded right now. > > > > I would like to know if there's a deeper reason for this, but the > > obvious shallow reason is that you can't have multiple threads > > executing at the same time if they're making thread-unsafe mutations > > to the executing code. This is similar to the major stumbling block > > for e.g. Pypy, which rewrites currently-executing assembly > > instructions at deopt/reopt safe points. > > > > I believe once the code has transitioned to native, you can execute > > that safely across threads...but this is opaque to your Truffle-based > > language, and it's unclear how you'd manage re-acquiring some sort of > > lock when transferring back to the interpreter. > > > > The fact that concurrency has so far been hand-waved (or so it seems > > to me from the outside) scares the living hell out of me, especially > > when there's talk about rolling this stuff into Java 9. > > > > Obviously some of this could be mitigated with an immutable AST > > structure or other thread-friendly tree-transformation algorithm, but > > making the Truffle AST thread-safe may also make it even more > > object-heavy during interpretation, aggravating startup time further. > > > > 4. Limited availability > > > > This is the chicken-and-egg issue. Truffle is just a library, so we > > can ignore that for the moment (given any JVM, you can run a Truffle > > language). > > > > Graal is required for Truffle to perform well at all. The Truffle > > interpreter is without a doubt the slowest interpreter we've ever had > > for JRuby, and that's saying something (there could be startup/warmup > > effects in play here too). In order for us to go 100% Truffle, we'd > > need a Graal VM. That limits us to either pre-release or hand-made > > builds of Graal/OpenJDK. Even if Graal somehow did get into Java 9, > > we'd still have legions of users on 8, 7, ... even 6 in some cases, > > though we're probably leaving them behind with JRuby 9000. Ignoring > > other platforms (non-OpenJDK, Android) and assuming Graal in Java 9, > > I'd conservatively estimate JRuby could still not go 100% Truffle > > until 2017 or later. > > > > And it gets worse. Graal will probably never exist on other JVMs. > > Graal will probably never exist in an Android VM. Graal may not even > > be available in other non-Oracle OpenJDK derivatives for a very long > > time. We have users on dozens of different platform/JVM combinations, > > so there's really no practical way for us to abandon our JVM bytecode > > runtimes in the near future. > > > > Now of course if Graal became essential to users, it would be > > available in more places. We recognize the potential of Truffle and > > Graal, which is why we've been thrilled to work with Oracle on a > > RubyTruffle that's part of JRuby. We also recognize that the > > Truffle/Graal approach has some very compelling features for our > > users, and that our users may often be comfortable running custom > > JVMs. We're allowing all flowers to bloom and our users will pick the > > ones that work for them. > > > > 5. Unclear benefits for real-world applications > > > > There have been many published microbenchmarks for Truffle-based > > languages, but very few benchmarks of real-world applications > > performing significantly better than custom-made VMs (JS versus V8). > > There have been practically no studies of a Truffle-based language > > running a large application for a long period of time...and by long I > > mean server-scale. > > > > Chris Seaton has pushed this forward recently for Ruby, getting > > general-purpose, numeric-heavy libraries to run and optimize very well > > (a png library and a psd library). Going deeper requires having more > > of the language's standard libraries to be available, and I believe > > this is where Chris has spent much of his time (RubyTruffle currently > > requires mostly-custom versions of JRuby's core classes...versions > > that Truffle can recognize, specialize, and escape-analyze away). > > > > * Conclusion > > > > I again want to emphasize that we think Truffle and Graal are really > > awesome technology. I spent years with my nose smooshed against the > > glass, watching the Pypy guys add optimizations I wanted and make good > > on their promise of "just implement an interpreter...we'll do the > > rest". Finally we have what I wanted: a Pypy for JVM (in Truffle) and > > an LLVM for JVM (in Graal). These are exciting times indeed. > > > > But reality steps in. There's a long road ahead. > > > > I think we need to separate the questions about Truffle from questions > > about Graal. Truffle is ultimately just a library that uses Graal. > > > > Graal is promising JIT technology. Graal is simpler than C2 and may be > > able to match or beat its performance. Graal provides a better way to > > communicate intent to the JIT. These facts are not in question. > > > > However, Graal is not (other than when used as the JVM's JIT) a JVM. > > Targeting Graal directly acts against the promise of a standard, > > platform-and-VM-agnostic bytecode -- and that's the promise that > > brought most of us here. Graal is not yet ready to replace C2, which > > would mean adding to the size and complexity of Java 9. And Graal is > > almost completely untested in large production settings. > > > > I personally would love to see Graal get into a Java release soon as > > an experimental feature, but Java 9 seems ambitious but any standard. > > It *might* be possible/reasonable to include Graal as experimental in > > 9. Java 10 is certainly feasible for experimental, and may be feasible > > for product. But even if Graal got into mainstream OpenJDK and Java, > > there's a very long adoption tail ahead. > > > > I'd like to hear more from folks on the Graal and Truffle teams. Prove > > me wrong :-) > > > > - Charlie > > _______________________________________________ > > mlvm-dev mailing list > > mlvm-dev@openjdk.java.net > > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev > >
_______________________________________________ mlvm-dev mailing list mlvm-dev@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
