ok, here is what I've got so far: Evaluate Janino: 882 micros. JDK: 1675 micros. Direct: 39 micros. DBody: 39 micros. Compile Janino: 1122 micros. JDK: 28131 micros. Direct: 4074 micros. DBody: 1457 micros.
please see the ClassBodyEvaluatorBuilder for 'DBody' implementation at https://github.com/immars/incubator-drill/commit/79f7cbc7cb8e7c0b116032bba902343a47e53aa4 On Tue, May 21, 2013 at 10:58 PM, Lisen Mu <[email protected]> wrote: > If testing compile time alone, my result is like this: > > Janino: 1077 micros. JDK: 23533 micros. Direct: 3587 micros. > > ExpressionEvaluator approach is faster than SimpleCompiler approach. I > think it's because ExpressionEvaluator parses only the expression string > and puts together the rest of the class with janino objects. Maybe it's > possible to get both better compile time and better evaluation time. I > would try this later some time. > > > > > > > On Tue, May 21, 2013 at 2:23 AM, Jacques Nadeau <[email protected]>wrote: > >> Yeah, my test was entirely focused on compile time. In the case of >> simple queries, the compile time differences would be important. In >> larger queries, this would be less of an issue. >> >> j >> >> >> On Mon, May 20, 2013 at 8:56 AM, Lisen Mu <[email protected]> wrote: >> > Jacques, >> > >> > I've tried Janino now, with your >> > org.apache.drill.exec.compile.TestClassCompilationTypes, >> > >> > One thing is that the compilation time dominates the benchmark. If we >> > compile once and benchmark evaluate() only, the performance of class >> > compiled by JDK compiler is comparable with the class compiled by >> Janino's >> > embed compiler. JDK compiler seems to invoke javac which is more >> > heavyweight than Janino. >> > >> > Another thing is that evalute() of ExpressionEvaluator by both Janino >> and >> > JDK implementation need to do reflect invocation once. I assume that in >> the >> > future utilization in drill, we would have several limited type of >> > evaluate() method signature; so by using SimpleCompiler to compile the >> > expression directly into a subclass of our Evaluator interface, we can >> > fully avoid reflection. >> > >> > Here is my benchmark result: >> > >> > Janino: 882 micros. JDK: 1641 micros. Direct: 41 micros. >> > >> > Please see the >> > >> https://github.com/immars/incubator-drill/commit/34c0c3db7287904dbfdef9751e41e252b264dc88 >> > >> > >> > >> > >> > On Sun, Apr 28, 2013 at 10:25 AM, Ted Dunning <[email protected]> >> wrote: >> > >> >> Is the actual execution time after warming up the JIT the same? >> >> >> >> >> >> On Fri, Apr 26, 2013 at 8:16 PM, Jacques Nadeau <[email protected]> >> >> wrote: >> >> >> >> > I'm working on some of the bytecode stuff for Drill and found some >> >> > interesting random testing data: >> >> > >> >> > Java6 adds a new JavaCompiler API. Janino supports utilizing that >> >> > instead of its internal parsing. The benefit of this is that all the >> >> > JDK 5 stuff like generics are fully supported. Since that seemed >> >> > nice, i figured we could use that. >> >> > >> >> > In initial testing, I've actually found it substantially slower than >> >> > the Janino compiler. I ran some crappy microbenchmarks just to see >> >> > why my stuff was going so slow. Running a basic comparison test of >> >> > the simple ExpressionEvaluator example utilizing Janino versus JDK >> >> > showed an average compile and single invocation time of ~18000 micros >> >> > on JDK versus ~950 micros on Janino. >> >> > >> >> > Has anyone else experimented with this/experienced this? >> >> > >> >> > Example block below (straight from Janino docs). >> >> > >> >> > ExpressionEvaluator ee = new ExpressionEvaluator("c > d ? c : >> d", >> >> > // expression >> >> > int.class, // expressionType >> >> > new String[] { "c", "d" }, // parameterNames >> >> > new Class[] { int.class, int.class } // parameterTypes >> >> > ); >> >> > >> >> > // Evaluate it with varying parameter values; very fast. >> >> > return (Integer) ee.evaluate(new Object[] { // parameterValues >> >> > new Integer(10), new Integer(11), }); >> >> > >> >> >> > >
