Re: benefits of code gen

[Sumedh Wale]

The difference is closure invocation instead of a static java.lang.Math call. In many cases JIT may not be able to perform inlining and related code optimizations though in this specific case it should. This is highly dependent on the specific case, but when inlining cannot be done and it leads to a method call (especially virtual call) then the difference is quite large: few nanoseconds per evaluation vs tens of nanoseconds in my experiments. Serialization of an additional object as a reference can have a measurable effect for low-latency jobs though usually can be ignored. What has been observed is that if an _expression_ uses CodegenFallback then it becomes an order of magnitude slower or more. Most of it is due to UnsafeRow read/write overhead which is avoided here, but still care needs to be taken for (virtual) function calls too. In some cases JIT does inline virtual calls but may not always happen. In my experience the only reliable case where it does inline is when the virtual call is on a local variable that does not change for multiple invocations (e.g. a final local variable outside the while loop of a doProduce). I think what should work better is encapsulating such code in methods of a scala object rather than a class and those can be invoked in generated code like static methods. Such calls should be equivalent to inline code generation in most cases since JIT will inline the calls where it will determine significant benefit. In some cases such method calls will have better CPU instruction cache hits (i.e. if same inline code is emitted multiple times vs common method calls). All this needs thorough micro/macro-benchmarking. However, I don't recall any large pieces of generated code where this can help. Most complex pieces like in HashAggregateExec/SortMergeJoinExec/BroadcastHashJoinExec are so because they generate schema specific code (to avoid virtual calls and boxing/unboxing, and UnsafeRow read/write in some cases) which is significantly faster than the equivalent generic code in doExecute. Or in your "castToDateCode" example, don't see how you can reduce it since bulk of code is already in the static stringToDate call. On Saturday 11 February 2017 03:02 AM, Koert Kuipers wrote: based on that i take it that math functions would be primary beneficiaries since they work on primitives. so if i take UnaryMathExpression as an example, would i not get the same benefit if i change it to this? abstract class UnaryMathExpression(val f: Double => Double, name: String)   extends UnaryExpression with Serializable with ImplicitCastInputTypes {   override def inputTypes: Seq[AbstractDataType] = Seq(DoubleType)   override def dataType: DataType = DoubleType   override def nullable: Boolean = true   override def toString: String = s"$name($child)"   override def prettyName: String = name   protected override def nullSafeEval(input: Any): Any = {     f(input.asInstanceOf[Double])   }   // name of function in java.lang.Math   def funcName: String = name.toLowerCase   def function(d: Double): Double = f(d)   override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {     val self = ctx.addReferenceObj(name, this, getClass.getName)     defineCodeGen(ctx, ev, c => s"$self.function($c)")   } } admittedly in this case the benefit in terms of removing complex codegen is not there (the codegen was only one line), but if i can remove codegen here i could also remove it in lots of other places where it does get very unwieldy simply by replacing it with calls to methods. Function1 is specialized, so i think (or hope) that my version does no extra boxes/unboxing. On Fri, Feb 10, 2017 at 2:24 PM, Reynold Xin <r...@databricks.com> wrote: With complex types it doesn't work as well, but for primitive types the biggest benefit of whole stage codegen is that we don't even need to put the intermediate data into rows or columns anymore. They are just variables (stored in CPU registers). On Fri, Feb 10, 2017 at 8:22 PM, Koert Kuipers <ko...@tresata.com> wrote: so i have been looking for a while now at all the catalyst expressions, and all the relative complex codegen going on. so first off i get the benefit of codegen to turn a bunch of chained iterators transformations into a single codegen stage for spark. that makes sense to me, because it avoids a bunch of overhead. but what i am not so sure about is what the benefit is of converting the actual stuff that happens inside the iterator transformations into codegen. say if we have an _expression_ that has 2 children and creates a struct for them. why would this be faster in codegen by re-creating the code to do this in a string (which is complex and error prone) compared to simply have the codegen call the normal method for this in my class? i see so much trivial code be re-created in codegen. stuff like this:   private[this] def castToDateCode(       from: DataType,       ctx: CodegenContext): CastFunction = from match {     case StringType =>       val intOpt = ctx.freshName("intOpt")       (c, evPrim, evNull) => s"""         scala.Option<Integer> $intOpt =           org.apache.spark.sql.catalyst.util.DateTimeUtils.stringToDate($c);         if ($intOpt.isDefined()) {           $evPrim = ((Integer) $intOpt.get()).intValue();         } else {           $evNull = true;         }        """ is this really faster than simply calling an equivalent functions from the codegen, and keeping the codegen logic restricted to the "unrolling" of chained iterators? --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org