[
https://issues.apache.org/jira/browse/DRILL-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349766#comment-14349766
]
Jason Altekruse commented on DRILL-2395:
----------------------------------------
There was another discussion on the same review about the impact of now using
reflection to pass inputs and outputs between expressions. This could also be
included with this optimization work. I have copied the comments for this
suggestion here as well, we can decide if we want to break this into separate
JIRAs when we need to look at improving the interpreter.
Jinfeng:
I assume that we are aware that there is performance overhead to use reflection
here. So, in case if we want to use interpreter to evaluate many batches, there
would be performance impact. On the other hand, we may argue that using
interpreter already means performance is not as good as the current run-time
compiling + evaluation model, and therefore, adding additional performance
overhead does not change a lot.
Jason:
While there are no instances of evaluating a full batch of records, much less a
series of batches in the current code with the interpreter, I am not too
concerned about this. If we need a little more perforamnce we could modify the
DrillFuncHolderExpr class to hold hard references to the Fields for the inputs
extracted out of the DrillFuncCalsses with reflection once during a setup step.
We would still be using some reflection to set the values, but we can remove
the overhead of finding the fields by name each time. Overall if we need
performance for lots of repeatitive expression evaluations, we already have a
mechanism for it. I will admit we could probably improve the situation for a
medium number of evaluations (maybe in the 10,000 range) with these
improvements.
> Improve interpreted expression evaluation to only call the setup method once
> per batch
> --------------------------------------------------------------------------------------
>
> Key: DRILL-2395
> URL: https://issues.apache.org/jira/browse/DRILL-2395
> Project: Apache Drill
> Issue Type: Improvement
> Components: Functions - Drill
> Reporter: Jason Altekruse
> Assignee: Daniel Barclay (Drill)
> Priority: Minor
>
> This enhancement request came out of the review for a patch for DRILL-2060.
> Copied below is the discussion from there:
> Jinfeng:
> Do you have a plan to move setup() call into places such that setup() will be
> called once for each VectorAccessible input?
> In the code compile + evaluation model, doSetup() will be called per batch,
> in stead of per row.
> Jason:
> I have started working on a fix for this. Its a little complicated with
> setting constant inputs before the setup method is called. I'm trying to
> figure out the best way to share code with the rest of the input passing used
> in the EvalVisitor. Would you be okay with this being opened as an
> enhancement request to be merged a little later? Considering the current use
> of the interpreter this won't have an impact on any actual queries today.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
