shuifeng lu created DRILL-6763:
----------------------------------
Summary: Codegen optimiztion of SQL functions with constant values
Key: DRILL-6763
URL: https://issues.apache.org/jira/browse/DRILL-6763
Project: Apache Drill
Issue Type: Improvement
Components: Execution - Codegen
Affects Versions: 1.14.0
Reporter: shuifeng lu
Assignee: shuifeng lu
Attachments: Query1.java, Query2.java, code_compare.png,
compilation_time.png
Codegen class compilation takes tens to hundreds of milliseconds, a class cache
is hit when generifiedCode of code generator is exactly the same.
It works fine when UDF only takes columns or symbols, but not efficient when
one or more parameters in UDF is always distinct from the other.
Take face recognition for example, the face images are almost distinct from
each other according to lighting, facial expressions and details.
It is important to reduce redundant class compilation especially for those low
latency queries.
Cache miss rate and metaspace gc can also be reduced by eliminating the
redundant classes.
Here is the query to get the persons whose last name is Brunner and hire from
1st Jan 1990:
SELECT full_name, hire_date FROM cp.`employee.json` where last_name =
'Brunner' and hire_date >= '1990-01-01 00:00:00.0';
Now get the persons whose last name is Bernard and hire from 1st Jan 1990.
SELECT full_name, hire_date FROM cp.`employee.json` where last_name =
'Bernard' and hire_date >= '1990-01-01 00:00:00.0';
Figure !compilation_time.png! shows the compilation time of the generated code
by the above query in FilterRecordBatch on my laptop
Figure !code_compare.png! shows the only difference of the generated code
from the attachments is the last_name value at line 156.
It is straightforward that the redundant class compilation can be eliminated
by making the string12 as a member of the class and set the value when the
instance is created
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
