Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#10).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s
After change: 1.54s
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M
fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A
java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M
testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
15 files changed, 615 insertions(+), 171 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/10
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 10
Gerrit-Owner: Balazs Hevele <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
