Daniel Becker has uploaded a new patch set (#17). (
http://gerrit.cloudera.org:8080/18526 )
Change subject: IMPALA-10851: Codegen for structs
......................................................................
IMPALA-10851: Codegen for structs
IMPALA-9495 added support for struct types in SELECT lists but only with
codegen turned off. This commit implements codegen for struct types.
To facilitate this, code generation for reading and writing 'AnyVal's
has been refactored. A new class, 'CodegenAnyValReadWriteInfo' is
introduced. This class is an interface between sources and destinations,
one of which is an 'AnyVal' object: sources generate an instance of this
class and destinations take that instance and use it to write the value.
The other side can for example be tuples from which we read (in the case
of 'SlotRef') or tuples we write into (in case of materialisation, see
Tuple::CodegenMaterializeExprs()). The main advantage is that sources do
not have to know how to write their destinations, only how to read the
values (and vice versa).
Before this change, many tests that involve structs ran only with
codegen turned off. Now that codegen is supported in these cases, these
tests are also run with codegen on.
Testing:
- enabed tests for structs in the select list with codegen on in
tests/query_test/test_nested_types.py
- enabled codegen in other tests where it used to be disabled because
it was not supported.
Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
---
M be/src/codegen/CMakeLists.txt
A be/src/codegen/codegen-anyval-read-write-info.cc
A be/src/codegen/codegen-anyval-read-write-info.h
M be/src/codegen/codegen-anyval.cc
M be/src/codegen/codegen-anyval.h
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/impala-ir.cc
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/exec/aggregator.cc
M be/src/exec/filter-context.cc
M be/src/exec/hash-table.cc
M be/src/exec/hash-table.h
M be/src/exprs/CMakeLists.txt
M be/src/exprs/hive-udf-call-ir.cc
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M be/src/exprs/kudu-partition-expr.cc
A be/src/exprs/scalar-expr-evaluator-ir.cc
M be/src/exprs/scalar-expr-evaluator.h
M be/src/exprs/slot-ref.cc
M be/src/exprs/slot-ref.h
M be/src/runtime/descriptors.cc
M be/src/runtime/descriptors.h
M be/src/runtime/fragment-state.cc
M be/src/runtime/fragment-state.h
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/tuple.cc
M be/src/udf/udf-internal.h
M be/src/udf/udf-ir.cc
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
M testdata/datasets/functional/functional_schema_template.sql
M
testdata/workloads/functional-query/queries/QueryTest/nested-struct-in-select-list.test
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_complex_types.test
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_struct_in_select_list.test
M tests/authorization/test_ranger.py
M tests/hs2/test_fetch_first.py
M tests/query_test/test_nested_types.py
40 files changed, 2,220 insertions(+), 833 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/18526/17
--
To view, visit http://gerrit.cloudera.org:8080/18526
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5272c3f095fd9f07877104ee03c8e43d0c4ec0b6
Gerrit-Change-Number: 18526
Gerrit-PatchSet: 17
Gerrit-Owner: Daniel Becker <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Daniel Becker <[email protected]>
Gerrit-Reviewer: Gabor Kaszab <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
