-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11133/
-----------------------------------------------------------
Review request for hive, Jitendra Pandey, Eric Hanson, Sarvesh Sakalanaga, and
Remus Rusanu.
Description
-------
This patch adds Column Column, and Column Scalar vectorized execution tests.
These tests are generated in parallel with the vectorized expressions. The
tests focus is on validating the column vector and the vectorized row batch
metadata regarding nulls, repeating, and selection.
Overview of Changes:
CodeGen.java:
+ joinPath, getCamelCaseType, readFile and writeFile made static for use in
TestCodeGen.java.
+ filter types now specify null as their output type rather than "doesn't
matter" to make detection for test generation easier.
+ support for test generation added.
TestCodeGen.java & Templates:
TestClass.txt
TestColumnColumnFilterVectorExpressionEvaluation.txt,
TestColumnColumnOperationVectorExpressionEvaluation.txt,
TestColumnScalarFilterVectorExpressionEvaluation.txt,
TestColumnScalarOperationVectorExpressionEvaluation.txt
+This class is mutable and maintains a hashmap of TestSuiteClassName to test
cases. The tests cases are added over the course of vectorized expressions
class generation, with test classes being outputted at the end. For each column
vector (inputs and/or outputs) a matrix of pairwise covering Booleans is used
to generate test cases across nulls and repeating dimensions. Based on the
input column vector(s) nulls and repeating states the states of the output
column vector (if there is one) is validated, along with the null vector. For
filter operations the selection vector is validated against the generated data.
Each template corresponds to a class representing a test suite.
VectorizedRowGroupUtil.java
+added methods generateLongColumnVector and generateDoubleColumnVector for
generating the respective column vectors with optional nulls and/or repeating
values.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java
53d9a7a
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestClass.txt
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestCodeGen.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnColumnFilterVectorExpressionEvaluation.txt
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnColumnOperationVectorExpressionEvaluation.txt
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnScalarFilterVectorExpressionEvaluation.txt
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnScalarOperationVectorExpressionEvaluation.txt
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/TestColumnColumnFilterVectorExpressionEvaluation.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/TestColumnColumnOperationVectorExpressionEvaluation.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/TestColumnScalarFilterVectorExpressionEvaluation.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/TestColumnScalarOperationVectorExpressionEvaluation.java
PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/VectorizedRowGroupGenUtil.java
8a07567
Diff: https://reviews.apache.org/r/11133/diff/
Testing
-------
generated tests, and ran them.
Thanks,
tony murphy
