[
https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhiqiu Kong updated HIVE-2249:
------------------------------
Attachment: HIVE-2249.2.patch.txt
This patch adds support to smartly infer the constant's type when encountering
the query like "column CMP constant", where CMP could be any of the comparators
supported by Hive. This aims to improving the performance by moving the type
conversion from runtime stage to compiling stage.
To be more detailed, the smart type inference will happen when the type of the
column is on e of the following:
* TINYINT
* SMALLINT
* INT
* BIGINT
* FLOAT
* DOUBLE
If the type of the columns fits any of the above, the constant on the other
hand side will be converted firstly to the column's type. When failing, the
constant will then be converted to DOUBLE. If both tries fail, the constant
will be left as what type it is.
One exception is when the column is STRING while the constant is BIGINT. In
this case, we do nothing. Otherwise, the constant will be converted to DOUBLE.
Other improvements include returning false immediately for the query like
"int_col = not_convertable_double_constant", such as "uid = 1.5".
NOTE:
~130 unit test cases need to be updated due to this diff. All updates are
limited to convert to the plan like "col = 10" to "col = 10.0", and are
carefully checked individually.
TWO test cases failed during the unit testing:
* testCliDriver_insert2_overwrite_partitions
* testCliDriver_ppr_pushdown
When looking into the query as well as the output, the plans generated were
found to be the same while the query results changed. As the queries in these
two cases are simple select queries, maybe the default sorting criteria was
changed unintentionally by this diff or other diffs.
> When creating constant expression for numbers, try to infer type from another
> comparison operand, instead of trying to use integer first, and then long and
> double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN>
> = 0", we always have to do a type conversion when comparing, which is
> unnecessary if it is slightly smarter to choose type when creating the
> constant expression. We can simply walk one level up the tree, find another
> comparison party and use the same type with that one if it is possible. For
> user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
