[
https://issues.apache.org/jira/browse/HIVE-20432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16594393#comment-16594393
]
Vineet Garg commented on HIVE-20432:
------------------------------------
I realized that for expressions such as {{between 1000 AND 2000}} earlier
patch would end up creating IN expression with 1000 ExprNodeConst children.
This could result in OOM therefore I have introduced a maximum threshold in
second patch to limit the rewrite. I have currently set it to 100 which is
arbitrary.
[~ashutoshc] Can you take a look at the patch? Review request is at
https://reviews.apache.org/r/68534/
> Rewrite BETWEEN to IN for integer types for stats estimation
> ------------------------------------------------------------
>
> Key: HIVE-20432
> URL: https://issues.apache.org/jira/browse/HIVE-20432
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 4.0.0
> Reporter: Vineet Garg
> Assignee: Vineet Garg
> Priority: Major
> Attachments: HIVE-20432.1.patch, HIVE-20432.2.patch
>
>
> IN clause provides better statistics estimation since predicates are equality
> based and it keeps into account selectivity of all values. BETWEEN is
> rewritten into {{>= AND <=}} i.e. non-equality predicates which cause worse
> estimation. For integer types BETWEEN could be rewritten into IN e.g. BETWEEN
> 4 and 7 is equivalent to IN (4,5,6,7).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
