[
https://issues.apache.org/jira/browse/HIVE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-14866:
-------------------------------------------
Description:
Currently, we set up the global limit for the query in the SemanticAnalyzer. In
addition, we have an optimization rule GlobalLimitOptimizer that prunes the
input depending on the global limit and under certain conditions (off by
default).
We would like to remove the dependency on the SemanticAnalyzer and set the
global limit within GlobalLimitOptimizer.
Further, we need to solve the problem with SimpleFetchOptimizer, which only
checks the limit but does not take into account the offset of the query, which
I think might lead to incorrect results if FetchOptimizer kicks in (not
verified yet).
was:
Currently, we set up the global limit for the query in two different places
through two different variables: SemanticAnalyzer and through an optimization
rule GlobalLimitOptimizer (the latest is off by default).
This leads to several problems that I have observed:
- Some optimizations are not checking both variables, thus missing
opportunities.
- The variable set by SemanticAnalyzer does not take into account offset of the
query, which I think might lead to incorrect results if FetchOptimizer kicks in
(not verified yet). GlobalLimitOptimizer does take into account offset of query.
This issue is to set hive.limit.optimize.enable to _true_ by default, i.e., use
GlobalLimitOptimizer, and thus getting rid of the variable set by
SemanticAnalyzer. Maybe there are some gaps (cases covered by SemanticAnalyzer
alternative and not covered by GlobalLimitOptimizer) that we will need to work
on.
> Remove logic to set global limit from SemanticAnalyzer
> ------------------------------------------------------
>
> Key: HIVE-14866
> URL: https://issues.apache.org/jira/browse/HIVE-14866
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 2.1.0
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14866.patch
>
>
> Currently, we set up the global limit for the query in the SemanticAnalyzer.
> In addition, we have an optimization rule GlobalLimitOptimizer that prunes
> the input depending on the global limit and under certain conditions (off by
> default).
> We would like to remove the dependency on the SemanticAnalyzer and set the
> global limit within GlobalLimitOptimizer.
> Further, we need to solve the problem with SimpleFetchOptimizer, which only
> checks the limit but does not take into account the offset of the query,
> which I think might lead to incorrect results if FetchOptimizer kicks in (not
> verified yet).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
