[jira] [Commented] (HIVE-10175) DynamicPartitionPruning lacks a fast-path exit for large IN() queries
[
https://issues.apache.org/jira/browse/HIVE-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718285#comment-14718285
]
Gopal V commented on HIVE-10175:
The issue is only visible in TPC-H q21, which hit the following error in the
nightly runs.
{code}
Caused by: java.lang.RuntimeException: Cannot find ExprNodeEvaluator for the
exprNodeDesc = RS[4]
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.get(ExprNodeEvaluatorFactory.java:57)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.(ExprNodeGenericFuncEvaluator.java:100)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.get(ExprNodeEvaluatorFactory.java:51)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.(ExprNodeGenericFuncEvaluator.java:100)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.get(ExprNodeEvaluatorFactory.java:51)
at
org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:59)
... 21 more
{code}
> DynamicPartitionPruning lacks a fast-path exit for large IN() queries
> -
>
> Key: HIVE-10175
> URL: https://issues.apache.org/jira/browse/HIVE-10175
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Tez
>Affects Versions: 1.2.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Minor
> Fix For: 1.3.0, 2.0.0
>
> Attachments: HIVE-10175.1.patch, HIVE-10175.profile.html
>
>
> TezCompiler::runDynamicPartitionPruning() & ppr.PartitionPruner() calls the
> graph walker even if all tables provided to the optimizer are unpartitioned
> (or temporary) tables.
> This makes it extremely slow as it will walk & inspect a large/complex
> FilterOperator later in the pipeline.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10175) DynamicPartitionPruning lacks a fast-path exit for large IN() queries
[ https://issues.apache.org/jira/browse/HIVE-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716263#comment-14716263 ] Jesus Camacho Rodriguez commented on HIVE-10175: +1 > DynamicPartitionPruning lacks a fast-path exit for large IN() queries > - > > Key: HIVE-10175 > URL: https://issues.apache.org/jira/browse/HIVE-10175 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer, Tez >Affects Versions: 1.2.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Attachments: HIVE-10175.1.patch, HIVE-10175.profile.html > > > TezCompiler::runDynamicPartitionPruning() & ppr.PartitionPruner() calls the > graph walker even if all tables provided to the optimizer are unpartitioned > (or temporary) tables. > This makes it extremely slow as it will walk & inspect a large/complex > FilterOperator later in the pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10175) DynamicPartitionPruning lacks a fast-path exit for large IN() queries
[ https://issues.apache.org/jira/browse/HIVE-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716123#comment-14716123 ] Gopal V commented on HIVE-10175: [~hagleitn]: Review please? > DynamicPartitionPruning lacks a fast-path exit for large IN() queries > - > > Key: HIVE-10175 > URL: https://issues.apache.org/jira/browse/HIVE-10175 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer, Tez >Affects Versions: 1.2.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Attachments: HIVE-10175.1.patch, HIVE-10175.profile.html > > > TezCompiler::runDynamicPartitionPruning() & ppr.PartitionPruner() calls the > graph walker even if all tables provided to the optimizer are unpartitioned > (or temporary) tables. > This makes it extremely slow as it will walk & inspect a large/complex > FilterOperator later in the pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
