[
https://issues.apache.org/jira/browse/IMPALA-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253755#comment-17253755
]
Tim Armstrong commented on IMPALA-9745:
---------------------------------------
I think this is a side-effect of the interaction between implicit casts and the
expr substitution and constant propagation - we determine that
* implicit_cast(cob as timestamp) = to_timestamp(cob, ...)
* to_timestamp(cob, ...) = implicit_cast('2018-06-07' as timestamp)
Therefore
* implicit_cast(cob as timestamp) = implicit_cast('2018-06-07' as timestamp)
This is valid so far, but it looks like the implicit cast gets dropped when
we're building the expression substitution map, and we use cob =
implicit_cast('2018-06-07' as timestamp), which is incorrect.
{noformat}
--- a/fe/src/main/java/org/apache/impala/analysis/Expr.java
+++ b/fe/src/main/java/org/apache/impala/analysis/Expr.java
@@ -1093,6 +1093,7 @@ abstract public class Expr extends TreeNode<Expr>
implements ParseNode, Cloneabl
public Expr substitute(ExprSubstitutionMap smap, Analyzer analyzer,
boolean preserveRootType) {
try {
+ LOG.debug("substitute: " + toSql() + "\n" + debugString() + "\n" +
smap.debugString());
return trySubstitute(smap, analyzer, preserveRootType);
} catch (Exception e) {
throw new IllegalStateException("Failed analysis after expr
substitution.", e);
{noformat}
{noformat}
I1222 12:16:59.188608 1786817 Expr.java:1096]
50462d2e4e26c3a0:ea88309200000000] substitute: cob = to_timestamp(cob,
'yyyy-MM-dd')
BinaryPredicate{op==, exprid=4 CastExpr{isImplicit=true, target=TIMESTAMP,
format=null, SlotRef{label=cob, path=cob, type=STRING, id=0}}
FunctionCallExpr{name=to_timestamp,
isStar=false, isDistinct=false, isIgnoreNulls=false, SlotRef{label=cob,
path=cob, type=STRING, id=0} StringLiteral{value=yyyy-MM-dd}}}
smap(default.test_replication.cob:TIMESTAMP '2018-06-07 00:00:00'
(SlotRef{label=default.test_replication.cob, path=cob, type=STRING, id=0}:))
I1222 12:16:59.189194 1786817 jni-util.cc:288]
50462d2e4e26c3a0:ea88309200000000] java.lang.IllegalStateException: Failed
analysis after expr substitution.
at org.apache.impala.analysis.Expr.substitute(Expr.java:1099)
at
org.apache.impala.analysis.ConstantPredicateHandler.propagateConstantPreds(ConstantPredicateHandler.java:115)
at org.apache.impala.analysis.Expr.propagateConstants(Expr.java:1285)
at org.apache.impala.analysis.Expr.optimizeConjuncts(Expr.java:1312)
at
org.apache.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1758)
at
org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:2067)
at
org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:940)
at
org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:768)
at
org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:276)
at
org.apache.impala.planner.SingleNodePlanner.createInlineViewPlan(SingleNodePlanner.java:1220)
at
org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:2078)
at
org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:940)
at
org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:768)
at
org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:276)
at
org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:169)
at
org.apache.impala.planner.Planner.createPlanFragments(Planner.java:118)
at org.apache.impala.planner.Planner.createPlans(Planner.java:245)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1507)
at
org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1838)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1695)
at
org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1588)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1558)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:159)
Caused by: org.apache.impala.common.AnalysisException: No matching function
with signature: to_timestamp(TIMESTAMP, STRING).
at
org.apache.impala.analysis.FunctionCallExpr.analyzeImpl(FunctionCallExpr.java:622)
at org.apache.impala.analysis.Expr.analyze(Expr.java:503)
at org.apache.impala.analysis.Expr.analyze(Expr.java:497)
at org.apache.impala.analysis.Expr.trySubstitute(Expr.java:1079)
at org.apache.impala.analysis.Expr.substitute(Expr.java:1097)
{noformat}
> SELECT from view fails with "AnalysisException: No matching function with
> signature: to_timestamp(TIMESTAMP, STRING)" after expression rewrite.
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-9745
> URL: https://issues.apache.org/jira/browse/IMPALA-9745
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 2.11.0, Impala 4.0
> Reporter: Andrew Sherman
> Priority: Critical
>
> Simple test case
> {code}
> drop view if exists test_replication_view;
> drop table if exists test_replication;
> create table test_replication(cob string);
> insert into test_replication values('2018-06-07');
> insert into test_replication values('2018-06-07');
> insert into test_replication values('2018-06-07');
> insert into test_replication values('2018-06-08');
> select * from test_replication;
> create view test_replication_view as select to_timestamp(cob, 'yyyy-MM-dd')
> cob_ts,cob trade_date from test_replication;
> select 1 from test_replication_view deal WHERE trade_date = deal.cob_ts AND
> deal.cob_ts = '2018-06-07';
> {code}
> The problem seems to be that after expression rewrite the type of cob has
> become a timestamp and so we look for the function "to_timestamp(TIMESTAMP,
> STRING)" instead of "to_timestamp(STRING, STRING)".
> A workaround is to run with
> {code}
> set enable_expr_rewrites=false;
> {code}
> For comparison a similar query runs OK in mysql
> {code}
> drop view if exists test_replication_view;
> drop table if exists test_replication;
> create table test_replication(cob varchar(255));
> insert into test_replication values('2018-06-07');
> insert into test_replication values('2018-06-07');
> insert into test_replication values('2018-06-07');
> insert into test_replication values('2018-06-08');
> select * from test_replication;
> create view test_replication_view as select str_to_date(cob, '%Y-%m-%d')
> cob_ts,cob trade_date from test_replication;
> select 1 from test_replication_view deal WHERE trade_date = deal.cob_ts AND
> deal.cob_ts = '2018-06-07'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
