[
https://issues.apache.org/jira/browse/IMPALA-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837465#comment-16837465
]
Fang-Yu Rao edited comment on IMPALA-8513 at 5/10/19 5:37 PM:
--------------------------------------------------------------
Taking a look at the error stack above (also available at
[https://master-02.jenkins.cloudera.com/job/impala-cdh6.x-core/781/testReport/org.apache.impala.planner/PlannerTest/testEmpty/]),
it seems that the deserialized result “val” (of type TResultRow) at
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L200]
is not as expected. It is expected to be a list of TColumnValue’s consisting
of exactly 1 element (this is also what I have seen when I attached a debugger
to impalad via port 30000), however, the message included in the error stack
shows that that list is of null value (colVals:null).
After comparing the variable exprBatch around
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L199],
I found that the contents of exprBatch provided above and the contents of
exprBatch I have observed on my dev box (via a debugger) are exactly the same.
On the other hand, I have also tried to observe the contents in the variable
result on my dev box after
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L199].
It is a byte array consisting of the following 14 elements: [15, 0, 1, 12, 0,
0, 0, 1, 2, 0, 1, 0, 0, 0].
The following are the two things I can think of that might go wrong.
(1) It may be possible that there is something wrong with the values in the
variable “result”, which are returned from the BE at
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L194].
This may require us to get more detailed information about the stack contents
when that java.lang.IllegalStateException happened.
(2) It may also be possible that there is something wrong with the function
call “deserializer.deserialize” at
“[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L199]“.
was (Author: fangyurao):
Taking a look at the error stack above (also available at
[https://master-02.jenkins.cloudera.com/job/impala-cdh6.x-core/781/testReport/org.apache.impala.planner/PlannerTest/testEmpty/]),
it seems that the deserialized result “val” (of type TResultRow) is not as
expected. It is expected to be a list of TColumnValue’s consisting of exactly 1
element (this is also what I have seen when I attached a debugger to impalad
via port 30000), however, the message included in the error stack shows that
that list is of null value (colVals:null).
After comparing the variable exprBatch around
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L199],
I found that the contents of exprBatch provided above and the contents of
exprBatch I have observed on my dev box (via a debugger) are exactly the same.
On the other hand, I have also tried to observe the contents in the variable
result on my dev box after
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L199].
It is a byte array consisting of the following 14 elements: [15, 0, 1, 12, 0,
0, 0, 1, 2, 0, 1, 0, 0, 0].
The following are the two things I can think of that might go wrong.
(1) It may be possible that there is something wrong with the values in the
variable “result”, which are returned from the BE at
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L194].
This may require us to get more detailed information about the stack contents
when that java.lang.IllegalStateException happened.
(2) It may also be possible that there is something wrong with the function
call “deserializer.deserialize” at
“[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/FeSupport.java#L199]“.
> PlannerTest.testEmpty fails with java.lang.IllegalStateException: Illegal
> expr eval result. Expr=ifnull(NULL, FALSE)
> --------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-8513
> URL: https://issues.apache.org/jira/browse/IMPALA-8513
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.3.0
> Reporter: Tim Armstrong
> Assignee: Fang-Yu Rao
> Priority: Blocker
> Labels: broken-build
>
> This failed with head commit "IMPALA-8341: Data cache for remote reads". It
> appears to be intermittent. It's very strange - I looked at the code and it
> looks "impossible" to me, but clearly it isn't. It could be related to recent
> code changes in this area, but I don't see a problem after having taken a
> second look. [~fangyurao] maybe you have some ideas?
> {noformat}
> org.apache.impala.planner.PlannerTest.testEmpty
> Failing for the past 1 build (Since Failed#781 )
> Took 5.5 sec.
> add description
> Error Message
> Query:
> select t1.int_col, count(*)
> from functional.alltypestiny t1
> left outer join functional.alltypes t2
> on t1.id = t2.id
> group by t1.int_col
> having ifnull(null, false)
> Error Stack:
> java.lang.IllegalStateException: Illegal expr eval result. Expr=ifnull(NULL,
> FALSE)
> TExpBatch=TExprBatch(exprs:[TExpr(nodes:[TExprNode(node_type:FUNCTION_CALL,
> type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]), num_children:2, is_constant:true,
> fn:TFunction(name:TFunctionName(db_name:_impala_builtins,
> function_name:ifnull), binary_type:BUILTIN,
> arg_types:[TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]),
> TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))])],
> ret_type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]), has_var_args:false,
> signature:ifnull(BOOLEAN, BOOLEAN), scalar_fn:TScalarFunction(symbol:),
> is_persistent:true, last_modified_time:-1)),
> TExprNode(node_type:NULL_LITERAL,
> type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]), num_children:0, is_constant:true),
> TExprNode(node_type:BOOL_LITERAL,
> type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]), num_children:0, is_constant:true,
> bool_literal:TBoolLiteral(value:false))])])
> Result=TResultRow(colVals:null)
> at
> org.apache.impala.service.FeSupport.EvalExprWithoutRowBounded(FeSupport.java:201)
> at org.apache.impala.service.FeSupport.EvalExprWithoutRow(FeSupport.java:183)
> at org.apache.impala.service.FeSupport.EvalPredicate(FeSupport.java:264)
> at
> org.apache.impala.analysis.Analyzer.markConstantConjunct(Analyzer.java:1089)
> at org.apache.impala.analysis.Analyzer.registerConjuncts(Analyzer.java:1060)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.buildResultExprs(SelectStmt.java:805)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:227)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:194)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:187)
> at
> org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:450)
> at
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:420)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1270)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1237)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1207)
> at
> org.apache.impala.planner.PlannerTestBase.testPlan(PlannerTestBase.java:516)
> at
> org.apache.impala.planner.PlannerTestBase.runTestCase(PlannerTestBase.java:421)
> at
> org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:887)
> at
> org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:852)
> at org.apache.impala.planner.PlannerTest.testEmpty(PlannerTest.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> Stacktrace
> java.lang.AssertionError:
> Query:
> select t1.int_col, count(*)
> from functional.alltypestiny t1
> left outer join functional.alltypes t2
> on t1.id = t2.id
> group by t1.int_col
> having ifnull(null, false)
> Error Stack:
> java.lang.IllegalStateException: Illegal expr eval result. Expr=ifnull(NULL,
> FALSE)
> TExpBatch=TExprBatch(exprs:[TExpr(nodes:[TExprNode(node_type:FUNCTION_CALL,
> type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]), num_children:2, is_constant:true,
> fn:TFunction(name:TFunctionName(db_name:_impala_builtins,
> function_name:ifnull), binary_type:BUILTIN,
> arg_types:[TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]),
> TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))])],
> ret_type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]), has_var_args:false,
> signature:ifnull(BOOLEAN, BOOLEAN), scalar_fn:TScalarFunction(symbol:),
> is_persistent:true, last_modified_time:-1)),
> TExprNode(node_type:NULL_LITERAL,
> type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]), num_children:0, is_constant:true),
> TExprNode(node_type:BOOL_LITERAL,
> type:TColumnType(types:[TTypeNode(type:SCALAR,
> scalar_type:TScalarType(type:BOOLEAN))]), num_children:0, is_constant:true,
> bool_literal:TBoolLiteral(value:false))])])
> Result=TResultRow(colVals:null)
> at
> org.apache.impala.service.FeSupport.EvalExprWithoutRowBounded(FeSupport.java:201)
> at
> org.apache.impala.service.FeSupport.EvalExprWithoutRow(FeSupport.java:183)
> at org.apache.impala.service.FeSupport.EvalPredicate(FeSupport.java:264)
> at
> org.apache.impala.analysis.Analyzer.markConstantConjunct(Analyzer.java:1089)
> at
> org.apache.impala.analysis.Analyzer.registerConjuncts(Analyzer.java:1060)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.buildResultExprs(SelectStmt.java:805)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:227)
> at
> org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:194)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:187)
> at
> org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:450)
> at
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:420)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1270)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1237)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1207)
> at
> org.apache.impala.planner.PlannerTestBase.testPlan(PlannerTestBase.java:516)
> at
> org.apache.impala.planner.PlannerTestBase.runTestCase(PlannerTestBase.java:421)
> at
> org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:887)
> at
> org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:852)
> at org.apache.impala.planner.PlannerTest.testEmpty(PlannerTest.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:908)
> at
> org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:852)
> at org.apache.impala.planner.PlannerTest.testEmpty(PlannerTest.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> {noformat}
> I'm going to try to loop the query locally to see if I can reproduce this:
> {noformat}
> while impala-shell.sh -q 'explain select t1.int_col, count(*) from
> functional.alltypestiny t1 left outer join functional.alltypes t2 on t1.id =
> t2.id group by t1.int_col having ifnull(null, false);' ; do date; done
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
