[
https://issues.apache.org/jira/browse/IMPALA-13886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17942146#comment-17942146
]
ASF subversion and git services commented on IMPALA-13886:
----------------------------------------------------------
Commit e7ea7fa6ef0d0f87a49ef691aecf3727e5d3d9c6 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e7ea7fa6e ]
IMPALA-13886: Fix order of recursion for complex types for tuple caching
Queries that use complex types fail with an IllegalStateException
when running with tuple caching. The issue is that the logic for
registering tuples is recursing in the wrong order. It processes
the parent before moving on to the child, but the parent references
tuple id's in the child and cannot be processed first. This
switches the ordering so that the children are processed first.
Testing:
- Added a sanity check for complex types in TupleCacheTest
Change-Id: I853c06e76422382c628850181e3c7e4fdf86efc2
Reviewed-on: http://gerrit.cloudera.org:8080/22667
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Tuple caching hits IllegalStateException for nested types
> ---------------------------------------------------------
>
> Key: IMPALA-13886
> URL: https://issues.apache.org/jira/browse/IMPALA-13886
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
>
> The tuple cache hits an assert when working on a table with nested types:
> {noformat}
> I0321 17:56:28.828924 244837 jni-util.cc:321]
> 25486520ccba2db5:6a28f6b600000000] java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:496)
> at
> org.apache.impala.planner.TupleCacheInfo.getLocalTupleId(TupleCacheInfo.java:405)
> at
> org.apache.impala.common.ThriftSerializationCtx.translateTupleId(ThriftSerializationCtx.java:113)
> at
> org.apache.impala.analysis.SlotDescriptor.toThrift(SlotDescriptor.java:489)
> at
> org.apache.impala.planner.TupleCacheInfo.registerTupleHelper(TupleCacheInfo.java:349)
> at
> org.apache.impala.planner.TupleCacheInfo.registerTuple(TupleCacheInfo.java:313)
> at
> org.apache.impala.common.ThriftSerializationCtx.registerTuple(ThriftSerializationCtx.java:79)
> at org.apache.impala.planner.PlanNode.initThrift(PlanNode.java:551)
> at
> org.apache.impala.planner.PlanNode.computeTupleCacheInfo(PlanNode.java:1386)
> at
> org.apache.impala.planner.PlanNode.computeTupleCacheInfo(PlanNode.java:1343)
> at
> org.apache.impala.planner.TupleCachePlanner.createPlans(TupleCachePlanner.java:55)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:325)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1975)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:3137)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2895)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2410)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2036)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:175){noformat}
> What's happening is that when we are registering the tuple descriptors, we
> are registering the top level tuple descriptor before registering the
> children. That is a problem, because the parent references in the tuple id of
> the child and expects it to already be registered. We should be registering
> all the children recursively before registering the parent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
