[
https://issues.apache.org/jira/browse/IMPALA-13166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18053732#comment-18053732
]
ASF subversion and git services commented on IMPALA-13166:
----------------------------------------------------------
Commit 62e08b8798f40c41508efb27381bdacad812af4c in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=62e08b879 ]
IMPALA-13166: Improve lookup of existing slot descriptors
Improves lookup for existing SlotDescriptors created by another Analyzer
by maintaining a map of Path to SlotDescriptor in TupleDescriptor. Only
adds entries that are also inserted into slotPathMap_.
Moves expensive lower-case slot path precondition to happen only when
adding new slot descriptors.
Speeds up PlannerTest#testManyExpressionPerformance by 1-2s (5-10%) in
local testing.
Change-Id: Id904ee8a641fb06ebca95b53ea29e78d120a2288
Reviewed-on: http://gerrit.cloudera.org:8080/21531
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Michael Smith <[email protected]>
> Queries with many nested views are still slow
> ---------------------------------------------
>
> Key: IMPALA-13166
> URL: https://issues.apache.org/jira/browse/IMPALA-13166
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 4.4.0
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Critical
> Fix For: Impala 5.0.0
>
>
> Further profiling on the query from IMPALA-12800 - and a larger version of
> the same - show significant time spent in {{Analyzer#registerSlotRef}}. The
> bulk of the work here is in checking whether a {{SlotDescriptor}} exists in
> the current {{TupleDescriptor}}, both because
> {{Path#getFullyQualifiedRawPath}} is expensive and because equals on the
> resulting list is expensive (especially repeated over every
> {{SlotDescriptor}} in the {{TupleDescriptor}}.
> There's room for improvement here:
> 1. avoid copying {{rawPath_}} every time we call {{getFullyQualifiedRawPath}}
> 2. use a map to identify whether a {{SlotDescriptor}} already exists at
> {{Path}} in the current {{TupleDescriptor}}
> 3 other small improvements may exist around memory allocation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
