[
https://issues.apache.org/jira/browse/IMPALA-13166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18053734#comment-18053734
]
ASF subversion and git services commented on IMPALA-13166:
----------------------------------------------------------
Commit dd942965474d36bbd48b51c4e351ba4a311f8752 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=dd9429654 ]
IMPALA-13166: Optimize ExprSubstitutionMap allocation
Allocates HashMap and Lists with known sizes to avoid resizing them
during initial construction. HashMap uses default capacity of 16 and
load factor of .75, and we increase initial capacity as needed.
Has no measurable impact on performance, but may help with garbage
collection.
Change-Id: If5e56401508f30e3066b1a3758d69d89a288adcd
Reviewed-on: http://gerrit.cloudera.org:8080/21533
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Queries with many nested views are still slow
> ---------------------------------------------
>
> Key: IMPALA-13166
> URL: https://issues.apache.org/jira/browse/IMPALA-13166
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 4.4.0
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Critical
> Fix For: Impala 5.0.0
>
>
> Further profiling on the query from IMPALA-12800 - and a larger version of
> the same - show significant time spent in {{Analyzer#registerSlotRef}}. The
> bulk of the work here is in checking whether a {{SlotDescriptor}} exists in
> the current {{TupleDescriptor}}, both because
> {{Path#getFullyQualifiedRawPath}} is expensive and because equals on the
> resulting list is expensive (especially repeated over every
> {{SlotDescriptor}} in the {{TupleDescriptor}}.
> There's room for improvement here:
> 1. avoid copying {{rawPath_}} every time we call {{getFullyQualifiedRawPath}}
> 2. use a map to identify whether a {{SlotDescriptor}} already exists at
> {{Path}} in the current {{TupleDescriptor}}
> 3 other small improvements may exist around memory allocation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
