[
https://issues.apache.org/jira/browse/IMPALA-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-9381.
-----------------------------------
Fix Version/s: Impala 3.4.0
Resolution: Fixed
> Lazily convert and/or cache different representations of the query profile
> --------------------------------------------------------------------------
>
> Key: IMPALA-9381
> URL: https://issues.apache.org/jira/browse/IMPALA-9381
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Fix For: Impala 3.4.0
>
>
> There are some obvious inefficiencies with how the query state record works:
> * We do an unnecessary copy of the archive string when adding it to the query
> log
> https://github.com/apache/impala/blob/79aae231443a305ce8503dbc7b4335e8ae3f3946/be/src/service/impala-server.cc#L1812.
> * We eagerly convert the profile to text and JSON, when in many cases they
> won't be needed -
> https://github.com/apache/impala/blob/79aae231443a305ce8503dbc7b4335e8ae3f3946/be/src/service/impala-server.cc#L1839
> . I think it is generally rare for more than one profile format to be
> downloaded from the web UI. I know of tools that scrape the thrift profile,
> but the human-readable version would usually only be consumed by humans. We
> could avoid this by only storing the thrift representation of the profile,
> then reconstituting the other representations from thrift if requested.
> * After ComputeExecSummary(), the profile shouldn't change, but we'll
> regenerate the thrift representation for every web request to get the
> encoded. This may waste a lot of CPU for tools scraping the profiles.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
