[
https://issues.apache.org/jira/browse/IMPALA-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114338#comment-17114338
]
ASF subversion and git services commented on IMPALA-9381:
---------------------------------------------------------
Commit a89b14ddec14cb05831f63cd26c862f536b08e8f in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a89b14d ]
IMPALA-9751: increase query log size to 100
This is both a flaky test fix and a change to the default.
I believe the test was flaky because queries had been
aged out of the log before the exec summary was fetched.
We have also found in production that queries often
age out of the log quickly, so the default size can
be too small.
IMPALA-9381 reduces the size of the query log entries,
so hopefully means that the total query log size won't
be significantly larger.
Change-Id: I70a1f8195574287f5df048c13f4a79acd88a81b3
Reviewed-on: http://gerrit.cloudera.org:8080/15972
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Lazily convert and/or cache different representations of the query profile
> --------------------------------------------------------------------------
>
> Key: IMPALA-9381
> URL: https://issues.apache.org/jira/browse/IMPALA-9381
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Fix For: Impala 3.4.0
>
>
> There are some obvious inefficiencies with how the query state record works:
> * We do an unnecessary copy of the archive string when adding it to the query
> log
> https://github.com/apache/impala/blob/79aae231443a305ce8503dbc7b4335e8ae3f3946/be/src/service/impala-server.cc#L1812.
> * We eagerly convert the profile to text and JSON, when in many cases they
> won't be needed -
> https://github.com/apache/impala/blob/79aae231443a305ce8503dbc7b4335e8ae3f3946/be/src/service/impala-server.cc#L1839
> . I think it is generally rare for more than one profile format to be
> downloaded from the web UI. I know of tools that scrape the thrift profile,
> but the human-readable version would usually only be consumed by humans. We
> could avoid this by only storing the thrift representation of the profile,
> then reconstituting the other representations from thrift if requested.
> * After ComputeExecSummary(), the profile shouldn't change, but we'll
> regenerate the thrift representation for every web request to get the
> encoded. This may waste a lot of CPU for tools scraping the profiles.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
