[
https://issues.apache.org/jira/browse/IMPALA-14179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18007941#comment-18007941
]
ASF subversion and git services commented on IMPALA-14179:
----------------------------------------------------------
Commit 3e6326538cbd060ab4f2f4a7df9c9d2ac2cc6463 in impala's branch
refs/heads/master from Surya Hebbar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3e6326538 ]
IMPALA-13795: Support serving webUI content with gzip compression
This patch adds support for serving all the webUI content with gzip
content encoding.
For large JSONs and text profiles, Impala's webUI renderings maybe
hindered by the user's network bandwidth.
As the browser's native gzip decompression is very fast e.g. 300-400MB/s,
combining it with a faster compression level(i.e. gzip Z_BEST_SPEED) in
backend results in significant increases in speed i.e. faster load times.
During compression, instead of multiple reallocations, existing string
data is reinterpreted to reduce memory usage.
In case of failure during compression, the content is served in plain
format as before.
As currently, none of the memory allocation's are being tracked for the
rapidjson's generated documents(or any daemon webserver's served string),
it would be helpful to display the peak memory usage of a single buffer
used to serve all webUI content.
In the future, it is recommended to implement and use custom allocators
for all large served strings and rapidjson generated documents.
(See IMPALA-14178, IMPALA-14179)
Memory trackers within ExecEnv are now initialized before enabling
the webserver, allowing their use as parent memory trackers.
For now, the memory used by the compressed buffer, for each compressed
response is being tracked.
(i.e. through the "WebserverCompressedBuffer" MemTracker)
Example:
For Impala daemon, it is included in the execution environment's
process memory tracker and displayed on the /memz page as follows.
# After serving a general webpage like /memz
WebserverCompressedBuffer: Total=0 Peak=227.56 KB
# After serving a query profile text / JSON
WebserverCompressedBuffer: Total=0 Peak=4.09 MB
Tests:
* Added new tests to validate plain and gzipped content encoding headers
in test_web_pages.py - TestWebPage:test_content_encoding
in util/webserver-test.cc - Webserver::ContentEncodingHeadersTest
* The pre-existing tests validate the content
in test_web_pages.py, all tests request and validate gzipped content
in util/webserver-test.cc, all tests request and validate plain text
* Performance:
Approximate improvements for a TPC-DS 14 query ran locally with 3 nodes
with defaults
-> JSON profile : 4.53MB to 428.94KB
Without throttling / Raw local: 421ms to 421ms
Based on firefox's throttling(8 mbps): 8s to 2s
-> Text profile : 1.24MB to 219KB
Without throttling / Raw local: 281ms to 281ms
Based on firefox's throttling(8 mbps): 1.3s to 281ms
Change-Id: I431088a30337bbef2c8d6e16dd15fb6572db0f15
Reviewed-on: http://gerrit.cloudera.org:8080/22599
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Riza Suminto <[email protected]>
> Track memory usage of template generated webpages
> -------------------------------------------------
>
> Key: IMPALA-14179
> URL: https://issues.apache.org/jira/browse/IMPALA-14179
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Surya Hebbar
> Priority: Major
>
> The memory usage of large strings generated from profiles/large files through
> /raw_text.tmpl or other templates is not being tracked.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
