bharath v created IMPALA-7934:
---------------------------------
Summary: Switch to using Java 8's Base64 impl for incremental
stats encoding
Key: IMPALA-7934
URL: https://issues.apache.org/jira/browse/IMPALA-7934
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 3.1.0
Reporter: bharath v
Attachments: base64.png
Incremental stats are compressed and Base64 encoded before they are chunked and
written to the HMS' partition parameters map. When they are read back, we need
to Base64 decode and decompress.
For certain incremental stats heavy tables, we noticed that a significant
amount of time is spent in these base64 classes (see the attached image for the
stack. Unfortunately, I don't have the text version of it).
Java 8 comes with its own Base64 implementation and that has shown much better
perf results [1] compared to apache codec's impl. So consider switching to Java
8's base64 impl.
[1] http://java-performance.info/base64-encoding-and-decoding-performance/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
