[
https://issues.apache.org/jira/browse/PARQUET-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Or Ozeri updated PARQUET-1704:
------------------------------
Description:
SerializedPageWriter currently allocates a one-time buffer for each page
encrypted.
We add a re-usable buffer to avoid the allocation and free overhead of these
buffers.
This optimization already exists when decrypting (in SerializedPageReader).
Testing this optimization we found that it improves running time performance
by ~25% (when encryption is on).
See attached screenshot of a profiler analysis, showing the overhead of
re-allocating and freeing the buffer.
was:
SerializedPageWriter currently allocates a one-time buffer for each page
encrypted.
We add a re-usable buffer to avoid the allocation and free overhead of these
buffers.
This optimization already exists when decrypting (in SerializedPageReader).
Testing this optimization we found that it improves running time performance by
~15% (when encryption is on).
See attached screenshot of a profiler analysis, showing the overhead of
re-allocating and freeing the buffer.
> [C++] Add re-usable encryption buffer to SerializedPageWriter
> -------------------------------------------------------------
>
> Key: PARQUET-1704
> URL: https://issues.apache.org/jira/browse/PARQUET-1704
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-cpp
> Reporter: Or Ozeri
> Priority: Major
> Attachments: profiler.png
>
>
> SerializedPageWriter currently allocates a one-time buffer for each page
> encrypted.
> We add a re-usable buffer to avoid the allocation and free overhead of these
> buffers.
> This optimization already exists when decrypting (in SerializedPageReader).
> Testing this optimization we found that it improves running time performance
> by ~25% (when encryption is on).
> See attached screenshot of a profiler analysis, showing the overhead of
> re-allocating and freeing the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
