[
https://issues.apache.org/jira/browse/HBASE-28485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17834406#comment-17834406
]
Charles Connell edited comment on HBASE-28485 at 4/5/24 5:59 PM:
-----------------------------------------------------------------
>From the [zstd
>manual|https://facebook.github.io/zstd/zstd_manual.html#Chapter4]:
{quote}When compressing many times, it is recommended to allocate a context
just once, and re-use it for each successive compression operation. This will
make workload friendlier for system's memory. Note : re-using context is just a
speed / resource optimization. It doesn't change the compression ratio, which
remains identical. Note 2 : In multi-threaded environments, use one different
context per thread for parallel execution.
{quote}
So this should be a safe change, and not change semantics.
was (Author: charlesconnell):
>From the [zstd
>manual|https://facebook.github.io/zstd/zstd_manual.html#Chapter4]:
> When compressing many times, it is recommended to allocate a context just
> once, and re-use it for each successive compression operation. This will make
> workload friendlier for system's memory. Note : re-using context is just a
> speed / resource optimization. It doesn't change the compression ratio, which
> remains identical. Note 2 : In multi-threaded environments, use one different
> context per thread for parallel execution.
So this should be a safe change, and not change semantics.
> Re-use ZstdDecompressCtx/ZstdCompressCtx for performance
> --------------------------------------------------------
>
> Key: HBASE-28485
> URL: https://issues.apache.org/jira/browse/HBASE-28485
> Project: HBase
> Issue Type: Improvement
> Reporter: Charles Connell
> Assignee: Charles Connell
> Priority: Major
> Labels: pull-request-available
> Attachments: async-prof-flamegraph-cpu_event-1712150670836-cpu.html,
> async-prof-pid-1324144-cpu-1.html
>
>
> The zstd documentation recommends re-using context objects when possible,
> because their creation has some expense. They can be more cheaply reset than
> re-created. In {{ZstdDecompressor}} and {{ZstdCompressor}}, we create a new
> context object for every call to {{decompress()}} and {{compress()}}. In CPU
> profiles I've taken at my company, the constructor of {{ZstdDecompressCtx}}
> can sometimes represent 10-25% of the time spent in zstd decompression, which
> itself is 5-10% of a RegionServer's total CPU time. Avoiding this performance
> penalty won't lead to any massive performance boost, but is a nice little win.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
