[
https://issues.apache.org/jira/browse/PARQUET-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610269#comment-17610269
]
ASF GitHub Bot commented on PARQUET-2184:
-----------------------------------------
shangxinli commented on PR #993:
URL: https://github.com/apache/parquet-mr/pull/993#issuecomment-1260143659
I wonder how much benefit get can gain of this fix?
> Improve SnappyCompressor buffer expansion performance
> -----------------------------------------------------
>
> Key: PARQUET-2184
> URL: https://issues.apache.org/jira/browse/PARQUET-2184
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-mr
> Affects Versions: 1.13.0
> Reporter: Andrew Baranec
> Priority: Minor
>
> The existing implementation of SnappyCompressor will only allocate enough
> bytes for the buffer passed into setInput(). This leads to suboptimal
> performance when there are patterns of writes that cause repeated buffer
> expansions. In the worst case it must copy the entire buffer for every
> single invocation of setInput()
> Instead of allocating a buffer of size current + write length, there should
> be an expansion strategy that reduces the amount of copying required.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
