[
https://issues.apache.org/jira/browse/TEZ-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063426#comment-17063426
]
Jonathan Turner Eagles commented on TEZ-4135:
---------------------------------------------
The size of this buffer can be controlled with configuration
io.compression.codec.snappy.buffersize which defaults to 256K. So could be
controlled globally easily. To be more extreme we could white box SnappyCodec
by modifying the settings before calling createInputStream in
IFile#readToMemory to get a perfect fit. Or were you thinking of wielding the
BlockDecompressorStream manually so that buffers could be reused?
> Improve memory allocation when executing in-memory reads
> --------------------------------------------------------
>
> Key: TEZ-4135
> URL: https://issues.apache.org/jira/browse/TEZ-4135
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Priority: Major
> Attachments: Screenshot 2020-03-16 at 6.34.42 PM.jpg
>
>
> !Screenshot 2020-03-16 at 6.34.42 PM.jpg|width=824,height=575!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
