[
https://issues.apache.org/jira/browse/ACCUMULO-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840634#comment-13840634
]
Keith Turner commented on ACCUMULO-1534:
----------------------------------------
I ran some test adjusting tfile.fs.input.buffer.size and it seems like 32K is a
good setting. I generated an Rfile with 5 million entries using TestIngest
The table below shows times for scanning the RFile directly with different
buffer sizes. The times are averages of running the test many times.
||buf size||time in ms||
|1k|5020|
|2k|4681|
|4k|4463|
|8k|4249|
|16k|4160|
|32k|4073|
|64k|4052|
|128k|4117|
|256k|4179|
I also tried randomly seeking within the file with different buffer sizes.
||buffer size||avg seek time in ms||
|1k|4.2|
|32k|3.8|
|256k|4.1|
I also tried scanning through Accumulo w/ 32k and 256k buffers and did not
really see a difference.
> Tablet Server using large number of decompressors during a scan
> ---------------------------------------------------------------
>
> Key: ACCUMULO-1534
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1534
> Project: Accumulo
> Issue Type: Bug
> Affects Versions: 1.4.3
> Reporter: Mike Drob
> Assignee: Keith Turner
> Priority: Critical
> Fix For: 1.5.1, 1.6.0
>
>
> I believe this issue is similar to ACCUMULO-665. We've run into a situation
> where a complex iterator tree creates a large number of decompressors from
> the underlying CodecPool for serving scans. Each decompressor holds on to a
> large buffer and the total volume ends up killing the tserver.
> We have verified that turning off compression makes this problem go away.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
