[
https://issues.apache.org/jira/browse/HBASE-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000559#comment-13000559
]
ryan rawson commented on HBASE-3514:
------------------------------------
a few comments here:
- why do you use a 256MB buffer in the "direct write" path? That is a pretty
damn big buffer, the default buffer is 32k. I'm not sure we should have such a
huge buffer (and ram sink).
- finishBlock() timings for writes dont measure anything useful anymore, the
long now = ... is too low, it needs to go up higher.
I'm poking at the unit tests, I'm still not sure why we got that negative
number, no unit tests sets the block size to 1GB as far as i know!
> Speedup HFile.Writer append
> ---------------------------
>
> Key: HBASE-3514
> URL: https://issues.apache.org/jira/browse/HBASE-3514
> Project: HBase
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.90.0
> Reporter: Matteo Bertozzi
> Priority: Minor
> Attachments: HBASE-3514-append-0.90-v2.patch,
> HBASE-3514-append-0.90-v3.patch, HBASE-3514-append-0.90.patch,
> HBASE-3514-append-trunk-v2.patch, HBASE-3514-append-trunk-v3.patch,
> HBASE-3514-append.patch, HBASE-3514-metaBlock-bsearch.patch
>
>
> Remove double writes when block cache is specified, by using, only, the
> ByteArrayDataStream.
> baos is flushed with the compress stream on finishBlock.
> On my machines HFilePerformanceEvaluation SequentialWriteBenchmark passes
> from 4000ms to 2500ms.
> Running SequentialWriteBenchmark for 1000000 rows took 4247ms.
> Running SequentialWriteBenchmark for 1000000 rows took 4512ms.
> Running SequentialWriteBenchmark for 1000000 rows took 4498ms.
> Running SequentialWriteBenchmark for 1000000 rows took 2697ms.
> Running SequentialWriteBenchmark for 1000000 rows took 2770ms.
> Running SequentialWriteBenchmark for 1000000 rows took 2721ms.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
