[
https://issues.apache.org/jira/browse/HBASE-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427615#comment-13427615
]
Jean-Daniel Cryans commented on HBASE-6497:
-------------------------------------------
bq. Is there a need to keep the logs small (typically 64-128 depending on the
HDFS config)?
bq. If current is 128 MB x 32 = 4096 MB (4 GB) of logs approx. before full
flush, then lets change that to have fewer than 32 files (reduces NN RPCs
during recovery and increases the sequential read length) on to 8 maxlogs at
512 MB default size (8x512 = 4096 again).
Issues with bigger files while having less of them:
- Less parallelization during distributed splitting since the unit of
distribution is a file.
- Less opportunities to get rid of logs without having to force flush regions.
The worst case would be having max 1 file meaning that when you roll you need
to force flush everything that hasn't been flushed yet.
> Revisit HLog sizing and roll parameters
> ---------------------------------------
>
> Key: HBASE-6497
> URL: https://issues.apache.org/jira/browse/HBASE-6497
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Lars George
>
> The last major update to the HLog sizing and roll features were done in
> HBASE-1394. I am proposing to revisit these settings to overcome recent
> issues where the HLog becomes a major bottleneck.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
