[
https://issues.apache.org/jira/browse/KUDU-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adar Dembo updated KUDU-1489:
-----------------------------
Target Version/s: 1.0.0
Fix Version/s: (was: 1.0.0)
> Use WAL directory for tablet metadata files
> -------------------------------------------
>
> Key: KUDU-1489
> URL: https://issues.apache.org/jira/browse/KUDU-1489
> Project: Kudu
> Issue Type: Bug
> Components: fs, tserver
> Affects Versions: 0.9.0
> Reporter: Adar Dembo
> Priority: Critical
>
> Today a tserver will place tablet metadata files (i.e. superblock and cmeta
> files) in the first configured data directory. I don't remember why we
> decided to do this (commit 691f97d introduced it), but upon reconsideration
> the WAL directory seems like a much better choice, because if the machine has
> different kinds of I/O devices, the WAL directory's device is typically the
> fastest.
> Mostafa has been testing Impala and Kudu on a cluster with many thousands of
> tablets. His cluster contains storage-dense machines, each configured with 14
> spinning disks and one flash device. Naturally, the WAL directory sits on
> that flash device and the data directories are on the spinning disks. With
> thousands of tablet metadata files on the first spinning disk, nearly every
> tablet in the tserver is bottlenecked on that device due to the sheer amount
> of I/O needed to maintain the running state of the tablet, specifically
> rewriting cmeta files on various Raft events (votes, term advancement, etc.).
> Many thousands of tablets is not really a good scale for Kudu right now, but
> moving the tablet metadata files to a faster device should at least help with
> the above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
