[
https://issues.apache.org/jira/browse/KUDU-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wong updated KUDU-1489:
------------------------------
Resolution: Fixed
Fix Version/s: 1.7.0
Status: Resolved (was: In Review)
Resolved as 64926335fe263b43f1493c03a91ea999759dfc71
> Use WAL directory for tablet metadata files
> -------------------------------------------
>
> Key: KUDU-1489
> URL: https://issues.apache.org/jira/browse/KUDU-1489
> Project: Kudu
> Issue Type: Improvement
> Components: consensus, fs, tserver
> Affects Versions: 0.9.0
> Reporter: Adar Dembo
> Assignee: Andrew Wong
> Priority: Major
> Fix For: 1.7.0
>
>
> Today a tserver will place tablet metadata files (i.e. superblock and cmeta
> files) in the first configured data directory. I don't remember why we
> decided to do this (commit 691f97d introduced it), but upon reconsideration
> the WAL directory seems like a much better choice, because if the machine has
> different kinds of I/O devices, the WAL directory's device is typically the
> fastest.
> Mostafa has been testing Impala and Kudu on a cluster with many thousands of
> tablets. His cluster contains storage-dense machines, each configured with 14
> spinning disks and one flash device. Naturally, the WAL directory sits on
> that flash device and the data directories are on the spinning disks. With
> thousands of tablet metadata files on the first spinning disk, nearly every
> tablet in the tserver is bottlenecked on that device due to the sheer amount
> of I/O needed to maintain the running state of the tablet, specifically
> rewriting cmeta files on various Raft events (votes, term advancement, etc.).
> Many thousands of tablets is not really a good scale for Kudu right now, but
> moving the tablet metadata files to a faster device should at least help with
> the above.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
