Adar Dembo created KUDU-1489:
--------------------------------
Summary: Use WAL directory for tablet metadata files
Key: KUDU-1489
URL: https://issues.apache.org/jira/browse/KUDU-1489
Project: Kudu
Issue Type: Bug
Components: fs, tserver
Affects Versions: 0.9.0
Reporter: Adar Dembo
Priority: Critical
Fix For: 1.0.0
Today a tserver will place tablet metadata files (i.e. superblock and cmeta
files) in the first configured data directory. I don't remember why we decided
to do this (commit 691f97d introduced it), but upon reconsideration the WAL
directory seems like a much better choice, because if the machine has different
kinds of I/O devices, the WAL directory's device is typically the fastest.
Mostafa has been testing Impala and Kudu on a cluster with many thousands of
tablets. His cluster contains storage-dense machines, each configured with 14
spinning disks and one flash device. Naturally, the WAL directory sits on that
flash device and the data directories are on the spinning disks. With thousands
of tablet metadata files on the first spinning disk, nearly every tablet in the
tserver is bottlenecked on that device due to the sheer amount of I/O needed to
maintain the running state of the tablet, specifically rewriting cmeta files on
various Raft events (votes, term advancement, etc.).
Many thousands of tablets is not really a good scale for Kudu right now, but
moving the tablet metadata files to a faster device should at least help with
the above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
