Yonghui Yang created KAFKA-2796:
-----------------------------------
Summary: add support for reassignment partition to specified logdir
Key: KAFKA-2796
URL: https://issues.apache.org/jira/browse/KAFKA-2796
Project: Kafka
Issue Type: Improvement
Components: clients, controller, core, log
Affects Versions: 0.9.0.0
Reporter: Yonghui Yang
Assignee: Neha Narkhede
Fix For: 0.9.0.0
Currently when creating a log, the directory is chosen by calculating the
number of partitions
in each directory and then choosing the data directory with the fewest
partitions.
However, the sizes of different TopicParitions are very different, which lead
to usage vary greatly between different logDirs. And usually each logDir
corresponds to a disk, so the disk usage between different disks is very
imbalance .
The possible solution is to reassign partitions in high-usage logDirs to
low-usage logDirs. I change the format of /admin/reassign_partitions，add
replicaDirs field. At reassigning Partitions, when broker’s
LogManager.createLog() is invoked , if replicaDir is specified , the specified
logDir will be chosen, otherwise the logDir with the fewest partitions will be
chosen.
the old /admin/reassign_partitions:
{"version":1,
"partitions":
[
{
"topic" : "Foo",
"partition": 1,
"replicas": [1, 2, 3]
}
]
}
the new /admin/reassign_partitions:
{"version":1,
"partitions":
[
{
"topic" : "Foo",
"partition": 1,
"replicas": [1, 2, 3],
"replicaDirs": {"1":"/data1/kafka_data", "3":"/data10/kakfa_data" }
}
]
}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
