We have the similar solution that have been working in production since 2014.
You can see it here:
The idea is very simple
1. Disk balancer runs in a separate thread inside scheduler pool.
2. It does not touch empty partitions
3. Before it moves a partition it forcibly creates new segment on a destination
4. It moves segment by segment from new to old.
5. Log class works with segments on both disks
Your approach seems too complicated, moreover it means that you have to patch
different components of the system
Could you clarify what do you mean by topicPartition.log? Is it
12.01.2017, 21:47, "Dong Lin" <lindon...@gmail.com>:
> Hi all,
> We created KIP-113: Support replicas movement between log directories.
> Please find the KIP wiki in the link
> This KIP is related to KIP-112
> Handle disk failure for JBOD. They are needed in order to support JBOD in
> Kafka. Please help review the KIP. You feedback is appreciated!