Re: log.dirs balance?

2016-11-30 Thread Tim Visher
Thanks for the report, Karolis. I have a potential theory for how this happened and I'm wondering if it's possibly valid: I have 9 partitions on a machine with 3 disks and they get assigned exactly as you'd expect: d1: t{1,2,3} d2: t{4,5,6} d3: t{7,8,9} Then, a disk fails or something

Re: log.dirs balance?

2016-11-29 Thread Karolis Pocius
It's difficult enough to balance kafka brokers with a single log directory, not to mention attempting to juggle multiple ones. While JBOD is great in terms of capacity, it's a pain in terms of management. After 6 months of constant manual reassignments I ended up going with RAID1+0 which is

log.dirs balance?

2016-11-29 Thread Tim Visher
Hello, My kafka deploy has 5 servers with 3 log disks each. Over the weekend I noticed that on 2 of the 5 servers the partitions appear to be imbalanced amongst the log.dirs. ``` kafka3 /var/lib/kafka/disk1 3 /var/lib/kafka/disk2 3 /var/lib/kafka/disk3 3 kafka5 /var/lib/kafka/disk1 3