Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-02-01 Thread Alexey Ozeritsky
4=5> > for the change of the KIP. I will come up with a solution to throttle > replica movement when a broker is moving a "leader" replica. Thanks. It looks great. > > On Tue, Jan 24, 2017 at 3:30 AM, Alexey Ozeritsky <aozerit...@yandex.ru> > wrote: > >>

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-01-23 Thread Alexey Ozeritsky
tOffset). In this case the write speed to topicPartition.move and topicPartition.log will be the same so this will allow us to move many partitions to one disk. > > And to answer your question, yes topicpartition.log refers to > topic-paritition/segment.log. > > Thanks, > Dong > >

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-01-24 Thread Alexey Ozeritsky
23.01.2017, 22:11, "Dong Lin" <lindon...@gmail.com>: > Thanks. Please see my comment inline. > > On Mon, Jan 23, 2017 at 6:45 AM, Alexey Ozeritsky <aozerit...@yandex.ru> > wrote: > >>  13.01.2017, 22:29, "Dong Lin" <lindon...@gmail.co

Re: Prevent broker from leading topic partitions

2016-08-22 Thread Alexey Ozeritsky
Hi 22.08.2016, 19:16, "Tom Crayford" : > Hi, > > I don't think I understand *why* you need this. Kafka is by default a > distributed HA system that balances data and leadership over nodes. Why do > you need to change this? In fact, kafka does not balance anything. It uses

Re: [DISCUSS] KIP-113: Support replicas movement between log directories

2017-01-13 Thread Alexey Ozeritsky
Hi, We have the similar solution that have been working in production since 2014. You can see it here: https://github.com/resetius/kafka/commit/20658593e246d2184906879defa2e763c4d413fb The idea is very simple 1. Disk balancer runs in a separate thread inside scheduler pool. 2. It does not touch