How to decrease number of replicas

2015-06-18 Thread Guangle Fan
Hi, Folks We have an online kafka cluster v0.8.1.1. After running a partition reassignment script which maps each partition to 3 replicas. But growth of data is out of my expectation, and I really need to decrease replicas for each partition to 2 or 1. What's the best way to do this ? Thanks !

Re: How to decrease number of replicas

2015-06-18 Thread Guangle Fan
) for partition [topic,0] On Thu, Jun 18, 2015 at 5:03 PM, Guangle Fan fanguan...@gmail.com wrote: Hi, Folks We have an online kafka cluster v0.8.1.1. After running a partition reassignment script which maps each partition to 3 replicas. But growth of data is out of my expectation, and I really

Re: How to decrease number of replicas

2015-06-18 Thread Guangle Fan
is rolling restart stuck leaders for each partition, and sent out replica reassignment once again. On Thu, Jun 18, 2015 at 5:37 PM, Guangle Fan fanguan...@gmail.com wrote: If I use the same approach to reassign smaller number of replicas to the same partition, I got this error : (0,5,1,6,2,3

Re: .deleted file descriptors

2015-03-02 Thread Guangle Fan
March 2015 10:30 AM, Guangle Fan wrote: Slightly different from what I observed. Broker box has 800GB disk space. By setting the appropriate log retention, it's supposed to hold the log size. But then the usage of disk hits 90%, and by doing nothing but restarting broker server. It free 40% disk

Re: .deleted file descriptors

2015-03-01 Thread Guangle Fan
, it only adds a suffix .deleted to the files such that it will not be access any more by other Kafka threads. The actual file deletion will be executed later, with period controlled by file.delete.delay.ms (default 1 minute). On Fri, Feb 27, 2015 at 9:49 PM, Guangle Fan fanguan...@gmail.com

.deleted file descriptors

2015-02-27 Thread Guangle Fan
Hi, After Kafka cleaned .log / .index files based on topic retention. I can still lsof a lot of .index.deleted files. And df shows usage on disk space is accumulated to full. When this happened, just by restarting broker, it will immediately free those disk space. I seems to me kafka after

rebalance brokers

2014-09-29 Thread Guangle Fan
Hi, All We have some old kafka nodes that are still running 0.7. We disable the replica of topics there. Recently we added some node brokers to that cluster, I found new nodes won't got written or read even though it's registered correctly in zookeeper. I think in this case, it's because all

Re: rebalance brokers

2014-09-29 Thread Guangle Fan
, Replication is only introduced in 0.8, with 0.7 there should not have leader / follower replicas. Could you verify the version of your Kafka cluster? Guozhang On Mon, Sep 29, 2014 at 4:30 PM, Guangle Fan fanguan...@gmail.com wrote: Hi, All We have some old kafka nodes that are still running 0.7

Join the list

2014-07-15 Thread Guangle Fan
Hi, Could you add me into the mailing list ?