I think the more automated/lazy way right now would be to shutdown one broker, rm -rf all its data, add the data directories in config, and restart to let the broker restore off the replicas. This may actually be okay though it is a little slower.
-Jay On Tue, Oct 7, 2014 at 3:25 PM, Jun Rao <jun...@gmail.com> wrote: > You can stop the broker and copy some of the log directories to the new > volumes. You have to be a bit careful when you do that. There are two > metadata files recovery-point-offset-checkpoint and > replication-offset-checkpoint that you have to manually split and copy > over. > > Ideally, we should be able to do this automatically. Perhaps when a broker > starts up, if a "storage.rebalancing" option is enabled, the broker can > automatically copy some of the log dirs around to balance the load among > the different volumes. Could you file a jira to track this? > > Thanks, > > Jun > > On Tue, Oct 7, 2014 at 7:02 AM, Steve Morin <st...@stevemorin.com> wrote: > > > Neha, > > I log volume or can it be volumes plural? > > -Steve > > > > On Tue, Oct 7, 2014 at 6:41 AM, Neha Narkhede <neha.narkh...@gmail.com> > > wrote: > > > > > Is it possible to perform this migration without losing the data > > currently > > > stored in the kafka cluster? > > > > > > Though I haven't tested this, the way this is designed should allow you > > to > > > shut down a broker, move some partition directories over to the new log > > > volume and restart the broker. You will have to do this manually per > > broker > > > though. > > > > > > Thanks, > > > Neha > > > > > > On Tue, Oct 7, 2014 at 3:31 AM, Javier Alba <m...@fjavieralba.com> > wrote: > > > > > > > Hi, > > > > > > > > I have a Kafka 0.8.1.1 cluster consisting in 4 servers with several > > > topics > > > > on it. > > > > > > > > The cluster was initially configured to store kafka log data in a > > single > > > > directory on each server (log.dirs = /tmp/kafka-logs) > > > > > > > > Now, I have assigned 3 new disks to each server and I would like to > use > > > > them to store log data, instead the old directory. (logs.dirs = > > > > /srv/data/1,/srv/data/2,/srv/data/3) > > > > > > > > What would be the recommended way of doing such a migration? > > > > > > > > Is it possible to perform this migration without losing the data > > > currently > > > > stored in the kafka cluster? > > > > > > > > Would it be possible to achieve that kind of change without having to > > > stop > > > > the cluster and losing service? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > >
