Reducing # partitions is going to be tricky. The data for those dropped partitions will just be lost.
Thanks, Jun On Tue, Jan 8, 2013 at 1:24 PM, David Ross <dyr...@klout.com> wrote: > Yeah that makes sense, but what if we do need to change the number of > partitions? What if we need to reduce it? > > On Tue, Jan 8, 2013 at 12:42 PM, Jun Rao <jun...@gmail.com> wrote: > > > If you don't have a lot of topics, one thing you can do is to > > over-partition a topic. > > > > Also, in 0.7, # of partitions grows with brokers. This is going to change > > in 0.8, in which # of partitions is specified at topic creation time and > > won't change as brokers change. One needs to use an admin DDL to change # > > of partitions. > > > > Thanks, > > > > Jun > > > > On Mon, Jan 7, 2013 at 10:23 PM, David Ross <dyr...@klout.com> wrote: > > > > > Hello, > > > > > > We have found that, for our application, having a number of total > > > partitions as a multiple of the number of consumer hosts is beneficial. > > > Because of this, whenever we add or remove consumer hosts, we have to > > > change the number of partitions in the server config. > > > > > > What are best practices for changing the number of partitions? It seems > > > like adding partitions is fine but removing partitions would result in > > data > > > loss - am I right? Is that avoidable? Is it preferable to bring in new > > > servers with new partitions? Anything else I should keep in mind on > this > > > issue? > > > > > > > > > Thanks! > > > > > > David > > > > > >
