Thats what we're doing as well - appending partition count to the kafka
topic name. This actually helps keep track of the #partitions for each
topic (since Kafka doesn't have a Metadata store yet).

In case of topic expansion - we actually just resort to creating a new
topic. Although that is an overhead - the thought process is that this will
minimize operational errors. Also, this is necessary to do in case we're
doing some kind of joins.

On Wed, Mar 18, 2015 at 5:59 PM, Jakob Homan <> wrote:

> On 18 March 2015 at 17:48, Chris Riccomini <> wrote:
> > One thing I haven't seen, but might be relevant, is including partition
> > counts in the topic.
> Yeah, but then if you change the partition count later on, you've got
> incorrect information forever. Or you need to create a new stream,
> which might be a nice forcing function to make sure your join isn't
> screwed up.  There'd need to be something somewhere to enforce that
> though.

Thanks and regards

Chinmay Soman

Reply via email to