Yep, I got that after digging a bit, but thanks for the additional wording.
I would update the wiki with this explanation. I also think it would be wise to add notes to other commands that will be replaced by TopicCommand (kafka-topics.sh) so people know where the commands are after they migrate from 0.8 to trunk. Is this transition to command+subcommand pretty much concluded or still subject to change? I already have write access to the wiki so I can update it if you don't mind. On Thu, Jan 30, 2014 at 4:29 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > Yeah this is confusing. > > What they are trying to say is that adding partitions doesn't move data > that is already in existing partitions. I.e. say you have 10 partitions > (0-9) each containing 1GB of data and you add an 11th partition. When this > is complete partitions 0-9 will contain the exact same data as before but > there will be an empty partition 10. > > This is relevant to people who are partitioning their data by some key or > other semantic feature. If you are partitioning by primary_key % > partition_count and the partition_count changes then your data is now in > the wrong partition. That is what the warning is trying to call out. > > -Jay > > > On Thu, Jan 30, 2014 at 12:26 PM, Marc Labbe <mrla...@gmail.com> wrote: > > > Hi, > > > > I am going through the tools documentations for our own ops team. While > > doing so, I noticed the following note from the add partition tool: > > > > ***Please note that this tool can only be used to add partitions when > data > > for a topic does not use a key.*** > > > > This message left me a bit dubious as to why it would affect my topics or > > even the producer. I don't quite follow everything that's happening in > > trunk but I saw the tool has been removed there and found the equivalent > in > > kafka.admin.TopicCommand alterTopic. The message there is a bit clearer > and > > leaves less place for questioning. > > > > >>> WARNING: If partitions are increased for a topic that has a key, the > > partition " + "logic or ordering of the messages will be affected" > > > > With this, I understand that adding partitions for a topic partitioned > with > > a key will potentially affect order of messages consumed because they > might > > end up in different partitions. This is not like adding partitions would > > not work :-) > > > > I would gladly update the wiki to clarify this for anyone looking for the > > same answer as I was. > > > > cheers, > > marc > > >
