In console producer you can specify the producer properties in command line
as metadata-expiry-ms.
You can type just ./kafka-console-producer.sh and it will show you all the
configs that you can specify.
Guozhang
On Wed, Jun 11, 2014 at 10:56 AM, Prakash Gowri Shankor <
prakash.shan...@gmail.co
Guozhang,
I set this in my producer.properties
topic.metadata.refresh.interval.ms=1000
Then I start the console producer as
./kafka-console-producer.sh --broker-list localhost:9092 --topic test2
I still dont see data being written to different partitions after every 1
second.
I wonder if the
Yes, reducing the refresh interval to 100ms will cause it to try to select
another partition every 100ms, not necessarily a different partition tough,
since it just gets a next random int % num.partitions.
Setting the key can also resolve this issue, as long as the key values are
evenly distribute
Can you please tell me how to set this property ?
topic.metadata.refresh.interval.ms
Is a value of 100 low enough to solve this issue ?
Im guessing I can set it to 100 and restart the command line producer and
the partitioning should work ? Please confirm.
Thanks
On Mon, Jun 9, 2014 at 5:09 PM,
Thank you Guozhang.
I've specified how i set and use the property in my previous mail. Can you
tell me if that is fine ?
I also noticed that the kafka-console-producer.sh takes a custom
property(key-value) on the command line. Would it help to set this property
directly on the command line of the p
In the new producer we are changing the default behavior back to pure
random partitioning and let users to customize their own partitioning
schemes if they want. For now reducing topic.metadata.refresh.interval.ms
should help because the stickiness only persists until a metadata refresh.
Guozhang
I have seen that mail thread. Here is what i tried:
In my producer.properties I set topic.metadata.refresh.interval.ms=1000. I
guess this means that the a different partition will be selected every
second.
Then I restart my producer as : ./kafka-console-producer.sh --broker-list
localhost:9092 --
Kane is right, please see this FAQ for details:
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified
?
On Mon, Jun 9, 2014 at 4:41 PM, Kane Kane wrote:
> Last time I've checked it, producer sticks to partition
Is there a way to modify this duration ? This is not adhering to the
"random" behavior that the documentation talks about.
On Mon, Jun 9, 2014 at 4:41 PM, Kane Kane wrote:
> Last time I've checked it, producer sticks to partition for 10 minutes.
>
> On Mon, Jun 9, 2014 at 4:13 PM, Prakash Gowri
Last time I've checked it, producer sticks to partition for 10 minutes.
On Mon, Jun 9, 2014 at 4:13 PM, Prakash Gowri Shankor
wrote:
> Hi,
>
> This is with 0.8.1.1 and I ran the command line console consumer.
> I have one broker, one producer and several consumers. I have one topic,
> many partit
10 matches
Mail list logo
