Re: Mirror multi-embedded consumer's configuration

I think we need to update the docs. This behavior got changed longtime ago. we need to pass only one consumer configuration. This config will be applicable for all the consumer threads/instances. On Mon, Oct 24, 2016 at 11:43 AM, ZHU Hua B wrote: > Hi, > > >

RE: A question about kafka

Hi, Anybody could help to answer below question? If compression type could be modified through command " bin/kafka-console-producer.sh --producer.config "? Thanks! Best Regards Johnny -Original Message- From: ZHU Hua B Sent: 2016年10月17日 14:52 To: users@kafka.apache.org;

Re: Kafka Streaming

-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Kafka 0.10.1.0 which was release last week does contain the fix already. The fix will be in CP 3.1 coming up soon! (sorry that I did mix up versions in a previous email) - -Matthias On 10/23/16 12:10 PM, Mohit Anchlia wrote: > So if I get it

RE: Mirror multi-embedded consumer's configuration

Hi, Many thanks for your confirm! I have another question, if I deleted a mirrored topic on the target Kafka cluster, if Kafka mirror maker could mirror the same topic again from source cluster when I launch mirror maker next time? Thanks! Best Regards Johnny -Original

RE: Mirror multi-embedded consumer's configuration

Hi, Because from some Kafka wiki I saw "At minimum, the mirror maker takes one or more consumer configurations, a producer configuration and either a whitelist or a blacklist", but the test failed, so I want to know if Kafka mirror really support more consumer configurations at minimum?

Re: Kafka unable to find topic details after sudden EC2 server restart

Thanks for the reply, I tried changing the data directory as follows:- dataDir=/data/zookeeper I also create a /data but still I am getting the following error. ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server. ZooKeeperServerMain) java.io.IOException: Unable to create

RE: Mirror multi-embedded consumer's configuration

Hi, Thanks for your info! Before I launch mirror maker first time, there is a topic include some messages, which have been produced and consumed on source Kafka cluster. Then I launch mirror maker first time on target cluster, but this topic did not mirror from source cluster. So I keep

RE: customised event time

To be more specific. What id do really need is the property of the retention time dor the window in the processor API. Because for the window i think that i can manage to do this. Hamza De : Hamza HACHANI Envoyé : dimanche 23

Re: Mirror multi-embedded consumer's configuration

Yes, it should mirror again. To mirror the topic from beginning, you may have to start with different consumer group, On Mon, Oct 24, 2016 at 12:39 PM, ZHU Hua B wrote: > Hi, > > > Many thanks for your confirm! > > I have another question, if I deleted a mirrored

Re: Mirror multi-embedded consumer's configuration

You need to set auto.offset.reset=smallest to mirror data from beginning On Mon, 24 Oct 2016 at 17:07 ZHU Hua B wrote: > Hi, > > > Thanks for your info! > > > Before I launch mirror maker first time, there is a topic include some > messages, which have been

RE: Mirror multi-embedded consumer's configuration

Hi, It is OK now, thanks for your help! Best Regards Johnny -Original Message- From: tao xiao [mailto:xiaotao...@gmail.com] Sent: 2016年10月24日 17:43 To: users@kafka.apache.org Subject: Re: Mirror multi-embedded consumer's configuration auto.offset.reset=smallest takes effect

RE: A question about kafka

Hi, Anybody could help to answer below question? If compression type could be modified through command " bin/kafka-console-producer.sh --producer.config "? Thanks! Best Regards Johnny -Original Message- From: ZHU Hua B Sent: 2016年10月17日 14:52 To: users@kafka.apache.org;

Re: kafka streaming rocks db lock bug?

Hello Ara, Your encountered issue seems to be KAFKA-3812 , and KAFKA-3938 . Could you try to upgrade to the newly released 0.10.1.0 version and see if this issue goes away? If not I would love to

RE: Mirror multi-embedded consumer's configuration

Yes. //h...@confluent.io Original message From: ZHU Hua B Date: 10/24/16 12:09 AM (GMT-08:00) To: users@kafka.apache.org Subject: RE: Mirror multi-embedded consumer's configuration Hi, Many thanks for your confirm! I have another question,

RE: Removing kafka topic contents from Java

There is a script kafka-topics.sh which calls kafka.admin.TopicCommand. I think you could probably call that same class from your application if you figure out which jars are needed. When I do testing I'll clear out queues by setting the retention time to a second and then wait for kafka to

Removing kafka topic contents from Java

Hi Guys, Is there a way to remove a kafka topic from the java api? I have the following scenario: We have a subscription service, which returns responses to subscribed clients through kafka topic. Per subscription, we create a new kafka topic and send the data by publishing in this new topic.

Re: Kafka unable to find topic details after sudden EC2 server restart

You probably didn't setup permissions properly on the /data directory you created. You might want to do some free online training to learn some more Linux if you are planning to become a Kafka administrator. https://en.wikipedia.org/wiki/Chmod

Re: Kafka Streaming

Would this be an issue if I connect to a remote Kafka instance running on the Linux box? Or is this a client issue. What's rockdb used for to keep state? On Mon, Oct 24, 2016 at 12:08 AM, Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Kafka

Re: Removing kafka topic contents from Java

There isn't a java API for this, you'd have to mess around with bash scripts which I haven't found to be worth it. Just let the data expire and get deleted. Set a short expiry time for the topic if necessary. On Mon, Oct 24, 2016 at 6:30 PM, Demian Calcaprina wrote: >

Re: Removing kafka topic contents from Java

Are you looking something like this String command = String.format("--delete --zookeeper %s --topic %s", "localhost:2181", "test_topic"); String[] split = command.split(" "); TopicCommand.main(split); t SunilKalva On Mon, Oct 24, 2016 at 9:09 PM, Ali Akhtar wrote: >

RE: Mirror multi-embedded consumer's configuration

Hi, Thanks for your info! I set auto.offset.reset=smallest in consumer.properties on target cluster and retest, it still failed. Seems the topic with original messages could not mirror to target cluster while running mirror maker first time, unless produce a new message to this topic to

Re: Kafka unable to find topic details after sudden EC2 server restart

No issues, I have solved the problem by changing the permission of /data Thanks, Gourab On Mon, Oct 24, 2016 at 2:20 PM, Gourab Chowdhury wrote: > Thanks for the reply, I tried changing the data directory as follows:- > dataDir=/data/zookeeper > > I also create a /data

Re: Mirror multi-embedded consumer's configuration

auto.offset.reset=smallest takes effect only when there is no offset committed prior or offset being requested is out of range. I believe your mirror maker has committed offset therefore auto.offset.reset=smallest wont take effect. You can create a new consumer group and try again On Mon, 24 Oct