Re: Understanding out of order message processing w/ Streaming

> But if they arrive out of order, I have to detect / process that myself in > the processor logic. Yes -- if your processing logic depends on the specific ordering of messages (which is the case for you), then you must manually implement this ordering-specific logic at the moment. Other use case

Re: Consumer offsets reset for _all_ topics after increasing partitions for one topic

Created a bug: https://issues.apache.org/jira/browse/KAFKA-4299 On Wed, Oct 5, 2016 at 12:34 PM, Juho Autio wrote: > Does anyone know about this? Altering topic partitions seems to reset > consumer offsets. > > On Tue, Sep 27, 2016 at 1:28 PM, Juho Autio wrote: > >> I increased partitions for o

AUTO: Yan Wang is out of the office (returning 10/17/2016)

I am out of the office until 10/17/2016. Note: This is an automated response to your message "Re: Understanding out of order message processing w/ Streaming" sent on 10/13/2016 2:42:16 AM. This is the only notification you will receive while this person is away. ** This email and any attac

Re: Mirror Maker - Message Format Issue?

Hi Craig, I think you may be running into: https://issues.apache.org/jira/browse/KAFKA-4073 Ismael On Thu, Oct 13, 2016 at 5:51 AM, Craig Swift < craig.sw...@returnpath.com.invalid> wrote: > Hello, > > Just to close this issue out. The 8 producer going to the 10 cluster was > the root issue. T

回复：Re: Re: I found kafka lsot message

Other question: if account of consumer over broker number, whether loss data? if some broker rebooted, whether loss data? Thanks. - 原始邮件 - 发件人：Guozhang Wang 收件人：yangy...@sina.com 抄送人：users 主题：Re: Re: I found kafka lsot message 日期：2016年10月12日 07点0

Call to poll () hangs when invalid port number is given in bootstrap.servers

Hi, We have run into a problem where the call to poll () hangs when an incorrect port number is specified in the bootstrap.servers property. Kafka version is 0.9.0.0. We have tried using both assign and subscribe. The poll problem seems to have been discussed in quite a few threads but we were wo

Re: Force producer topic metadata refresh.

Hello Ismael, yes, I went through that PR and this is what I was actually looking for. Thanks! 2016-10-12 14:37 GMT+02:00 Ismael Juma : > Hi Alexandru, > > I think your issue will be fixed by KAFKA-4254. There's a PR available and > should be merged shortly. Can you please verify? > > Thanks, >

Kafka 0.10 stores offsets in ZooKeeper instead of a Kafka topic

Hi, I use Kafka 0.10 with ZK 3.4.6 and my consumers' offsets aren't stored in the __consumer_offsets topic but in ZK instead. That happens whether I let the consumer commit automatically, or commit manually with enable.auto.commit set to false. Same behavior with `offsets.storage=kafka`, which

Re: Kafka 0.10 stores offsets in ZooKeeper instead of a Kafka topic

I *think* Spark 2.0.0 has a Kafka 0.8 consumer, which would still use the old Zookeeper method. The use the new consumer offsets the consumer needs to be atleast Kafka 0.9 compatible. On Thu, Oct 13, 2016 at 1:55 PM, Samy Dindane wrote: > Hi, > > I use Kafka 0.10 with ZK 3.4.6 and my consumers'

Re: Kafka 0.10 stores offsets in ZooKeeper instead of a Kafka topic

Spark 2.0 has experemental support of kafka 10.0 and you have to explicitly define this in your build e.g. spark-streaming-kafka-0-10 On 13 Oct 2016 16:10, "Ben Davison" wrote: > I *think* Spark 2.0.0 has a Kafka 0.8 consumer, which would still use the > old Zookeeper method. > > The use the new

Re: Kafka 0.10 stores offsets in ZooKeeper instead of a Kafka topic

Ben, Vasilij, thanks for you answer. I forgot to mention I use spark-streaming-kafka-0-10. On 10/13/2016 03:17 PM, Vasilij Syc wrote: Spark 2.0 has experemental support of kafka 10.0 and you have to explicitly define this in your build e.g. spark-streaming-kafka-0-10 On 13 Oct 2016 16:10, "Ben

Problem with ke-value in the KvStorre

Hi, I've designed two processors with a different topology. The issue is that in the firs topology in one node i was able to associate diffrent messages (key,value) where the key could be the same and so i was able to do something like countbyKey. In the second example when i liked to do the

RE: HL7 messages to Kafka consumer

> From: artemerv...@gmail.com > Date: Thu, 13 Oct 2016 00:49:50 -0400 > Subject: Re: HL7 messages to Kafka consumer > To: users@kafka.apache.org > > Nifi HL7 processor is built using HAPI API, which supports z-segments > http://hl7api.sourceforge.net/xref/ca/uhn/hl7v2/examples/CustomModelClasse

RE: Problem with ke-value in the KvStorre

Sorry i was saying anything. Please consider as if i didn't say anything. Kafka does ensure the unicity of the key. De : Hamza HACHANI Envoyé : jeudi 13 octobre 2016 01:38:42 À : users@kafka.apache.org Objet : Problem with ke-value in the KvStorre Hi, I've de

Occasional NPE in NamedCache

Hi Kafka people, I'm joining a bunch of Kafka Topics using Kafka Streams, with the Kafka 0.10.1 release candidate. It runs ok for a few thousand of messages, and then it dies with the following exception: Exception in thread "StreamThread-1" java.lang.NullPointerException at org.apache.kafka.str

Re: Understanding out of order message processing w/ Streaming

Makes sense. Thanks On 13 Oct 2016 12:42 pm, "Michael Noll" wrote: > > But if they arrive out of order, I have to detect / process that myself > in > > the processor logic. > > Yes -- if your processing logic depends on the specific ordering of > messages (which is the case for you), then you mu

Re: Understanding out of order message processing w/ Streaming

I am probably being too ocd anyway. It will almost never happen that messages from another vm in the same network on ec2 arrive out of order. Right? On 13 Oct 2016 8:47 pm, "Ali Akhtar" wrote: > Makes sense. Thanks > > On 13 Oct 2016 12:42 pm, "Michael Noll" wrote: > >> > But if they arrive out

Re: I found kafka lsot message

Watch this talk. Kafka will not lose messages it configured correctly. http://www.confluent.io/kafka-summit-2016-ops-when-it-absolutely-positively-has-to-be-there -hans > On Oct 13, 2016, at 2:48

Re: Occasional NPE in NamedCache

Hi Frank, Thanks for reporting. Can you provide a sample of the join you are running? Thanks, Damian On Thu, 13 Oct 2016 at 16:10 Frank Lyaruu wrote: > Hi Kafka people, > > I'm joining a bunch of Kafka Topics using Kafka Streams, with the Kafka > 0.10.1 release candidate. > > It runs ok for a

Re: Occasional NPE in NamedCache

Hi, i believe i found the problem. If possible could you please try with this: https://github.com/dguy/kafka/tree/cache-bug Thanks, Damian On Thu, 13 Oct 2016 at 17:46 Damian Guy wrote: > Hi Frank, > > Thanks for reporting. Can you provide a sample of the join you are > running? > > Thanks, > D

Re: [kafka-clients] [VOTE] 0.10.1.0 RC2

Hi Jason, Really sorry, but we are going to need to cut another RC. There was a report on the user list w.r.t the NamedCache (in KafkaStreams) throwing a NullPointerException. I've looked into it and it is definitely a bug that needs to be fixed. jira is https://issues.apache.org/jira/browse/KAFK

KafkaProducer 0.9.0.1 Client - Async `send` stops sending

I am running a map-reduce job to queue data from HDFS into Kafka. The mappers simply open a file and senda the data to Kafka using KafkaProducer 0.9.0.1 client. The issue I am debugging is that the Kafka Producer async send fails to `send` to brokers after sometime. I was able to capture the behav

Re: [kafka-clients] [VOTE] 0.10.1.0 RC2

No worries, it wouldn't be interesting unless there were a couple last-minute blockers! We're also got KAFKA-4298 to get in. -Jason On Thu, Oct 13, 2016 at 10:04 AM, Damian Guy wrote: > Hi Jason, > > Really sorry, but we are going to need to cut another RC. There was a > report on the user lis

Re: Occasional NPE in NamedCache

The issue seems to be gone. Amazing work, thanks...! On Thu, Oct 13, 2016 at 6:56 PM, Damian Guy wrote: > Hi, i believe i found the problem. If possible could you please try with > this: https://github.com/dguy/kafka/tree/cache-bug > > Thanks, > Damian > > On Thu, 13 Oct 2016 at 17:46 Damian Guy

Re: Occasional NPE in NamedCache

Thanks Frank for reporting the bug, and many thanks to Damian for the quick catch! On Thu, Oct 13, 2016 at 12:30 PM, Frank Lyaruu wrote: > The issue seems to be gone. Amazing work, thanks...! > > On Thu, Oct 13, 2016 at 6:56 PM, Damian Guy wrote: > > > Hi, i believe i found the problem. If poss

Re: Occasional NPE in NamedCache

BTW this is tracked and resolved as https://issues.apache.org/jira/browse/KAFKA-4300. On Thu, Oct 13, 2016 at 1:17 PM, Guozhang Wang wrote: > Thanks Frank for reporting the bug, and many thanks to Damian for the > quick catch! > > On Thu, Oct 13, 2016 at 12:30 PM, Frank Lyaruu wrote: > >> The i

Re: How can I delete a topic programatically?

Hi Jianbin; I tried like this; Where I provided my zookeeper host. But it says[1] . I use kafka 0.10. And I see my topic available using KafkaManagementTool. How can I overcome this issue? [1]Error while executing topic command : Topic targettopic does not exist on ZK path zookeeper.xx.com:2181/c

kafka-topics.sh ---delete --topic ''x'' is not working for kafka V 0.10.1

HI all; I try to delete a existing topic (I checked using kafka management console) using following command; #./kafka-topics.sh --zookeeper zookeeper.xx.com:2181/chroot --delete --topic testTopic But it says topic not available in zookeeper.[1] I create those topics at runtime . (I think it is c

Re: kafka-topics.sh ---delete --topic ''x'' is not working for kafka V 0.10.1

I overcame the issue. Issue was I had "/chroot" at the end.That is how mentioned in the documentationkafka.apache.org/documentation.html#basic_ops_modify_topic. Thank you. On 14 October 2016 at 12:25, Ratha v wrote: > HI all; > > I try to delete a existing topic (I checked using kafka manageme