Occasional NPE in NamedCache

2016-10-13 Thread Frank Lyaruu
Hi Kafka people, I'm joining a bunch of Kafka Topics using Kafka Streams, with the Kafka 0.10.1 release candidate. It runs ok for a few thousand of messages, and then it dies with the following exception: Exception in thread "StreamThread-1" java.lang.NullPointerException at

Re: I found kafka lsot message

2016-10-13 Thread Hans Jespersen
Watch this talk. Kafka will not lose messages it configured correctly. http://www.confluent.io/kafka-summit-2016-ops-when-it-absolutely-positively-has-to-be-there -hans > On Oct 13, 2016, at 2:48

Re: Understanding out of order message processing w/ Streaming

2016-10-13 Thread Ali Akhtar
Makes sense. Thanks On 13 Oct 2016 12:42 pm, "Michael Noll" wrote: > > But if they arrive out of order, I have to detect / process that myself > in > > the processor logic. > > Yes -- if your processing logic depends on the specific ordering of > messages (which is the

Re: Occasional NPE in NamedCache

2016-10-13 Thread Damian Guy
Hi Frank, Thanks for reporting. Can you provide a sample of the join you are running? Thanks, Damian On Thu, 13 Oct 2016 at 16:10 Frank Lyaruu wrote: > Hi Kafka people, > > I'm joining a bunch of Kafka Topics using Kafka Streams, with the Kafka > 0.10.1 release candidate. >

Re: [kafka-clients] [VOTE] 0.10.1.0 RC2

2016-10-13 Thread Damian Guy
Hi Jason, Really sorry, but we are going to need to cut another RC. There was a report on the user list w.r.t the NamedCache (in KafkaStreams) throwing a NullPointerException. I've looked into it and it is definitely a bug that needs to be fixed. jira is

KafkaProducer 0.9.0.1 Client - Async `send` stops sending

2016-10-13 Thread Praveen
I am running a map-reduce job to queue data from HDFS into Kafka. The mappers simply open a file and senda the data to Kafka using KafkaProducer 0.9.0.1 client. The issue I am debugging is that the Kafka Producer async send fails to `send` to brokers after sometime. I was able to capture the

Re: Occasional NPE in NamedCache

2016-10-13 Thread Damian Guy
Hi, i believe i found the problem. If possible could you please try with this: https://github.com/dguy/kafka/tree/cache-bug Thanks, Damian On Thu, 13 Oct 2016 at 17:46 Damian Guy wrote: > Hi Frank, > > Thanks for reporting. Can you provide a sample of the join you are >

Re: Occasional NPE in NamedCache

2016-10-13 Thread Guozhang Wang
BTW this is tracked and resolved as https://issues.apache.org/jira/browse/KAFKA-4300. On Thu, Oct 13, 2016 at 1:17 PM, Guozhang Wang wrote: > Thanks Frank for reporting the bug, and many thanks to Damian for the > quick catch! > > On Thu, Oct 13, 2016 at 12:30 PM, Frank

Re: Occasional NPE in NamedCache

2016-10-13 Thread Frank Lyaruu
The issue seems to be gone. Amazing work, thanks...! On Thu, Oct 13, 2016 at 6:56 PM, Damian Guy wrote: > Hi, i believe i found the problem. If possible could you please try with > this: https://github.com/dguy/kafka/tree/cache-bug > > Thanks, > Damian > > On Thu, 13 Oct

Re: [kafka-clients] [VOTE] 0.10.1.0 RC2

2016-10-13 Thread Jason Gustafson
No worries, it wouldn't be interesting unless there were a couple last-minute blockers! We're also got KAFKA-4298 to get in. -Jason On Thu, Oct 13, 2016 at 10:04 AM, Damian Guy wrote: > Hi Jason, > > Really sorry, but we are going to need to cut another RC. There was a >

Re: kafka-topics.sh ---delete --topic ''x'' is not working for kafka V 0.10.1

2016-10-13 Thread Ratha v
I overcame the issue. Issue was I had "/chroot" at the end.That is how mentioned in the documentationkafka.apache.org/documentation.html#basic_ops_modify_topic. Thank you. On 14 October 2016 at 12:25, Ratha v wrote: > HI all; > > I try to delete a existing topic (I

kafka-topics.sh ---delete --topic ''x'' is not working for kafka V 0.10.1

2016-10-13 Thread Ratha v
HI all; I try to delete a existing topic (I checked using kafka management console) using following command; #./kafka-topics.sh --zookeeper zookeeper.xx.com:2181/chroot --delete --topic testTopic But it says topic not available in zookeeper.[1] I create those topics at runtime . (I think it is

Re: How can I delete a topic programatically?

2016-10-13 Thread Ratha v
Hi Jianbin; I tried like this; Where I provided my zookeeper host. But it says[1] . I use kafka 0.10. And I see my topic available using KafkaManagementTool. How can I overcome this issue? [1]Error while executing topic command : Topic targettopic does not exist on ZK path

Re: Consumer offsets reset for _all_ topics after increasing partitions for one topic

2016-10-13 Thread Juho Autio
Created a bug: https://issues.apache.org/jira/browse/KAFKA-4299 On Wed, Oct 5, 2016 at 12:34 PM, Juho Autio wrote: > Does anyone know about this? Altering topic partitions seems to reset > consumer offsets. > > On Tue, Sep 27, 2016 at 1:28 PM, Juho Autio

AUTO: Yan Wang is out of the office (returning 10/17/2016)

2016-10-13 Thread Yan Wang
I am out of the office until 10/17/2016. Note: This is an automated response to your message "Re: Understanding out of order message processing w/ Streaming" sent on 10/13/2016 2:42:16 AM. This is the only notification you will receive while this person is away. ** This email and any

Re: Understanding out of order message processing w/ Streaming

2016-10-13 Thread Michael Noll
> But if they arrive out of order, I have to detect / process that myself in > the processor logic. Yes -- if your processing logic depends on the specific ordering of messages (which is the case for you), then you must manually implement this ordering-specific logic at the moment. Other use

回复：Re: Re: I found kafka lsot message

2016-10-13 Thread yangyuqi
Other question: if account of consumer over broker number, whether loss data? if some broker rebooted, whether loss data? Thanks. - 原始邮件 - 发件人：Guozhang Wang 收件人：yangy...@sina.com 抄送人：users 主题：Re: Re: I

Re: Mirror Maker - Message Format Issue?

2016-10-13 Thread Ismael Juma
Hi Craig, I think you may be running into: https://issues.apache.org/jira/browse/KAFKA-4073 Ismael On Thu, Oct 13, 2016 at 5:51 AM, Craig Swift < craig.sw...@returnpath.com.invalid> wrote: > Hello, > > Just to close this issue out. The 8 producer going to the 10 cluster was > the root issue.

Re: Kafka 0.10 stores offsets in ZooKeeper instead of a Kafka topic

2016-10-13 Thread Samy Dindane
Ben, Vasilij, thanks for you answer. I forgot to mention I use spark-streaming-kafka-0-10. On 10/13/2016 03:17 PM, Vasilij Syc wrote: Spark 2.0 has experemental support of kafka 10.0 and you have to explicitly define this in your build e.g. spark-streaming-kafka-0-10 On 13 Oct 2016 16:10,

Re: Kafka 0.10 stores offsets in ZooKeeper instead of a Kafka topic

2016-10-13 Thread Vasilij Syc
Spark 2.0 has experemental support of kafka 10.0 and you have to explicitly define this in your build e.g. spark-streaming-kafka-0-10 On 13 Oct 2016 16:10, "Ben Davison" wrote: > I *think* Spark 2.0.0 has a Kafka 0.8 consumer, which would still use the > old Zookeeper

Re: Kafka 0.10 stores offsets in ZooKeeper instead of a Kafka topic

2016-10-13 Thread Ben Davison
I *think* Spark 2.0.0 has a Kafka 0.8 consumer, which would still use the old Zookeeper method. The use the new consumer offsets the consumer needs to be atleast Kafka 0.9 compatible. On Thu, Oct 13, 2016 at 1:55 PM, Samy Dindane wrote: > Hi, > > I use Kafka 0.10 with ZK

RE: HL7 messages to Kafka consumer

2016-10-13 Thread Martin Gainty
> From: artemerv...@gmail.com > Date: Thu, 13 Oct 2016 00:49:50 -0400 > Subject: Re: HL7 messages to Kafka consumer > To: users@kafka.apache.org > > Nifi HL7 processor is built using HAPI API, which supports z-segments >