Re: Kafka Streams 0.10.2.1 client crash - .checkpoint.tmp (No such file or directory)

Hello, I am able to reproduce this. It occurs during rebalancing when the service is restarted. kafka-clients and kafka-streams are both at version 0.10.2.1. 3 instances of the service, 4 threads per instance, 100 partitions. log excerpt: Wed Jul 26 05:32:07 UTC 2017 Streams state:

How do I setup Kafka Streams application to have one task per partition?

Hello All, I have more than 100 topics in Kafka with one partition each. These 100 topics are configured through a regex. When running the application, I found that there is only one task that is being spawned as the default partition grouper in Kafka spawns as many tasks as the maximum number of

Re: Kafka with SSL enabled - Not able to publish messages (org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms)

enable debug logs to find out the actual error. On Wed, Jul 26, 2017 at 12:49 AM, karan alang wrote: > hi - I've enabled SSL for Kafka & i'm trying to publish messages using > console Producer > > Error is as shown below, any ideas ? > >> >>1.

Consumer group metadata retention

I am writing an exactly-once Kafka sink for Apache Beam. In order to avoid duplicates due to retries, it stores a sequential id and producer signature in consumer group metadata, and commits it atomically with messages (using sendOffsetsToTransaction()). I have a couple of clarification questions

Re: consumer group offset lag

I am also facing the same issue. On Tue, Jul 25, 2017 at 9:58 PM, Tarun Garg wrote: > I am sending this mail to DEV because i think this is a bug in AdminClient > while getting the group offset. > > Hi, > > I run through a issue. > i am trying to get the status of a consumer

Re: [DISCUSS] KIP-174 - Deprecate and remove internal converter configs in WorkerConfig

Umesh, Thanks for the KIP. Straightforward and I think it's a good change. Unfortunately it is hard to tell how many people it would affect since we can't tell how many people have adjusted that config, but I think this is the right thing to do long term. A couple of quick things that might be

Kafka with SSL enabled - Not able to publish messages (org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms)

hi - I've enabled SSL for Kafka & i'm trying to publish messages using console Producer Error is as shown below, any ideas ? > >1. /usr/hdp/2.5.3.0-37/kafka/bin/kafka-console-producer.sh --broker-list >nwk2-bdp-kafka-05.gdcs-qa.apple.com:6668,nwk2-bdp-kafka-04.gdcs-qa. >

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

Hi, If there is no further feedback on this KIP, I'll start the vote tomorrow. Thanks. --Vahid From: Vahid S Hashemian/Silicon Valley/IBM To: dev , "Kafka User" Date: 07/03/2017 04:06 PM Subject:[DISCUSS] KIP-175: Additional

consumer group offset lag

I am sending this mail to DEV because i think this is a bug in AdminClient while getting the group offset. Hi, I run through a issue. i am trying to get the status of a consumer group. for this i have consumer and producer with topic named test_7 with partition count 1. It is working fine.

Re: Schema Registry on DC/OS

The reason I was looking for the CLI was to inspect into the service and get information like the host, port etc. But later I found I can get the same using the task command of dcos .. $ dcos marathon task list --json /schema-registry So, it's ok .. I don't need the specific CLI .. regards. On

Re: Schema Registry on DC/OS

The Confluent Schema Registry is a RESTful service, so no CLI really. What's your use case? If you are not familiar with it, the quickstart docs is a good place to start: http://docs.confluent.io/current/schema-registry/docs/intro.html#quickstart On Tue, Jul 25, 2017 at 2:09 AM, Debasish Ghosh

Re: Event sourcing with Kafka and Kafka Streams. How to deal with atomicity

I didn't say anything... my producer wasn't working properly, messages are arriving now. Finally I got it working :-) Thanks so much!! 2017-07-25 12:23 GMT+02:00 José Antonio Iñigo : > Hi Ben, > > now I can see what you meant previously about using a Transformer. I

Re: Event sourcing with Kafka and Kafka Streams. How to deal with atomicity

Hi Ben, now I can see what you meant previously about using a Transformer. I was following a wrong approach dividing the processing between a Listener and a Stream processor. There's only one thing left that I don't know how to work out, this a draft of my code based on yours: @Bean

Re: DAG processing in Kafka Streams

Thanks Mathhias for answering this. I lost your reply in my inbox, for sorry for replying late. I get what you are trying to say. I was actually exploring opportunities where can we make Kafka Streams faster since I am currently evaluating migrating a project from Spark Streaming to Kafka

Re: Schema Registry on DC/OS

Thanks a lot .. I found it from the community supported packages on the DC/OS UI. Installed it and it runs ok. One question - is there any CLI for confluent-schema-registry ? dcos package install confluent-schema-registry --cli does not give anything .. regards. On Tue, Jul 25, 2017 at 9:50 AM,