Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

Yes. That looks promising to me. Feel free to open an PR after we have a JIRA -- or just create the JIRA right away. -Matthias On 4/4/18 2:57 PM, Ted Yu wrote: > How about the following change ? > > diff --git >

Re: advertised.listeners

Hi, User name is ANONYMOUS, not CN=ANONYMOUS. You can enable authorizer logs (kafka-authorizer.log) and check for any deny errors. supers.users can be configured with same value across all brokers. sh kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal

Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

How about the following change ? diff --git a/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java b/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java index 8d6e56a..92bedad 100644 ---

Kafka high IO wait on certain nodes

Hi, I'm running a 9 node Kafka 0.10.2.1 cluster in AWS and 3-4 of the nodes experience high IO wait but others don't. In terms read/write load, the high IO nodes look identical to others. All nodes were at around 40% disk space usage. No errors or warns in logs. Here is part of the output

Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

https://github.com/apache/kafka/pull/4826 I will fill in JIRA Id once Frederic creates the JIRA. Cheers On Wed, Apr 4, 2018 at 4:29 PM, Matthias J. Sax wrote: > Yes. That looks promising to me. Feel free to open an PR after we have a > JIRA -- or just create the JIRA

SSL Keymanager implementation using OS managed keystore

Hi, Running kafka on Windows OS, I would like to take advantage of windows managed keystore (eg: Windows-MY, can be loaded without a user, pass) .Eventually wants to use a custom keymanager implementation. >From the code I see, SSLFactory doesn't allow to use an existing keystore and >without

kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

Hello, I running tests against kafka-streams 1.1 and get the following stack trace (everything was working alright using kafka-streams 1.0): ERROR org.apache.kafka.streams.processor.internals.AssignedStreamsTasks - stream-thread [feedBuilder-XXX-StreamThread-4] Failed to close stream task,

kafka-streams TopologyTestDriver problem with EXACTLY_ONCE

Hello, I can't successfully test my processing application which require EXACTLY_ONCE processing guarantee using the new TopologyTestDriver. I always get the following exception: java.lang.IllegalStateException: MockProducer hasn't been initialized for transactions The tests all work fine

Re: advertised.listeners

From: Joe Hammerman Sent: Tuesday, April 3, 2018 10:40 PM To: users@kafka.apache.org Subject: Re: advertised.listeners Hi all, Is it possible to run mixed mode with PLAINTEXT and SSL with no SASL? MG>setup SSL on kafka

Re: kafka-streams TopologyTestDriver problem with EXACTLY_ONCE

You saw the error because TopologyTestDriverTest doesn't explicitly call MockProducer#initTransactions(). To do that, TopologyTestDriver needs to expose such method to the TopologyTestDriverTest. You can log a JIRA if you think adding such capability is needed. Cheers On Wed, Apr 4, 2018 at

Re: TimeoutException not being triggered, stuck on OUT_OF_ORDER_SEQUENCE_NUMBER

Which Kafka release are you using ? I was looking at some potentially related JIRA(s), such as KAFKA-6015. FYI On Wed, Apr 4, 2018 at 3:05 AM, Saheb Motiani wrote: > Hi all, > > We have been seeing this issue intermittently, and hence it's difficult to > give a step

Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

Looking at isTransitionValid(): case ABORTING_TRANSACTION: return source == IN_TRANSACTION || source == ABORTABLE_ERROR; The source state is not supposed to be READY. I don't see READY in the log you posted. Please consider logging a JIRA where you can

TimeoutException not being triggered, stuck on OUT_OF_ORDER_SEQUENCE_NUMBER

Hi all, We have been seeing this issue intermittently, and hence it's difficult to give a step by step instructions to reproduce it. I have been studying the code base of the Sender.java (org.apache.kafka.clients.producer.internals.Sender.java), but haven't been able to find the possible bug. We

Re: kafka-streams TopologyTestDriver problem with EXACTLY_ONCE

Thanks Frederic for reporting the issue, I think it is indeed a missing piece that can be added. Would you mind creating a JIRA for this issue? Guozhang On Wed, Apr 4, 2018 at 8:17 AM, Ted Yu wrote: > You saw the error because TopologyTestDriverTest doesn't explicitly

Re: kafka-streams TopologyTestDriver problem with EXACTLY_ONCE

Just a side remark. As a workaround it should be fine to remove the config. TopologyTestDriver will not produce duplicates anyway and is also not suitable to test EOS. -Matthias On 4/4/18 1:26 PM, Guozhang Wang wrote: > Thanks Frederic for reporting the issue, I think it is indeed a missing >

Re: kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION

Thanks for reporting this. It's indeed a bug in Kafka Streams. It's related to this fix: https://issues.apache.org/jira/browse/KAFKA-6634 -- the corresponding PR introduces the issue. Because, we initialize TX delayed, for your case, we never initialize TX and thus aborting the TX fails. Please

Re: advertised.listeners

Hi Manikumar I pushed ACLs for User:ANONYMOUS and when I list them they are listed as shown. Can you please suggest if server.properties needs a change ? *[alpha: user@Kafka1 kafka_2.12-0.10.2.1]$ bin/kafka-acls.sh --authorizer-properties zookeeper.connect=Kafka1-1:2181 --list --topic topic05*

message queueing questions?

Can someone explain how message queueing in partition(s) work? Specifically: Is there any buffering before message is committed to log (written to disk) ? Does the term "partition" refer to sequence of messages written to disk, or are the messages stored in memory queue and written to disk only

RE: Kafka topic retention

Hi Tony, the "log.retention.hours" property is only for topic retention period, this does not affect the topic's deletion. Your data is correctly deleted but, the only way to remove a topic is to use "./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic testTopic" After that

SSL does not work

Hey! I've configured SSL for client connection and it does not work for me: My configuration: listeners=PLAINTEXT://0.0.0.0:7032,SSL://0.0.0.0:9093 security.inter.broker.protocol=PLAINTEXT ssl.client.auth=required ssl.truststore.location=/home/ec2-user/kafka/config/kafka.server.truststore.jks

Re: SSL does not work

Have you seen this thread ? http://search-hadoop.com/m/Kafka/uyzND1aC37obDa1W1?subj=Re+advertised+listeners On Tue, Apr 3, 2018 at 12:28 AM, Eduard wrote: > Hey! > > I've configured SSL for client connection and it does not work for me: > > My configuration: > >