kafka streams and broadcast topic

I'd like to have the following: One large stream of content coming through a topic, with Kafka Stream filtering to identify records of interest. I can see how this would be sharded to allow scale out to handle a large stream of content. I would like to have a 2nd, smaller, topic to define the

Re: Interesting error message du jour

K - so no big deal. TY On Fri, Dec 30, 2016 at 12:13 PM, Ewen Cheslack-Postava wrote: > Jon, > > This looks the same as https://issues.apache.org/jira/browse/KAFKA-4563, > although for a different invalid transition. The temporary fix suggested > there is to simply convert

Re: Interesting error message du jour

Jon, This looks the same as https://issues.apache.org/jira/browse/KAFKA-4563, although for a different invalid transition. The temporary fix suggested there is to simply convert the exception to log a warning, which should be a pretty trivial patch against trunk. It seems there are some

Re: Setup Kerberos for Kafka on Ubuntu Linux

Thanks Gerrit, let me try the pointers you mentioned above. Thanks a lot for your help. On Fri, Dec 30, 2016 at 8:57 AM, Gerrit Jansen van Vuuren < gerrit...@gmail.com> wrote: > also, before tinkering with Kerberos, I'd advise to read the first 4 > chapters of "Kerberos: The Definite Guide" >

Re: Setup Kerberos for Kafka on Ubuntu Linux

also, before tinkering with Kerberos, I'd advise to read the first 4 chapters of "Kerberos: The Definite Guide" https://www.amazon.com/Kerberos-Definitive-Guide-Jason-Garman/dp/0596004036/ref=sr_1_1?ie=UTF8=1483116842=8-1=kerberos It helped me and after a week of reading I could get started

Re: Setup Kerberos for Kafka on Ubuntu Linux

make sure kafka1 is the FQN and that the server kafka1 can resolve properly from you're kerberos server, EXAMPLE.COM should be a realm that is configured in krb5.conf and kdc.conf, with the adequate domain mappings for kafka1 to this realm. Kerberos is a pain and there are tons of stuff that can

Setup Kerberos for Kafka on Ubuntu Linux

Hi I have never dabbled Kafka with security settings. I was trying to follow this blog to get it working: http://kafka.apache.org/documentation.html#security_sasl But I can't seem to add principals for Kafka in Kerberos server, and I am getting the following error: dp@kdc:~$ sudo

Re: Interesting error message du jour

Attaching the debug log On Fri, Dec 30, 2016 at 6:39 AM, Jon Yeargers wrote: > Using 0.10.2.0-snapshot: > > java.lang.IllegalStateException: Incorrect state transition from > ASSIGNING_PARTITIONS to ASSIGNING_PARTITIONS > > at

Interesting error message du jour

Using 0.10.2.0-snapshot: java.lang.IllegalStateException: Incorrect state transition from ASSIGNING_PARTITIONS to ASSIGNING_PARTITIONS at org.apache.kafka.streams.processor.internals.StreamThread.setState(StreamThread.java:163) at

Re: Memory / resource leak in 0.10.1.1 release

FWIW: I went through and removed all the 'custom' serdes from my code and replaced them with 'string serdes'. The memory leak problem went away. The code is a bit more cumbersome now as it's constantly flipping back and forth between Objects and JSON.. but that seems to be what it takes to keep