Can't use different KDCs/realms for Kafka Connect and HDFS Sink Connector?

Greetings, I have setup my Kafka Connect with kerberized Kafka Cluster .(Say the KDC is "kafka-auth101.hadoop.local" and the realm "KAFKA.MYCOMPANY.COM") Now I am trying to setup HDFS Sink with Kerberized Hadoop Cluster with a different KDC (Say the KDC is "hadoop-auth101.hadoop.local" and the

Re: [VOTE] 2.2.0 RC2

+1 (non-binding) - Ran testAll/releaseTarGzAll successfully with no failures. - Ran through quickstart of core/streams on builds generated from 2.2.0-rc2 tag - Ran few internal apps targeting to topics on 3 node cluster. Thanks for running the release Matthias! On Wed, Mar 20, 2019 at 12:43 AM

Re: Using kafka with RESTful API

Thank you!

Re: kafka latency for large message

that's very good information from the slides, thanks. Our design to use kafka has 2 purpose. one is use it as a cache, we use ktable for that purpose, second purpose is use as message delivery mechanism to send it to other system. Because we very much care the latency, the ktable with a compact

Re: [VOTE] 2.2.0 RC2

+1 (non-binding) - Verified the artifacts, build from src, ran tests - Verified the quickstart, ran producer/consumer performance tests. Thanks for running release!. Thanks, Manikumar On Wed, Mar 20, 2019 at 12:19 AM David Arthur wrote: > +1 > > Validated signatures, and ran through

Re: [VOTE] 2.2.0 RC2

+1 Validated signatures, and ran through quick-start. Thanks! On Mon, Mar 18, 2019 at 4:00 AM Jakub Scholz wrote: > +1 (non-binding). I used the staged binaries and run some of my tests > against them. All seems to look good to me. > > On Sat, Mar 9, 2019 at 11:56 PM Matthias J. Sax > wrote:

Re: NOT_LEADER_FOR_PARTITION - Kafka Producer

Appreciate if anyone could shed some light to this issue. Thank you. On Sun, Mar 17, 2019, 12:06 PM Devang Shah wrote: > Dear Kafka Users, > > We have recently discovered *NotLeaderForPartition *in our Kafka producer. > > *Below are the versions we are using,* > Kafka Version (server) -

Re: Proxying the Kafka protocol

You might want to take a look at kafka-proxy ( see https://github.com/grepplabs/kafka-proxy ). It’s a true kafka protocol proxy and modified the metadata like advertized listeners so it works when there is no ip routing between the client and the

Re: Proxying the Kafka protocol

You might follow along with the Envoy proxy team and the work they are doing to support the Kafka binary protocol: https://github.com/envoyproxy/envoy/issues/2852 On Tue, Mar 19, 2019 at 11:46 AM Peter Bukowinski wrote: > https://docs.confluent.io/3.0.0/kafka-rest/docs/intro.html > > The Kafka

Re: Proxying the Kafka protocol

https://docs.confluent.io/3.0.0/kafka-rest/docs/intro.html The Kafka REST proxy may be what you need. You can put multiple instances behind a load balancer to scale to your needs. -- Peter (from phone) > On Mar 19, 2019, at 8:30 AM, Ryanne Dolan wrote: > > Hello James, I'm not aware of

Re: Proxying the Kafka protocol

Hello James, I'm not aware of anything like that for Kafka, but you can use MirrorMaker for network segmentation. With this approach you have one Kafka cluster in each segment and a MM cluster in the more privileged segment. You don't need to expose the privileged segment at all -- you just need

Proxying the Kafka protocol

Hello, We would like to expose a Kafka cluster running on one network to clients that are running on other networks without having to have full routing between the two networks. In this case these networks are in different AWS accounts but the concept applies more widely. We would like to access

Re: Rebalancing and Its Impact

Hey everyone, Requesting some help here.. Thanks, Janardhanan V S On Sat, Mar 16, 2019 at 7:36 AM Janardhanan V S wrote: > Hi, > > I'm new to Kafka and I'm trying to design a wrapper library in both Java > and Go (uses Confluent/Kafka-Go) for Kafka to be used internally. For my > use-case,