Re: ProcessorTopology

2018-02-23 Thread Bill Bejeck
Hi Pravin, Assume you have two topologies, A and B. When building topology "A" you'll have a sink node writing to "topic-A-results", and when building topology "B" you'll specify "topic-A-results" as its source. You'll need to create these topics ahead of time. HTH Bill On Fri, Feb 23, 2018

Re: connecting to 2 different clusters with different sasl realms in a single process

2018-02-23 Thread Kamal
I'd like to point out that you can pass multiple jaas config within a jvm using `sasl.jaas.config` in producer & consumer config. On Mon, Feb 19, 2018 at 8:10 PM, Michal Hariš wrote: > Hi all, > > I have one cluster with kerberos authenticator and another with simple > authenticator. I need to b

Re: ProcessorTopology

2018-02-23 Thread Jacek Laskowski
Hi, Bill gave the perfect answer with a sink and a source as the connecting point, but thought I'd add a note. A topology is just a description of what "processing code" to execute when the topology gets "materialized". In other words, it's a graph of nodes where nodes are processing units. It wo

Kafka Consumer v1.0 terminates prematurely

2018-02-23 Thread Javed, Haseeb
Hello, I have a test cluster for Kafka consisting of one broker, one producer and one consumer, each on a separate node. Kafka broker and producer are v1.0.0. I am able to use the ProducerPerformance benchmark to write records to the broker. However, when I try to read the records using Consum

Restricting client access to zookeeper metadata

2018-02-23 Thread Reema Chugani
Hi, I am using Kafka 0.10.2. I have multiple topics & consumers set up with ACLS such that consumer can only read from a particular topic. I am wondering how I can prevent a consumer from accessing metadata in zookeeper about other topics? i.e, prevent consumers from listing or getting info ab

Re: Restricting client access to zookeeper metadata

2018-02-23 Thread Sönke Liebau
Hi Reema, if your departments have access to Zookeeper then there probably is not much you can do about them accessing data on other departments topics. I assume that you have enabled Zookeeper ACLs, but even with those in place, the topic metadata is world readable, so listing topics can be done