Thanks, Manikumar for pointing the typo. Upon setting the ACL rules are
told by you above, it worked fine. I was able to run secure (on external
interface) and insecure (on internal interface) mode.

As a reference to others in the forum, in addition to the server.properties
that I have posted above, these are the two things I did:

1. Changed super.users to


*super.users=User:CN=Kafka1;UserANONYMOUS2. Used the ACL rule as told by
Manikumar:bin/kafka-acls.sh --authorizer-properties
zookeeper.connect=localhost:2181 --add --allow-principal User:ANONYMOUS
--allow-host \* --operation Read --topic test*
Many thanks Manikumar.

Thanks.


On Wed, Apr 4, 2018 at 7:25 PM, Manikumar <manikumar.re...@gmail.com> wrote:

> Hi,
>
> User name is ANONYMOUS, not CN=ANONYMOUS. You can enable authorizer  logs
> (kafka-authorizer.log) and check for any deny errors.
> supers.users can be configured with same value across all brokers.
>
> sh kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181
> --add --allow-principal User:ANONYMOUS --allow-host \* --operation Read
> --topic test
>
> On Thu, Apr 5, 2018 at 2:39 AM, Darshan <purandare.dars...@gmail.com>
> wrote:
>
>> Hi Manikumar
>>
>> I pushed ACLs for User:ANONYMOUS and when I list them they are listed as
>> shown. Can you please suggest if server.properties needs a change ?
>>
>> *[alpha: user@Kafka1 kafka_2.12-0.10.2.1]$ bin/kafka-acls.sh
>> --authorizer-properties zookeeper.connect=Kafka1-1:2181 --list --topic
>> topic05*
>> *Current ACLs for resource `Topic:topic05`:*
>> *        User:CN=ANONYMOUS has Allow permission for operations: Describe
>> from hosts: **
>> *        User:CN=Producer05 has Allow permission for operations: Write
>> from hosts: **
>> *        User:CN=Producer05 has Allow permission for operations: Describe
>> from hosts: **
>> *        User:CN=ANONYMOUS has Allow permission for operations: Write
>> from hosts: **
>>
>> When I start sending messages using kafka console client, I see the
>> following error. topic05 is created with 16 partitions, but no directory is
>> created in logs directory. Can someone please help ?
>>
>> *[alpha: user@Kafka1 kafka_2.12-0.10.2.1]$ bin/kafka-console-producer.sh
>> --broker-list Kafka1:9092 --topic topic05*
>> *WorldHello*
>> *[2018-04-04 21:01:40,338] WARN Error while fetching metadata with
>> correlation id 1 : {topic05=UNKNOWN_TOPIC_OR_PARTITION}
>> (org.apache.kafka.clients.NetworkClient)*
>> *[2018-04-04 21:01:40,437] WARN Error while fetching metadata with
>> correlation id 2 : {topic05=UNKNOWN_TOPIC_OR_PARTITION}
>> (org.apache.kafka.clients.NetworkClient)*
>> *[2018-04-04 21:01:40,538] WARN Error while fetching metadata with
>> correlation id 3 : {topic05=UNKNOWN_TOPIC_OR_PARTITION}
>> (org.apache.kafka.clients.NetworkClient)*
>> *[2018-04-04 21:01:40,640] WARN Error while fetching metadata with
>> correlation id 4 : {topic05=UNKNOWN_TOPIC_OR_PARTITION}
>> (org.apache.kafka.clients.NetworkClient)*
>> *[2018-04-04 21:01:40,740] WARN Error while fetching metadata with
>> correlation id 5 : {topic05=UNKNOWN_TOPIC_OR_PARTITION}
>> (org.apache.kafka.clients.NetworkClient)*
>> *[2018-04-04 21:01:40,841] WARN Error while fetching metadata with
>> correlation id 6 : {topic05=UNKNOWN_TOPIC_OR_PARTITION}
>> (org.apache.kafka.clients.NetworkClient)*
>> *[2018-04-04 21:01:40,943] WARN Error while fetching metadata with
>> correlation id 7 : {topic05=UNKNOWN_TOPIC_OR_PARTITION}
>> (org.apache.kafka.clients.NetworkClient)*
>> *[2018-04-04 21:01:41,045] WARN Error while fetching metadata with
>> correlation id 8 : {topic05=UNKNOWN_TOPIC_OR_PARTITION}
>> (org.apache.kafka.clients.NetworkClient)*
>>
>>
>>
>>
>>
>>
>>
>> *server.properties for our Kafka-1,2,3. They are identical
>> except broker.id <http://broker.id/> and super.users properties.# ID and
>> basic topic creationbroker.id
>> <http://broker.id/>=1auto.create.topics.enable=truedelete.topic.enable=true#
>> LISTERN Settingslisteners=INTERNAL://1.1.1.165:9092
>> <http://1.1.1.165:9092/>,EXTERNAL://172.21.190.176:9093
>> <http://172.21.190.176:9093/>advertised.listeners=INTERNAL://1.1.1.165:9092
>> <http://1.1.1.165:9092/>,EXTERNAL://172.21.190.176:9093
>> <http://172.21.190.176:9093/>listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSLinter.broker.listener.name
>> <http://inter.broker.listener.name/>=INTERNALhost.name
>> <http://host.name/>=172.21.190.176# Security
>> Settingsssl.keystore.location=keystore.jksssl.keystore.password=passwordssl.key.password=passwordssl.truststore.location=truststore.jksssl.truststore.password=passwordssl.keystore.type=JKSssl.truststore.type=JKSsecurity.protocol=SSLssl.client.auth=requiredallow.everyone.if.no.acl.found=falseauthorizer.class.name
>> <http://authorizer.class.name/>=kafka.se
>> <http://kafka.se>curity.auth.SimpleAclAuthorizersuper.users=User:CN=Kafka1*
>>
>>
>> On Tue, Apr 3, 2018 at 10:42 PM, Manikumar <manikumar.re...@gmail.com>
>> wrote:
>>
>>> @Darshan,
>>> For PLAINTEXT channels, principal will be "ANONYMOUS". You need to give
>>> produce/consume permissions
>>>  to "User:ANONYMOUS"
>>>
>>>
>>> On Wed, Apr 4, 2018 at 8:10 AM, Joe Hammerman <
>>> jhammer...@squarespace.com.invalid> wrote:
>>>
>>> > Hi all,
>>> >
>>> > Is it possible to run mixed mode with PLAINTEXT and SSL with no SASL?
>>> What
>>> > should port and advertised listeners values in kafka-server.properties
>>> be
>>> > set in order to configure such an access profile? We want to so in
>>> order to
>>> > be able to perform healthchecking on the loopback device without
>>> having to
>>> > negotiate an SSL connection.
>>> >
>>> > Thanks in advance for any assistance anyone can provide,
>>> > Joseph Hammerman
>>> >
>>> > On Tue, Apr 3, 2018 at 8:06 PM, Martin Gainty <mgai...@hotmail.com>
>>> wrote:
>>> >
>>> > > (from MessageBroker consumer)
>>> > >
>>> > > >tracert MessageProducer
>>> > >
>>> > > if zk server is found in tracert
>>> > >
>>> > > then yes you have a MB quorom
>>> > >
>>> > > FWIR Mixing PLAINTEXT with SASL-SSL on ZK  is not supported
>>> > > https://stackoverflow.com/questions/46912937/is-it-
>>> > > possible-to-connect-zookeeper-and-kafka-via-sasl-kafka-broke
>>> r-and-its-cl
>>> > >
>>> > > but there is a ZK SASL SASL-Auth exception backdoor  in
>>> > > zookeeper.properties when
>>> > > zookeeper.main_connection_despite_sasl_failure="yes"
>>> > >
>>> > >
>>> > >  catch (SaslException e) {
>>> > >     LOG.warn("Client failed to SASL authenticate: " + e);
>>> > >     if ((System.getProperty("zookeeper.maintain_connection_
>>> > despite_sasl_failure")
>>> > > != null)
>>> > >         &&
>>> > >         (System.getProperty("zookeeper.maintain_connection_
>>> > > despite_sasl_failure").equals("yes"))) {
>>> > >       LOG.warn("Maintaining client connection despite SASL
>>> authentication
>>> > > failure.");
>>> > >     } else {
>>> > >       LOG.warn("Closing client connection due to SASL authentication
>>> > > failure.");
>>> > >       cnxn.close();
>>> > >    ..
>>> > >  }
>>> > >
>>> > >
>>> > > does this conform to your findings?
>>> > >
>>> > > M
>>> > > ______________________________________________
>>> > >
>>> > >
>>> > >
>>> > > ________________________________
>>> > > From: Darshan <purandare.dars...@gmail.com>
>>> > > Sent: Tuesday, April 3, 2018 6:47 PM
>>> > > To: users@kafka.apache.org
>>> > > Subject: Re: advertised.listeners
>>> > >
>>> > > We are using 0.10.2.1 and ZK 3.4.9. Can something be derived from
>>> this
>>> > > piece of info ? Thanks.
>>> > >
>>> > > On Tue, Apr 3, 2018 at 3:13 PM, Martin Gainty <mgai...@hotmail.com>
>>> > wrote:
>>> > >
>>> > > > tracert MessageBrokerIP
>>> > > >
>>> > > > do you see ZK server in the trace?
>>> > > >
>>> > > > if yes then you are running kafka-cluster
>>> > > >
>>> > > > (ZK does not support mixed mode but there is a backdoor
>>> > > > zookeeper.properties config attribute that allows plaintext
>>> clients to
>>> > > > bypass sasl auth)
>>> > > >
>>> > > > ?
>>> > > >
>>> > > > Martin
>>> > > > ______________________________________________
>>> > > >
>>> > > >
>>> > > >
>>> > > > ________________________________
>>> > > > From: Darshan <purandare.dars...@gmail.com>
>>> > > > Sent: Tuesday, April 3, 2018 5:45 PM
>>> > > > To: rajinisiva...@gmail.com
>>> > > > Cc: users@kafka.apache.org
>>> > > > Subject: Re: advertised.listeners
>>> > > >
>>> > > > Hi Rajini
>>> > > >
>>> > > > The above configuration that you mentioned a while back helped me
>>> sort
>>> > > the
>>> > > > issue of listeners and I was also able to run Kafka 0.10.2.1 with
>>> SSL
>>> > and
>>> > > > ACLs as well from one of your other posts.
>>> > > >
>>> > > > I wanted to ask you if it is possible to run Kafka in a mixed
>>> security
>>> > > > mode? i.e. external producers who are on 172.x.x.x interface can
>>> use
>>> > the
>>> > > > SSL to send/receive data from/to our Kafka brokers, but our
>>> internal
>>> > > > consumer can read/write on PLAINTEXT channel to Kafka.
>>> > > >
>>> > > > Here is my server.properties, but my internal producer which does
>>> not
>>> > > have
>>> > > > any keystore and truststore is getting the following error:
>>> > > > *[2018-04-03 21:21:04,378] WARN Error while fetching metadata with
>>> > > > correlation id 1 : {Topic4006=TOPIC_AUTHORIZATION_FAILED}
>>> > > > (org.apache.kafka.clients.NetworkClient)*
>>> > > >
>>> > > >
>>> > > > *server.properties for our Kafka-1,2,3. They are identical except
>>> > > > broker.id
>>> > > > <http://broker.id> and super.users properties.*
>>> > > broker.id -&nbspThis website is for sale! -&nbspbroker Resources and
>>> > > Information.<http://broker.id/>
>>> > > broker.id
>>> > > This website is for sale! broker.id is your first and best source
>>> for
>>> > all
>>> > > of the information you’re looking for. From general topics to more of
>>> > what
>>> > > you would expect to find here, broker.id has it all. We hope you
>>> find
>>> > > what you are searching for!
>>> > >
>>> > >
>>> > >
>>> > > > broker.id -&nbspThis website is for sale! -&nbspbroker Resources
>>> and
>>> > > > Information.<http://broker.id/>
>>> > > > broker.id
>>> > > > This website is for sale! broker.id is your first and best source
>>> for
>>> > > all
>>> > > > of the information you’re looking for. From general topics to more
>>> of
>>> > > what
>>> > > > you would expect to find here, broker.id has it all. We hope you
>>> find
>>> > > > what you are searching for!
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > # ID and basic topic creation
>>> > > > broker.id=1
>>> > > > auto.create.topics.enable=true
>>> > > > delete.topic.enable=true
>>> > > >
>>> > > > # LISTERN Settings
>>> > > > listeners=INTERNAL://1.1.1.165:9092,EXTERNAL://172.21.190.176:9093
>>> > > > advertised.listeners=INTERNAL://1.1.1.165:9092,EXTERNAL://
>>> > > > 172.21.190.176:9093
>>> > > > listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
>>> > > > inter.broker.listener.name=INTERNAL
>>> > > > host.name=172.21.190.176
>>> > > >
>>> > > > # Security Settings
>>> > > > ssl.keystore.location=keystore.jks
>>> > > > ssl.keystore.password=password
>>> > > > ssl.key.password=password
>>> > > > ssl.truststore.location=truststore.jks
>>> > > > ssl.truststore.password=password
>>> > > > ssl.keystore.type=JKS
>>> > > > ssl.truststore.type=JKS
>>> > > > security.protocol=SSL
>>> > > > ssl.client.auth=required
>>> > > > allow.everyone.if.no.acl.found=false
>>> > > > authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
>>> > > > super.users=User:CN=Kafka1
>>> > > >
>>> > > > Can you please point out if anything needs to be modified ?
>>> > > >
>>> > > > Many thanks.
>>> > > >
>>> > > > --Darshan
>>> > > >
>>> > > > On Wed, May 31, 2017 at 11:31 AM, Rajini Sivaram <
>>> > > rajinisiva...@gmail.com>
>>> > > > wrote:
>>> > > >
>>> > > > > If you want to use different interfaces with the same security
>>> > > protocol,
>>> > > > > you can specify listener names. You can then also configure
>>> different
>>> > > > > security properties for internal/external if you need.
>>> > > > >
>>> > > > > listeners=INTERNAL://1.x.x.x:9092,EXTERNAL://172.x.x.x:9093
>>> > > > >
>>> > > > > advertised.listeners=INTERNAL://1.x.x.x:9092,EXTERNAL://172.
>>> > x.x.x:9093
>>> > > > >
>>> > > > > listener.security.protocol.map=INTERNAL:SSL,EXTERNAL:SSL
>>> > > > >
>>> > > > > inter.broker.listener.name=INTERNAL
>>> > > > >
>>> > > > > On Wed, May 31, 2017 at 6:22 PM, Raghav <raghavas...@gmail.com>
>>> > wrote:
>>> > > > >
>>> > > > > > Hello Darshan
>>> > > > > >
>>> > > > > > Have you tried SSL://0.0.0.0:9093 ?
>>> > > > > >
>>> > > > > > Rajani had suggested something similar to me a week back while
>>> I
>>> > was
>>> > > > > > trying to get a ACL based setup.
>>> > > > > >
>>> > > > > > Thanks.
>>> > > > > >
>>> > > > > > On Wed, May 31, 2017 at 8:58 AM, Darshan <
>>> > > purandare.dars...@gmail.com>
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > >> Hi
>>> > > > > >>
>>> > > > > >> Our Kafka broker has two IPs on two different interfaces.
>>> > > > > >>
>>> > > > > >> eth0 has 172.x.x.x for external leg
>>> > > > > >> eth1 has 1.x.x.x for internal leg
>>> > > > > >>
>>> > > > > >>
>>> > > > > >> Kafka Producer is on 172.x.x.x subnet, and Kafka Consumer is
>>> on
>>> > > > 1.x.x.x
>>> > > > > >> subnet.
>>> > > > > >>
>>> > > > > >> If we use advertised.listeners=SSL://172.x.x.x:9093, then
>>> > Producer
>>> > > > can
>>> > > > > >> producer the message, but Consumer cannot receive the message.
>>> > > > > >>
>>> > > > > >> What value should we use for advertised.listeners so that
>>> Producer
>>> > > can
>>> > > > > >> write and Consumers can read ?
>>> > > > > >>
>>> > > > > >> Thanks.
>>> > > > > >>
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > --
>>> > > > > > Raghav
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Reply via email to