Issues while setting up RBAC for Apache Kafka using Ranger
Hi All,
I'm working on setting up RBAC for Apache Kafka using Ranger. Right now,
I'm facing an authorization issue while testing the console producer script
in Kafka. I need help in properly configuring Kafka with Ranger. Below are
the steps I performed.
- I successfully installed the ranger service.
- Integrated Ranger with AD using UserSync.
- Installed Ranger Kafka Plugin on Kafka and made the following changes
to Kafka server.properties file
- *authorizer.class.name
<http://authorizer.class.name/>=org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer*
- Created Kafka service in Ranger Admin
- Created a policy in ranger admin to restrict access to topic named
test for everyone except one user.
I'm using PLAINTEXT://HOSTIP:PORT for listeners.
Now, when I try write to that topic using *./kafka-console-producer.sh
--broker-list hostip:port --topic test*
I'm unable to produce to it, and I'm getting authorization error messages.
which seems okay. But I don't know how to produce the topic with an
authorized user. I tried using a producer config file with the below config
*client.id <http://client.id/>=
testusersasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="testuser" password="testpass";*
Below is the output
*./kafka-console-producer.sh --broker-list * *hostip:port* * --topic test
--producer.config producer.properties*
[2024-03-08 16:54:09,034] WARN The configuration 'sasl.jaas.config' was
supplied but isn't a known config.
(org.apache.kafka.clients.producer.ProducerConfig)
>hi
[2024-03-08 16:54:15,309] WARN [Producer clientId= testuser] Error while
fetching metadata with correlation id 3 : {test=TOPIC_AUTHORIZATION_FAILED}
(org.apache.kafka.clients.NetworkClient)
[2024-03-08 16:54:15,321] ERROR [Producer clientId= testuser] Topic
authorization failed for topics [test] (org.apache.kafka.clients.Metadata)
[2024-03-08 16:54:15,325] ERROR Error when sending message to topic test
with key: null, value: 2 bytes with error:
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized
to access topics: [test]
Please provide steps to connect and produce to the topic with test user
(This user is from AD).
Regards,
*Karthik Suvarnasa*
Re: Authorization Issues while implementing Kafka RBAC using Ranger
++ Ranger Team
Regards,
*Karthik Suvarnasa*
EPIC Engineering & Consulting Group, LLC
1049 Willa Springs Drive, Ste. 1001, Winter Springs, FL 32708
(cell) 860-776-7951 | (work) 407-381-3742
Web: www.epicgroupllc.com
On Fri, Mar 8, 2024 at 12:10 PM Karthik Suvarnasa
wrote:
> Hi All,
>
> I'm working on setting up RBAC for Apache Kafka using Ranger. Right now,
> I'm facing an authorization issue while testing the console producer script
> in Kafka. I need help in properly configuring Kafka with Ranger. Below are
> the steps I performed.
>
>
>- I successfully installed the ranger service.
>- Integrated Ranger with AD using UserSync.
>- Installed Ranger Kafka Plugin on Kafka and made the following
>changes to Kafka server.properties file
> - *authorizer.class.name
>
> <http://authorizer.class.name>=org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer*
>- Created Kafka service in Ranger Admin
>- Created a policy in ranger admin to restrict access to topic named
>test for everyone except one user.
>
> I'm using PLAINTEXT://HOSTIP:PORT for listeners.
>
> Now, when I try write to that topic using *./kafka-console-producer.sh
> --broker-list hostip:port --topic test*
>
> I'm unable to produce to it, and I'm getting authorization error messages.
> which seems okay. But I don't know how to produce the topic with an
> authorized user. I tried using a producer config file with the below config
>
>
> *client.id <http://client.id>=
> testusersasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
> required username="testuser" password="testpass";*
>
> Below is the output
> *./kafka-console-producer.sh --broker-list * *hostip:port* * --topic
> test --producer.config producer.properties*
>
> [2024-03-08 16:54:09,034] WARN The configuration 'sasl.jaas.config' was
> supplied but isn't a known config.
> (org.apache.kafka.clients.producer.ProducerConfig)
> >hi
> [2024-03-08 16:54:15,309] WARN [Producer clientId= testuser] Error while
> fetching metadata with correlation id 3 : {test=TOPIC_AUTHORIZATION_FAILED}
> (org.apache.kafka.clients.NetworkClient)
> [2024-03-08 16:54:15,321] ERROR [Producer clientId= testuser] Topic
> authorization failed for topics [test] (org.apache.kafka.clients.Metadata)
> [2024-03-08 16:54:15,325] ERROR Error when sending message to topic test
> with key: null, value: 2 bytes with error:
> (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
> org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized
> to access topics: [test]
>
> Please provide steps to connect and produce to the topic with test user
> (This user is from AD).
>
> Regards,
> *Karthik Suvarnasa*
> EPIC Engineering & Consulting Group, LLC
> 1049 Willa Springs Drive, Ste. 1001, Winter Springs, FL 32708
> (cell) 860-776-7951 | (work) 407-381-3742
> Web: www.epicgroupllc.com
>
>
Authorization Issues while implementing Kafka RBAC using Ranger
Hi All,
I'm working on setting up RBAC for Apache Kafka using Ranger. Right now,
I'm facing an authorization issue while testing the console producer script
in Kafka. I need help in properly configuring Kafka with Ranger. Below are
the steps I performed.
- I successfully installed the ranger service.
- Integrated Ranger with AD using UserSync.
- Installed Ranger Kafka Plugin on Kafka and made the following changes
to Kafka server.properties file
- *authorizer.class.name
<http://authorizer.class.name>=org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer*
- Created Kafka service in Ranger Admin
- Created a policy in ranger admin to restrict access to topic named
test for everyone except one user.
I'm using PLAINTEXT://HOSTIP:PORT for listeners.
Now, when I try write to that topic using *./kafka-console-producer.sh
--broker-list hostip:port --topic test*
I'm unable to produce to it, and I'm getting authorization error messages.
which seems okay. But I don't know how to produce the topic with an
authorized user. I tried using a producer config file with the below config
*client.id <http://client.id>=
testusersasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="testuser" password="testpass";*
Below is the output
*./kafka-console-producer.sh --broker-list * *hostip:port* * --topic test
--producer.config producer.properties*
[2024-03-08 16:54:09,034] WARN The configuration 'sasl.jaas.config' was
supplied but isn't a known config.
(org.apache.kafka.clients.producer.ProducerConfig)
>hi
[2024-03-08 16:54:15,309] WARN [Producer clientId= testuser] Error while
fetching metadata with correlation id 3 : {test=TOPIC_AUTHORIZATION_FAILED}
(org.apache.kafka.clients.NetworkClient)
[2024-03-08 16:54:15,321] ERROR [Producer clientId= testuser] Topic
authorization failed for topics [test] (org.apache.kafka.clients.Metadata)
[2024-03-08 16:54:15,325] ERROR Error when sending message to topic test
with key: null, value: 2 bytes with error:
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized
to access topics: [test]
Please provide steps to connect and produce to the topic with test user
(This user is from AD).
Regards,
*Karthik Suvarnasa*
EPIC Engineering & Consulting Group, LLC
1049 Willa Springs Drive, Ste. 1001, Winter Springs, FL 32708
(cell) 860-776-7951 | (work) 407-381-3742
Web: www.epicgroupllc.com
