[
https://issues.apache.org/jira/browse/FLUME-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14958902#comment-14958902
]
Alexander Bij edited comment on FLUME-2792 at 10/19/15 8:49 AM:
----------------------------------------------------------------
We are using HDP stack 2.3 with simular setup. Kerberos and Ranger up and
running.
I can consume messages from a topic, but ingesting from Flume does not work.
We have a sink to HDFS using the flume keytab from /etc/security/keytabs/.
This keytab file + pricipal is a setting in the HDFSSink!
I want to use the KafkaSink write to Kerberos secured Kafka-Cluster.
(PLAINTEXTSASL)
I tried the setting you suggested at point 1. Unfortunately the setting is not
used.
log: flume[agent].log
{noformat}
15 Oct 2015 15:08:55,268 WARN [lifecycleSupervisor-1-8]
(kafka.utils.Logging$class.warn:83) - Property security.protocol is not valid
{noformat}
Flume tries to connect with KafkaBroker, there I get error; GSSHeader dit not
find right tag. I have the feeling its not sending with SASL.
Do you have other ideas?
kafka-broker.log:
{noformat}
[2015-10-15 14:50:34,142] ERROR Closing socket for /10.xxxx3.19 because of
error (kafka.network.Processor)
java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed
[Caused by GSSException: Defective token detected (Mechanism level: GSSHeader
did not find the right tag)]
at
org.apache.kafka.common.network.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:190)
at org.apache.kafka.common.network.Channel.connect(Channel.java:71)
at kafka.network.Processor.handshake(SocketServer.scala:520)
at kafka.network.Processor.run(SocketServer.scala:409)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: Defective token detected (Mechanism level: GSSHeader did not find
the right tag)]
at
com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:177)
at
org.apache.kafka.common.network.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:180)
... 4 more
Caused by: GSSException: Defective token detected (Mechanism level: GSSHeader
did not find the right tag)
at sun.security.jgss.GSSHeader.<init>(GSSHeader.java:97)
at
sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:306)
at
sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
at
com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:155)
... 5 more
{noformat}
was (Author: abij):
We are using HDP stack 2.3 with simular setup. Kerberos and Ranger up and
running.
I can consume messages from a topic, but ingesting from Flume does not work.
We have a sink to HDFS using the flume keytab from /etc/security/keytabs/.
This keytab file + pricipal is a setting in the HDFSSink!
I want to use the KafkaSink write to Kerberos secured Kafka-Cluster.
(PLAINTEXTSASL)
I tried the setting you suggested at point 1. Unfortunately the setting is not
used.
log: flume-[agent].log
15 Oct 2015 15:08:55,268 WARN [lifecycleSupervisor-1-8]
(kafka.utils.Logging$class.warn:83) - Property security.protocol is not valid
Flume tries to connect with KafkaBroker, there I get error; GSSHeader dit not
find right tag. I have the feeling its not sending with SASL.
Do you have other ideas?
kafka-broker.log:
[2015-10-15 14:50:34,142] ERROR Closing socket for /10.xxxx3.19 because of
error (kafka.network.Processor)
java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed
[Caused by GSSException: Defective token detected (Mechanism level: GSSHeader
did not find the right tag)]
at
org.apache.kafka.common.network.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:190)
at org.apache.kafka.common.network.Channel.connect(Channel.java:71)
at kafka.network.Processor.handshake(SocketServer.scala:520)
at kafka.network.Processor.run(SocketServer.scala:409)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: Defective token detected (Mechanism level: GSSHeader did not find
the right tag)]
at
com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:177)
at
org.apache.kafka.common.network.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:180)
... 4 more
Caused by: GSSException: Defective token detected (Mechanism level: GSSHeader
did not find the right tag)
at sun.security.jgss.GSSHeader.<init>(GSSHeader.java:97)
at
sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:306)
at
sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
at
com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:155)
... 5 more
> Flume Kafka Kerberos Support
> ----------------------------
>
> Key: FLUME-2792
> URL: https://issues.apache.org/jira/browse/FLUME-2792
> Project: Flume
> Issue Type: Bug
> Components: Configuration, Docs, Sinks+Sources
> Affects Versions: v1.6.0, v1.5.2
> Environment: HDP 2.3 fully kerberized including Kafka 0.8.2.2 + Flume
> 1.5.2 or Apache Flume 1.6 downloaded from apache.org
> Reporter: Hari Sekhon
> Priority: Blocker
>
> Following on from FLUME-2790 it appears as though Flume doesn't yet have
> support for Kafka + Kerberos as there are is no setting documented in the
> Flume 1.6.0 user guide under the Kafka source section to tell Flume to use
> plaintextsasl as the connection mechanism to Kafka and Kafka rejects
> unauthenticated plaintext mechanism:
> {code}15/09/10 16:51:22 INFO consumer.ConsumerFetcherManager:
> [ConsumerFetcherManager-1441903874830] Added fetcher for partitions
> ArrayBuffer()
> 15/09/10 16:51:22 WARN consumer.ConsumerFetcherManager$LeaderFinderThread:
> [flume_<custom_scrubbed>-1441903874763-abdc98ec-leader-finder-thread], Failed
> to find leader for Set([<custom_scrubbed>,0], [<custom_scrubbed>,1])
> kafka.common.BrokerEndPointNotAvailableException: End point PLAINTEXT not
> found for broker 0
> at kafka.cluster.Broker.getBrokerEndPoint(Broker.scala:140)
> at
> kafka.utils.ZkUtils$$anonfun$getAllBrokerEndPointsForChannel$1.apply(ZkUtils.scala:124)
> at
> kafka.utils.ZkUtils$$anonfun$getAllBrokerEndPointsForChannel$1.apply(ZkUtils.scala:124)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
> at scala.collection.AbstractTraversable.map(Traversable.scala:105)
> at
> kafka.utils.ZkUtils$.getAllBrokerEndPointsForChannel(ZkUtils.scala:124)
> at
> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
> at
> kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
