Hari Sekhon created FLUME-2792:
----------------------------------
Summary: Flume Kafka Kerberos Support
Key: FLUME-2792
URL: https://issues.apache.org/jira/browse/FLUME-2792
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: 1.6, v1.6.0
Environment: HDP 2.3 fully kerberized including Kafka 0.8.2.2 + Apache
Flume 1.6 downloaded from apache.org
Reporter: Hari Sekhon
Priority: Blocker
Following on from FLUME-2790 it appears as though Flume doesn't yet have
support for Kafka + Kerberos as there are is no setting documented in the Flume
1.6.0 user guide under the Kafka source section to tell Flume to use
plaintextsasl as the connection mechanism to Kafka and Kafka rejects
unauthenticated plaintext mechanism:
{code}15/09/10 16:51:22 INFO consumer.ConsumerFetcherManager:
[ConsumerFetcherManager-1441903874830] Added fetcher for partitions
ArrayBuffer()
15/09/10 16:51:22 WARN consumer.ConsumerFetcherManager$LeaderFinderThread:
[flume_<custom_scrubbed>-1441903874763-abdc98ec-leader-finder-thread], Failed
to find leader for Set([<custom_scrubbed>,0], [<custom_scrubbed>,1])
kafka.common.BrokerEndPointNotAvailableException: End point PLAINTEXT not found
for broker 0
at kafka.cluster.Broker.getBrokerEndPoint(Broker.scala:140)
at
kafka.utils.ZkUtils$$anonfun$getAllBrokerEndPointsForChannel$1.apply(ZkUtils.scala:124)
at
kafka.utils.ZkUtils$$anonfun$getAllBrokerEndPointsForChannel$1.apply(ZkUtils.scala:124)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at
kafka.utils.ZkUtils$.getAllBrokerEndPointsForChannel(ZkUtils.scala:124)
at
kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
