[
https://issues.apache.org/jira/browse/FLUME-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lior Zeno resolved FLUME-2651.
------------------------------
Resolution: Not A Problem
> Connection problem for KAFKA via ZOOKEEPER
> ------------------------------------------
>
> Key: FLUME-2651
> URL: https://issues.apache.org/jira/browse/FLUME-2651
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: cloudera 5.3
> Reporter: Serdar Mustaoglu
> Fix For: v0.9.5
>
>
> When I try to get messages from kafka into hdfs file via FLUME, I have
> connections problems.
> metadata.broker.list must be used , props.put("zk.connect", "node2:2181")
> not supported.
> How can connect to kafka via zookeper.
> tier1.sources = source1
> tier1.channels = channel1
> tier1.sinks = sink1
> tier1.sources.source1.type = org.apache.flume.source.kafka.KafkaSource
> tier1.sources.source1.zookeeperConnect = quickstart.cloudera:2181
> tier1.sources.source1.topic = topic1
> tier1.sources.source1.groupId = flume
> tier1.sources.source1.channels = channel1
> tier1.sources.source1.interceptors = i1
> tier1.sources.source1.interceptors.i1.type = timestamp
> tier1.sources.source1.kafka.consumer.timeout.ms = 100
> tier1.channels.channel1.type = memory
> tier1.channels.channel1.capacity = 10000
> tier1.channels.channel1.transactionCapacity = 1000
> tier1.sinks.sink1.type = hdfs
> tier1.sinks.sink1.hdfs.path = /user/cloudera/kafka/%{topic}/%y-%m-%d
> tier1.sinks.sink1.hdfs.rollInterval = 5
> tier1.sinks.sink1.hdfs.rollSize = 0
> tier1.sinks.sink1.hdfs.rollCount = 0
> tier1.sinks.sink1.hdfs.fileType = DataStream
> tier1.sinks.sink1.channel = channel1
>
> ----
> actually, how can configure kafka listening port ?
> java
> Properties props = new Properties();
> ///props.put("zk.connect", "192.168.6.129:2181"); - not working
> reqires metadata.broker.list
> props.put("metadata.broker.list", "192.168.6.129:2181");
> props.put("serializer.class", "kafka.serializer.StringEncoder");
>
> ProducerConfig config = new ProducerConfig(props);
> producer = new Producer<String, String>(config);
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
