Hello,
I am testing the producer to send a message from my local PC to a
remote Kafka server with 0.8.0 installed. However, I was only able to
create the log file with that specific topic but not able to write any
message to the log file.
Here is what the log from the kafka server:
[2013-01-11 10:43:59,787] INFO [KafkaApi-1] Auto creation of topic
topic_227 with 1 partitions and replication factor 1 is successful!
(kafka.server.KafkaApis)
[2013-01-11 10:43:59,789] INFO [TopicChangeListener on Controller 1]:
New topics: [Set(topic_227)], deleted topics: [Set()], new partition
replica assignment [Map([topic_227,0] -> List(1))]
(kafka.controller.PartitionStateMachine$TopicChangeListener)
[2013-01-11 10:43:59,789] INFO [Controller 1]: New topic creation
callback for [topic_227,0] (kafka.controller.KafkaController)
[2013-01-11 10:43:59,790] INFO [Controller 1]: New partition creation
callback for [topic_227,0] (kafka.controller.KafkaController)
.......
[2013-01-11 10:43:59,830] INFO [Log Manager on Broker 1] Created log
for topic topic_227 partition 0 in /opt/kafka-0.8.0/data/kafka-logs.
(kafka.log.LogManager)
[2013-01-11 10:43:59,831] WARN No previously checkpointed
highwatermark value found for topic topic_227 partition 0. Returning 0
as the highwatermark (kafka.server.HighwaterMarkCheckpoint)
[2013-01-11 10:43:59,831] INFO Replica Manager on Broker 1: Completed
the leader state transition for topic topic_227 partition 0
(kafka.server.ReplicaManager)
[2013-01-11 10:43:59,831] INFO Replica Manager on Broker 1: Completed
leader and isr request LeaderAndIsrRequest(0,0,,1000,Map((topic_227,0)
-> PartitionStateInfo(LeaderIsrAndControllerEpoch({
"ISR":"1","leader":"1","leaderEpoch":"0"
},6),1)),Set(id:1,host:kafka.icare.com,port:9092),6)
(kafka.server.ReplicaManager)
[2013-01-11 10:43:59,959] INFO Closing socket connection to
/173.162.***.** (kafka.network.Processor)
So at this point there is a folder created on the server in the /logs
directory named after the topic and partition: /logs/topic_227-0
And there are two files inside:
-rw-r--r--. 1 root root 10485760 Jan 11 10:43 00000000000000000000.index
-rw-r--r--. 1 root root 0 Jan 11 10:43 00000000000000000000.log
However, the content of the actual message was not written to the .log
file. I couldn't find any error in the kafka server log. However, from
my JBoss log I see the following error:
10:47:11,554 INFO [kafka.client.ClientUtils$]
(http-localhost-127.0.0.1-8080-2) Fetching metadata for topic
Set(topic_227)
10:47:11,568 INFO [kafka.producer.SyncProducer]
(http-localhost-127.0.0.1-8080-2) Connected to 208.39.***.**:9092 for
producing
10:47:11,591 INFO [kafka.producer.SyncProducer]
(http-localhost-127.0.0.1-8080-2) Disconnecting from
208.39.***.**:9092
10:47:11,592 WARN [kafka.producer.async.DefaultEventHandler]
(http-localhost-127.0.0.1-8080-2) Failed to send to broker -1 with
data Map([topic_227,0] ->
ByteBufferMessageSet(MessageAndOffset(Message(magic = 0, attributes =
0, crc = 818361588, key = java.nio.HeapByteBuffer[pos=0 lim=1 cap=51],
payload = java.nio.HeapByteBuffer[pos=0 lim=46 cap=46]),0)))
10:47:11,695 INFO [kafka.client.ClientUtils$]
(http-localhost-127.0.0.1-8080-2) Fetching metadata for topic
Set(topic_227)
10:47:11,710 INFO [kafka.producer.SyncProducer]
(http-localhost-127.0.0.1-8080-2) Connected to 208.39.105.14:9092 for
producing
10:47:11,742 INFO [kafka.producer.SyncProducer]
(http-localhost-127.0.0.1-8080-2) Disconnecting from
208.39.105.14:9092
10:47:11,885 ERROR [kafka.producer.SyncProducer]
(http-localhost-127.0.0.1-8080-2) Producer connection to
kafka.icare.com:9092 unsuccessful:
java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:30) [classes.jar:1.6.0_33]
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:512)
[classes.jar:1.6.0_33]
at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
at kafka.producer.SyncProducer.connect(SyncProducer.scala:139)
at
kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:154)
at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68)
The server ip is 208.39.***.** and the server name is kafka.icare.com.
I am not sure why it first connected to the server with the ip address
and successfully created the topic with default partition. Then it
tried to use the server name to connect and thus failed? I feel like I
might be missing some setup in config files but I am not sure exactly
what I should change. Can anyone point out why this error occurs and
why kafka first tried to connect to ip and then server name?
thanks!
Jason
