[ https://issues.apache.org/jira/browse/HADOOP-14623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16079033#comment-16079033 ]
Hongyuan Li edited comment on HADOOP-14623 at 7/8/17 8:46 AM: -------------------------------------------------------------- I highly recommend Four points: 1、should use {{acks}} = {{1}}. 2、add {{https://repository.apache.org/content/repositories/releases}} repo, the {{apache snapshot rep}} doesnot have a higher version kafka module, the version of which is less than {{0.8.2}} 3、update kafka client version to at least {{0.10.1.0}}, which has a IntegerSerializer class If kafka sink want to generate a kafka producer with the the type of key being Integer. 4、add {{callback}} when using new {{KafkaProducer}}#{{send}} Thanks for any advice. was (Author: hongyuan li): I highly recommend Two points: 1、should use {{acks}} = {{1}}. 2、add {{https://repository.apache.org/content/repositories/releases}} repo, the {{apache snapshot rep}} doesnot have a higher version kafka module, the version of which is less than {{0.8.2}} 2、update kafka client version to at least {{0.10.1.0}}, which has a IntegerSerializer class If kafka sink want to generate a kafka producer with the the type of key being Integer. 3、add {{callback}} when using new {{KafkaProducer}}#{{send}} Thanks for any advice. > fixed some bugs in KafkaSink > ----------------------------- > > Key: HADOOP-14623 > URL: https://issues.apache.org/jira/browse/HADOOP-14623 > Project: Hadoop Common > Issue Type: Bug > Components: common, tools > Affects Versions: 3.0.0-alpha3 > Reporter: Hongyuan Li > Assignee: Hongyuan Li > Attachments: HADOOP-14623-001.patch > > > {{KafkaSink}}#{{init}} should set ack to *1* to make sure the message has > been written to the broker at least. > current code list below: > {code} > > props.put("request.required.acks", "0"); > {code} > *Update* > find another bug about this class, {{key.serializer}} used > {{org.apache.kafka.common.serialization.ByteArraySerializer}}, however, the > key properties of Producer is Integer, codes list below: > {code} > props.put("key.serializer", > "org.apache.kafka.common.serialization.ByteArraySerializer"); > …………… > producer = new KafkaProducer<Integer, byte[]>(props); > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org