Are you sure your string is in utf-8 in the first place? What if you pass your string via something like:
System.out.println( new String( args[0].getBytes(StandardCharsets.UTF8), StandardCharsets.UTF8) ) – Best regards, Radek Gruchalski [email protected] On November 9, 2016 at 12:14:03 PM, Baris Akgun (Garanti Teknoloji) ( [email protected]) wrote: Hi, Producer Side// Properties props = *new* Properties(); props.put("metadata.broker.list", brokerList); props.put("serializer.class", “kafka.serializer.StringEncoder”); props.put("request.required.acks", "1"); Consumer side// I am using Spark Streaming Kafka API, I also try with Kafka CLI and Java kafka api but I always face with same issue. Thanks *From:* Radoslaw Gruchalski [mailto:[email protected]] *Sent:* Wednesday, November 9, 2016 1:49 PM *To:* Baris Akgun (Garanti Teknoloji); [email protected] *Subject:* Re: Kafka UTF 8 encoding problem Baris, Kafka does not care about encoding, everything is transported as bytes. What’s the configueration of your producer / consumer? Are you using Java / JVM? – Best regards, Radek Gruchalski [email protected] On November 9, 2016 at 11:42:02 AM, Baris Akgun (Garanti Teknoloji) ( [email protected]) wrote: Hi All, We are using Kafka 0,9.0.0 and we want to send our messages to topic in UTF-8 format but when we consume the messages from topic we saw that kafka does not keep the original utf-8 format and we did not see the messages exactly. For example our message that includes turkish characters is "Barış" but when we consume it we saw Bar?? . How can we solve that problem? Is there any way to set kafka topic encoding? Thanks Barış Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz. This message and attachments are confidential and intended solely for the individual(s) stated in this message. If you received this message although you are not the addressee, you are responsible to keep the message confidential. The sender has no responsibility for the accuracy or correctness of the information in the message and its attachments. Our company shall have no liability for any changes or late receiving, loss of integrity and confidentiality, viruses and any damages caused in anyway to your computer system. Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz. This message and attachments are confidential and intended solely for the individual(s) stated in this message. If you received this message although you are not the addressee, you are responsible to keep the message confidential. The sender has no responsibility for the accuracy or correctness of the information in the message and its attachments. Our company shall have no liability for any changes or late receiving, loss of integrity and confidentiality, viruses and any damages caused in anyway to your computer system.
