Manjeet Duhan created KAFKA-7751: ------------------------------------ Summary: Kryo convertor in kafka connect Key: KAFKA-7751 URL: https://issues.apache.org/jira/browse/KAFKA-7751 Project: Kafka Issue Type: Improvement Reporter: Manjeet Duhan
We had earlier SFDC logs project which was serializing complete request and response data in file store and our support team was deserializing it on demand whenever there was any issue in production. Approach :- We wanted to send data to elasticsearch via kafka connect to make it search optimistic. Problem :- data size was high ~1GB so we wanted to save some space using any available compression Technics. At the same time , we wanted to utilize our confluent infrastructure. Solution :- I have written Kryo serializer and converter for same to send and receive data into kafka connect and send it to elasticsearch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)