Re: Kafka Producer for Sending Log Data

2017-03-20 Thread Ram Vittal
Hi Tony, We are using custom logging API that wraps Kafka producer as singleton for each app. The custom API takes structured log data and converts to JSON, writes to app specific Kafka topic. Then that topic is bridged to Logstash consumer and logs get ingested to elastic search. Now these lo

Kafka Producer for Sending Log Data

2017-03-20 Thread Tony S. Wu
Hi, I am looking to send log file periodically to Kafka. Before I set out to write my own producer, I was wonder what everyone uses to send log file to Kafka through HTTP API. Ideally it should also prune the log and have some sort of error handling. Thanks very much. Tony S. Wu