Thanks Liam, I'm trying with Filebeat but i'm unsure what filter should i use for kafka logs. I need all the controller.log, server.log and state-change.log to be shipped. I tried using grok filter but that is throwing error. Do you have any info on the filter to be used for Kafka logs?
Regards On Tue, Sep 24, 2019 at 1:11 PM Liam Clarke <liam.cla...@adscale.co.nz> wrote: > Cc list > > ---------- Forwarded message --------- > From: Liam Clarke <liam.cla...@adscale.co.nz> > Date: Tue, 24 Sep. 2019, 7:40 pm > Subject: Re: Shipping Kafka logs > To: Eva Sheeva <sheevae...@gmail.com> > > > Ah, for that you need something like Filebeat - it's an agent that forwards > monitored log files to ELK. > > On Tue, 24 Sep. 2019, 7:17 pm Eva Sheeva, <sheevae...@gmail.com> wrote: > > > Thank you Jacob and Liam! > > > > I 'm looking for the steps on shipping Kafka application logs. I want the > > controller.log, state-change.log and server.log to be shipped to elk. Can > > you pls help! > > > > Regards! > > > > On Tue, Sep 24, 2019 at 8:08 AM Botuck, Jacob (STL) < > > jbot...@express-scripts.com> wrote: > > > >> You can also try > >> > https://docs.confluent.io/current/connect/kafka-connect-elasticsearch/index.html > >> I think both links are about shipping kafka messages to elk, not actual > >> kafka application logs > >> > >> -----Original Message----- > >> From: Liam Clarke <liam.cla...@adscale.co.nz> > >> Sent: Monday, September 23, 2019 5:21 PM > >> To: users@kafka.apache.org > >> Subject: [EXTERNAL] Re: Shipping Kafka logs > >> > >> Hi Eva, > >> > >> Hope this helps. > >> <company email system killed this link> > >> Kind regards, > >> > >> Liam Clarke > >> > >> On Tue, Sep 24, 2019 at 2:57 AM Eva Sheeva <sheevae...@gmail.com> > wrote: > >> > >> > Hello > >> > > >> > Can you please point me to the doc or guide me on shipping kafka logs > >> > to elk. I understand it is a little different than how we ship other > >> > logs to elk. Kindly help me with this > >> > > >> > Thanks > >> > > >> > > >
