Hi Cameron and Botond Thanks for the advice here, this does make sense! Will test it out tomorrow and check the results.
Regards Daniel Sent from my iPhone > On 30 Sep 2014, at 19:40, "Botond Botyanszki" <b...@nxlog.org> wrote: > > Hi, > > Cameron is correct. You should first verify that nxlog can parse the > multiline by writing a to a file after calling to_json(). > > Since you are sending to logstash using om_tcp without any encapsulation, > logstash will treat each line in your multiline event as a separate > record, i.e. your multiline magic becomes effectively useless when it > reaches logstash. > > Regards, > Botond > > On Wed, 1 Oct 2014 07:31:07 +1300 > Cameron Kerr <cameron.kerr...@gmail.com> wrote: > >> Youshould convert the format to something like JSON, and the on the LogStash >> receiver you read it in using the json_lines codec. >> >> Sent from my iPhone >> >>> On 1/10/2014, at 5:35 am, Daniel Zorab <daniel.zo...@derivco.co.uk> wrote: >>> >>> Hi >>> >>> I am attempting to create a proof of concept for visualizing log files by >>> leveraging nxlog -> logstash -> elasticsearch -> kibana. I am having issues >>> at the stage of filling up logtstash with information from a text log file >>> using nxlog and in particular the multiline portions of the log file. >>> >>> I am using a Windows 7 x64 VM as a test machine with all the services and >>> applications localized to this VM. >>> >>> I have been able to send log file (single line) entries successfully from >>> the log file using nxlog through to logstash which then parses and stores >>> each event as it should. The issue comes in when attempting to support/send >>> multline entries through to logstash which is producing some inconsistent >>> results. (If I send through each as a single line it works fine but >>> obviously the multiline log entries get stored into their own event) >>> .Basically it looks like it is sending several single line entries and >>> logstash is viewing them as one single event (as per debugoutput.txt eg: >>> line 77 ->154 ) which can be compared against the input of SampleLog.txt >>> >>> If using logstash on its own (no nxlog involvement) it is able to parse and >>> read multiline and single line inputs absoluytely fine as per the and the >>> logtash config file : Logstash.conf. >>> I have tried out multiple scenarios in the nxlog config by >>> enabling/disabling the xm_multiline module and utlising the HeaderLine and >>> EndLine. I have also tried disabling the multline config portion of the >>> logtstash unto no avail. >>> >>> Could anyone shed some more light on this issue or have I misunderstood how >>> to utilise the config for nxlog? >>> >>> Regards >>> Daniel >>> <Logstash.conf.txt> >>> <nxlog.conf.txt> >>> <rubydebug output.txt> >>> <SampleLog.txt> >>> ------------------------------------------------------------------------------ >>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports >>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> nxlog-ce-users mailing list >>> nxlog-ce-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users > > ------------------------------------------------------------------------------ > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer > http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk > _______________________________________________ > nxlog-ce-users mailing list > nxlog-ce-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users ------------------------------------------------------------------------------ Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk _______________________________________________ nxlog-ce-users mailing list nxlog-ce-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
