Hi,
How are you "adding" a line to the file?
I tested with this configuration and it works ok on debian.
Regards,
Botond
On Thu, 25 Jul 2013 10:42:25 -0400
Enrique Cadalso <enrique.cada...@blendedperspectives.com> wrote:
> Hi Botond,
>
> Thanks for the advice, I followed it and changed nxlog.conf to
>
> root@Debian607:~# cat /etc/nxlog/nxlog.conf
> > ########################################
> > # Global directives #
> > ########################################
> > User nxlog
> > Group nxlog
> > LogFile /var/log/nxlog/nxlog.log
> > LogLevel INFO
> > ########################################
> > # Inputs #
> > ########################################
> > <Input internal>
> > Module im_internal
> > </Input>
> >
> > <Input confluence>
> > Module im_file
> > File "*/var/log/test.log*"
> > SavePos TRUE
> > Exec $raw_event=$raw_event+'|server:'+hostname_fqdn()+'|';
> > </Input>
> >
> > ########################################
> > # Output #
> > ########################################
> >
> > <Output confluence_out>
> > Module om_tcp
> > Host localhost
> > Port 3516
> > </Output>
> > *<Output nxlog_out>
> > Module om_file
> > File "/var/log/nxlog/nxlog.out"
> > </Output>
> > *
> > ########################################
> > # Routes #
> > ########################################
> >
> > <Route confluence>
> > Path confluence => confluence_out, *nxlog_out*
> > </Route>
>
>
>
> However when adding a line "Test" to the */var/log/test.log *file I get in
> the */var/log/nxlog/nxlog.out* file the truncated line, first character
> missing. All is working in a vagrant vm with Debian 607, running in a Mac,
> just in case that matters.
>
> root@Debian607:~# tail /var/log/nxlog/nxlog.log
> > ...
> > 2013-07-25 14:32:07 INFO nxlog-ce-2.5.1089 started
> > 2013-07-25 14:32:07 INFO connecting to localhost:3516
> > root@Debian607:~# tail /var/log/nxlog/nxlog.out
> > *est*|server:Debian607.vagrantup.com|
> > root@Debian607:~#
>
>
>
> I also enabled maximum debug level on logstash side and this is the first
> notice of the event, with the message already truncated.
>
> {:timestamp=>"2013-07-25T14:17:01.773000+0000", :message=>"heartbeat",
> > :level=>:debug, :file=>
> > "/opt/logstash/logstash-1.1.13-flatjar.jar!/logstash/logging.rb", :line=>
> > "35", :method=>"debug"}
> > {:timestamp=>"2013-07-25T14:17:03.779000+0000", :message=>"heartbeat",
> > :level=>:debug, :file=>
> > "/opt/logstash/logstash-1.1.13-flatjar.jar!/logstash/logging.rb", :line=>
> > "35", :method=>"debug"}
> > {:timestamp=>"2013-07-25T14:17:04.139000+0000", :message=>"Received
> > newevent",
> > :source=>"tcp://127.0.0.1:46544/", :event=>#<LogStash::Event:0x20c885fe
> > @data={"@source"=>"tcp://127.0.0.1:46544/", "@tags"=>["nxlog"],
> > "@fields"=>{},
> > "@timestamp"=>"2013-07-25T14:17:04.135Z", "@source_host"=>"127.0.0.1",
> > "@source_path"=>"/", "@message"=>"*est*|server:Debian607.vagrantup.com|\n",
> > "@type"=>"confluence"}, @cancelled=false>, :level=>:debug, :file=>
> > "/opt/logstash/logstash-1.1.13-flatjar.jar!/logstash/logging.rb", :line=>
> > "35", :method=>"debug"}
> > {:timestamp=>"2013-07-25T14:17:04.151000+0000", :message=>"est|server:
> > Debian607.vagrantup.com|\n", :pattern=>"^20", :match=>false, :negate=>true,
> > :level=>:debug, :file=>
> > "/opt/logstash/logstash-1.1.13-flatjar.jar!/logstash/logging.rb", :line=>
> > "35", :method=>"debug"}
>
>
> Regards,
>
> Enrique.
>
> ---------- Forwarded message ----------
> From: Botond Botyanszki <b...@nxlog.org>
> Date: Thu, Jul 25, 2013 at 3:56 AM
> Subject: Re: [nxlog-ce-users] Truncated logs from nxlog
> To: nxlog-ce-users@lists.sourceforge.net
>
>
> Hi,
>
> I suggest changing the output to om_file or using tcpdump/wireshark to
> see what gets sent over the network.
>
> The problem is most likely on the receiver (logstash) side.
>
> Regards,
> Botond
>
>
> On Tue, 23 Jul 2013 17:55:35 -0400
> Enrique Cadalso <enrique.cada...@blendedperspectives.com> wrote:
>
> > Hi,
> >
> > I want to include the fqdn of the server sending logs in the raw_event. I
> > am using this configuration
> >
> > <Input confluence>
> > Module im_file
> > File "/var/log/test.log"
> > SavePos TRUE
> > Exec $raw_event=$raw_event+'|server:'+hostname_fqdn()+'|';
> > </Input>
> >
> > Sending the log to logstash in the same host for testing using
> >
> > <Output confluence_out>
> > Module om_tcp
> > Host localhost
> > Port 3516
> > </Output>
> >
> > And I am receiving the log truncated, missing first character of the log
> > line. Testing with "Test" string I receive
> >
> > {
> > "@source" => "tcp://127.0.0.1:49388/",
> > "@tags" => [
> > [0] "nxlog",
> > [1] "_jsonparsefailure",
> > [2] "multiline",
> > [3] "_grokparsefailure"
> > ],
> > "@fields" => {
> > "log_level_alert" => [
> > [0] "%{@fields.log_level}"
> > ]
> > },
> > "@timestamp" => "2013-07-23T21:11:59.945Z",
> > "@source_host" => "127.0.0.1",
> > "@source_path" => "/",
> > "@message" => "*est*|server:Debian607.vagrantup.com|\n",
> > "@type" => "confluence"
> > }
> >
> > What is truncating the first character of the log? Is there a better way
> to
> > include the fqdn in the event?
> >
> > Thanks
> >
> > Enrique.
> >
> >
> >
> >
> > Full nxlog.conf (in the sender)
> >
> > root@Debian607:~# cat /etc/nxlog/nxlog.conf
> > ########################################
> > # Global directives #
> > ########################################
> >
> > User nxlog
> > Group nxlog
> > LogFile /var/log/nxlog/nxlog.log
> > LogLevel INFO
> >
> > ########################################
> > # Inputs #
> > ########################################
> >
> > <Input internal>
> > Module im_internal
> > </Input>
> >
> >
> > <Input confluence>
> > Module im_file
> > File "/var/log/test.log"
> > SavePos TRUE
> > Exec $raw_event=$raw_event+'|server:'+hostname_fqdn()+'|';
> > </Input>
> >
> >
> > ########################################
> > # Output #
> > ########################################
> >
> >
> > <Output confluence_out>
> > Module om_tcp
> > Host localhost
> > Port 3516
> > </Output>
> >
> >
> > ########################################
> > # Routes #
> > ########################################
> >
> >
> > <Route confluence>
> > Path confluence => confluence_out
> > </Route>
> >
> >
> -------------------------------------------------------------------------------------------------
> > Full logstash.conf (in the receiver)
> >
> >
> > input {
> > tcp {
> > type => "confluence"
> > port => 3516
> > format => 'json'
> > tags => ["nxlog"]
> > }
> > }
> >
> > filter {
> > multiline {
> > type => "confluence"
> > pattern => "^20"
> > negate => true
> > what => "previous"
> > add_tag => ["multiline_confluence"]
> > }
> >
> > grok {
> > type => "confluence"
> > pattern => "%{DATESTAMP},%{NUMBER} %{LOGLEVEL:log_level}
> > \[%{JAVAFILE:thread}\] \[%{JAVACLASS:class}\]"
> > add_tag => ["matched_confluence"]
> > }
> >
> > }
> >
> > output {
> > stdout {
> > type => "confluence"
> > debug => true
> > }
> > }
>
> ------------------------------------------------------------------------------
> See everything from the browser to the database with AppDynamics
> Get end-to-end visibility with application monitoring from AppDynamics
> Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
> _______________________________________________
> nxlog-ce-users mailing list
> nxlog-ce-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
