*graylog2 didn't recognize gelf output from logstash，*
*and still cann't find the way to fix it 。*
*but when i try nc test direct to graylog2 ，everything is ok。 just not work
when get from logstash gelf output*
*when i trying debug logstash，it looks like the gelf output format （it's
should be ： not ＝>）*
*when i trying tcpdump compare nc and logstash， the message show from the
tcp stream totally not recognize but it's ok for nc test！*
*here is the some detail，**really appreciate for any suggestion*
*a: *
logstash(gelf output)--------->--------graylog2(gelf udp input)
*b: log*
55.3.244.1 GET /index.html 15824 0.043
*c: logstash conf*
input {
file {
path => "/etc/logstash/ces"
}
}
filter {
grok {
match => { "message" => "%{IP:client} %{WORD:method}
%{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}"}
}
}
output {
gelf {
host => "192.168.88.109"
codec => json
custom_fields => ["environment", "uat"]
}
stdout { codec => json }
}
*d: logstash debug message （Version : 2.1.1）*
_globbed_files: /etc/logstash/ces: glob is: ["/etc/logstash/ces"]
{:level=>:debug, :file=>"filewatch/watch.rb", :line=>"190",
:method=>"_globbed_files"}
_globbed_files: /etc/logstash/ces: glob is: ["/etc/logstash/ces"]
{:level=>:debug, :file=>"filewatch/watch.rb", :line=>"190",
:method=>"_globbed_files"}
_globbed_files: /etc/logstash/ces: glob is: ["/etc/logstash/ces"]
{:level=>:debug, :file=>"filewatch/watch.rb", :line=>"190",
:method=>"_globbed_files"}
_globbed_files: /etc/logstash/ces: glob is: ["/etc/logstash/ces"]
{:level=>:debug, :file=>"filewatch/watch.rb", :line=>"190",
:method=>"_globbed_files"}
/etc/logstash/ces: file grew, old size 1239918, new size 1239957
{:level=>:debug, :file=>"filewatch/watch.rb", :line=>"121", :method=>"each"}
Received line {:path=>"/etc/logstash/ces", :text=>"55.3.244.1 GET
/index.html 15824 0.043", :level=>:debug, :file=>"logstash/inputs/file.rb",
:line=>"207", :method=>"log_line_received"}
filter received {:event=>{"message"=>"55.3.244.1 GET /index.html 15824
0.043", "@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces"}, :level=>:debug,
:file=>"(eval)", :line=>"44", :method=>"filter_func"}
writing sincedb (delta since last write = 369) {:level=>:debug,
:file=>"filewatch/tail.rb", :line=>"195", :method=>"_read_file"}
Running grok filter {:event=>#<LogStash::Event:0x397aec42
@metadata={"path"=>"/etc/logstash/ces"},
@accessors=#<LogStash::Util::Accessors:0x1cfaefa5
@store={"message"=>"55.3.244.1 GET /index.html 15824 0.043",
"@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces"},
@lut={"host"=>[{"message"=>"55.3.244.1 GET /index.html 15824 0.043",
"@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces"}, "host"],
"path"=>[{"message"=>"55.3.244.1 GET /index.html 15824 0.043",
"@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces"}, "path"]}>,
@data={"message"=>"55.3.244.1 GET /index.html 15824 0.043",
"@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces"},
@metadata_accessors=#<LogStash::Util::Accessors:0x4542ca0
@store={"path"=>"/etc/logstash/ces"},
@lut={"[path]"=>[{"path"=>"/etc/logstash/ces"}, "path"]}>,
@cancelled=false>, :level=>:debug, :file=>"logstash/filters/grok.rb",
:line=>"283", :method=>"filter"}
Regexp match object {:names=>["IP:client", "WORD:method",
"URIPATHPARAM:request", "NUMBER:bytes", "NUMBER:duration"],
:captures=>["55.3.244.1", "GET", "/index.html", "15824", "0.043"],
:level=>:debug, :file=>"grok-pure.rb", :line=>"179",
:method=>"match_and_capture"}
Event now: {:event=>#<LogStash::Event:0x397aec42
@metadata={"path"=>"/etc/logstash/ces"},
@accessors=#<LogStash::Util::Accessors:0x1cfaefa5
@store={"message"=>"55.3.244.1 GET /index.html 15824 0.043",
"@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1",
"method"=>"GET", "request"=>"/index.html", "bytes"=>"15824",
"duration"=>"0.043"}, @lut={"host"=>[{"message"=>"55.3.244.1 GET
/index.html 15824 0.043", "@version"=>"1",
"@timestamp"=>"2014-08-17T22:11:13.806Z", "host"=>"gz3.test",
"path"=>"/etc/logstash/ces", "client"=>"55.3.244.1", "method"=>"GET",
"request"=>"/index.html", "bytes"=>"15824", "duration"=>"0.043"}, "host"],
"path"=>[{"message"=>"55.3.244.1 GET /index.html 15824 0.043",
"@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1",
"method"=>"GET", "request"=>"/index.html", "bytes"=>"15824",
"duration"=>"0.043"}, "path"], "message"=>[{"message"=>"55.3.244.1 GET
/index.html 15824 0.043", "@version"=>"1",
"@timestamp"=>"2014-08-17T22:11:13.806Z", "host"=>"gz3.test",
"path"=>"/etc/logstash/ces", "client"=>"55.3.244.1", "method"=>"GET",
"request"=>"/index.html", "bytes"=>"15824", "duration"=>"0.043"},
"message"], "client"=>[{"message"=>"55.3.244.1 GET /index.html 15824
0.043", "@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1",
"method"=>"GET", "request"=>"/index.html", "bytes"=>"15824",
"duration"=>"0.043"}, "client"], "method"=>[{"message"=>"55.3.244.1 GET
/index.html 15824 0.043", "@version"=>"1",
"@timestamp"=>"2014-08-17T22:11:13.806Z", "host"=>"gz3.test",
"path"=>"/etc/logstash/ces", "client"=>"55.3.244.1", "method"=>"GET",
"request"=>"/index.html", "bytes"=>"15824", "duration"=>"0.043"},
"method"], "request"=>[{"message"=>"55.3.244.1 GET /index.html 15824
0.043", "@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1",
"method"=>"GET", "request"=>"/index.html", "bytes"=>"15824",
"duration"=>"0.043"}, "request"], "bytes"=>[{"message"=>"55.3.244.1 GET
/index.html 15824 0.043", "@version"=>"1",
"@timestamp"=>"2014-08-17T22:11:13.806Z", "host"=>"gz3.test",
"path"=>"/etc/logstash/ces", "client"=>"55.3.244.1", "method"=>"GET",
"request"=>"/index.html", "bytes"=>"15824", "duration"=>"0.043"}, "bytes"],
"duration"=>[{"message"=>"55.3.244.1 GET /index.html 15824 0.043",
"@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1",
"method"=>"GET", "request"=>"/index.html", "bytes"=>"15824",
"duration"=>"0.043"}, "duration"]}>, @data={"message"=>"55.3.244.1 GET
/index.html 15824 0.043", "@version"=>"1",
"@timestamp"=>"2014-08-17T22:11:13.806Z", "host"=>"gz3.test",
"path"=>"/etc/logstash/ces", "client"=>"55.3.244.1", "method"=>"GET",
"request"=>"/index.html", "bytes"=>"15824", "duration"=>"0.043"},
@metadata_accessors=#<LogStash::Util::Accessors:0x4542ca0
@store={"path"=>"/etc/logstash/ces"},
@lut={"[path]"=>[{"path"=>"/etc/logstash/ces"}, "path"]}>,
@cancelled=false>, :level=>:debug, :file=>"logstash/filters/grok.rb",
:line=>"303", :method=>"filter"}
output received {:event=>{"message"=>"55.3.244.1 GET /index.html 15824
0.043", "@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1",
"method"=>"GET", "request"=>"/index.html", "bytes"=>"15824",
"duration"=>"0.043"}, :level=>:debug, :file=>"(eval)", :line=>"50",
:method=>"output_func"}
["Sending GELF event", {"short_message"=>"55.3.244.1 GET /index.html 15824
0.043", "full_message"=>"55.3.244.1 GET /index.html 15824 0.043",
"host"=>"gz3.test", "_path"=>"/etc/logstash/ces", "_client"=>"55.3.244.1",
"_method"=>"GET", "_request"=>"/index.html", "_bytes"=>"15824",
"_duration"=>"0.043", "_environment"=>"uat", "level"=>6}] {:level=>:debug,
:file=>"logstash/outputs/gelf.rb", :line=>"212", :method=>"receive"}
{"message":"55.3.244.1 GET /index.html 15824
0.043","@version":"1","@timestamp":"2014-08-17T22:11:13.806Z","host":"gz3.test","path":"/etc/logstash/ces","client":"55.3.244.1","method":"GET","request":"/index.html","bytes":"15824","duration":"0.043"}_globbed_files:
/etc/logstash/ces: glob is: ["/etc/logstash/ces"] {:level=>:debug,
:file=>"filewatch/watch.rb", :line=>"190", :method=>"_globbed_files"}
_globbed_files: /etc/logstash/ces: glob is: ["/etc/logstash/ces"]
{:level=>:debug, :file=>"filewatch/watch.rb", :line=>"190",
:method=>"_globbed_files"}
_globbed_files: /etc/logstash/ces: glob is: ["/etc/logstash/ces"]
{:level=>:debug, :file=>"filewatch/watch.rb", :line=>"190",
:method=>"_globbed_files"}
*tcpdump message when test different way*
*1: use nc test （ok）*
echo -e '{"message":"55.3.244.1 GET /index.html 15824
0.043","@version":"1","@timestamp":"2014-08-17T19:36:45.825Z","host":"gz3.test","path":"/etc/logstash/ces","client":"55.3.244.1","method":"GET","request":"/index.html","bytes":"15824","duration":"0.043"}'
| nc -u 192.168.88.109 12201
0.000000 192.168.88.64 192.168.88.109 UDP 291 Source port: 34983
Destination port: 12201
{"message":"55.3.244.1 GET /index.html 15824
0.043","@version":"1","@timestamp":"2014-08-17T19:36:45.825Z","host":"gz3.test","path":"/etc/logstash/ces","client":"55.3.244.1","method":"GET","request":"/index.html","bytes":"15824","duration":"0.043"}
*2: use logstash test （not ok）*
[root@gz3 logstash]# cat c
55.3.244.1 GET /index.html 15824 0.043
[root@gz3 logstash]# cat c >> ces
[root@gz3 logstash]#
32.343310 192.168.88.64 192.168.88.109 UDP 261 Source port: 34000
Destination port: 12201
x..P.N.0.......k....?.{.&.......(..gM%.Wn.3;3..`;..Rd#.\...T...]q..[.a.....-fr......C.......W6J-.%.~V|.t_l.x&...H..\...\6./..sq.7n1.F..Q7....s.....|5hI..N.O+.Ul7A....D.cW..h.L0Y.=.m...~.T.r.BjO5....
>..y...r.....d._..i.
*3: use nc test （not ok）*
echo -e '{"message"=>"55.3.244.1 GET /index.html 15824 0.043",
"@version"=>"1", "@timestamp"=>"2014-08-17T22:11:13.806Z",
"host"=>"gz3.test", "path"=>"/etc/logstash/ces", "client"=>"55.3.244.1",
"method"=>"GET", "request"=>"/index.html", "bytes"=>"15824",
"duration"=>"0.043"}' | nc -u 192.168.88.109 12201
208.101006 192.168.88.64 192.168.88.109 UDP 310 Source port: 41607
Destination port: 12201
{"message"=>"55.3.244.1 GET /index.html 15824 0.043", "@version"=>"1",
"@timestamp"=>"2014-08-17T22:11:13.806Z", "host"=>"gz3.test",
"path"=>"/etc/logstash/ces", "client"=>"55.3.244.1", "method"=>"GET",
"request"=>"/index.html", "bytes"=>"15824", "duration"=>"0.043"}
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/0af2bbfc-1fee-44ae-952e-4090913bf62d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
