I'm interested but is there java ruby dependencies,? sorry I'm not yet experienced with elasticsearch. On Feb 21, 2014 7:09 PM, "Eugene Istomin" <[email protected]> wrote:
> Thanks, > > became working after a few changes + CEE-like JSONs. > > Right now we are testing RSysLog + Elasticsearch + Kibana3, looking > awesome and pretty flexible. > Does anybody using such configuration? > /---/ > */Best regards,/* > /Eugene Istomin/ > > > > > two quick pointers as I head to dinner: > > > > a) RELP requieres a proper syslog header. you can't just simply send the > > user data to it > > b) you need to use mmjsonparse on the receiver to re-populate the JSON > > structure > > > > Note that b) requires a) + a proper CEE header for mmjsonparse to > work. > > > > HTH at least a little bit;) > > > > Rainer > > > > On Fri, Feb 21, 2014 at 6:47 PM, Eugene Istomin <[email protected]> > wrote: > > > Another run for our app logging: > > > > > > > > > client have rule for normalizing and action rule for server sending via > > > relp: > > > > > > template(name="NginxFileFormat" type="list") { > > > > > > constant(value="{") > > > property(name="$!date" format="jsonf") > > > constant(value=", ") > > > property(name="$!x_client" format="jsonf") > > > constant(value=", ") > > > property(name="$!request_method" format="jsonf") > > > constant(value=", ") > > > property(name="$!status" format="jsonf") > > > constant(value=", ") > > > property(name="$!request_uri" format="jsonf") > > > constant(value=", ") > > > property(name="$!gzip_ratio" format="jsonf") > > > constant(value=", ") > > > property(name="$!request_time" format="jsonf") > > > constant(value=", ") > > > property(name="$!body_bytes_sent" format="jsonf") > > > constant(value=", ") > > > property(name="$!user_agent" format="jsonf" position.To="8") > > > constant(value="}\n") > > > > > > } > > > > > > template(name="NginxFileFormatSaver" type="string" > > > string="/web/_deploy/%$!folder%/logs/nginx/%$month%. > %$day%/%$!http_host%") > > > > > > if $programname == 'nginx' then > > > > > > action(type="mmnormalize" useRawMsg="on" > > > > > > ruleBase="/etc/rsyslog.d/rules/nginx") > > > > > > & action(type="omrelp" target="10.100.101.250" port="20514" > > > > > > template="NginxFileFormat") > > > > > > & action(type="omfile" Template="NginxFileFormat" > > > > > > DynaFile="NginxFileFormatSaver") > > > > > > & stop > > > > > > Local messaging works ok: > > > > > > tailf /web/_deploy/eds.work/logs/nginx/02.21/XXXX > > > > > > {"date":"Feb 21 19:37:52", "x_client":"XXX", "request_method":"GET", > > > "status":"200", "request_uri":"XXXX", "gzip_ratio":"2.23", > > > "request_time":"0.000", "body_bytes_sent":"292", > "user_agent":"Mozilla/"} > > > > > > > > > > > > On the server side we have following conf: > > > > > > ...... > > > module(load="imrelp" Ruleset="nginx") > > > input(type="imrelp" Port="20514") > > > > > > template (name="nginx" type="list" sql.option="on") { > > > constant(value="INSERT INTO nginx (http_host, ........... > > > user_agent)") > > > constant(value=" VALUES ('") > > > property(name="$!http_host") > > > constant(value=",") > > > property(name="$!var_x_forwarded_for") > > > > > > ....... > > > > > > property(name="$!var_user_agent") > > > constant(value="')") > > > } > > > > > > ruleset(name="nginx") { > > > > > > action(type="mmjsonparse") > > > > > > set $!var_http_host = $http_host; > > > > > > ...... > > > > > > set $!var_user_agent = $user_agent; > > > > > > action(type="ommysql" server="localhost" serverport="3306" > > > > > > db="rsyslog" uid="rsyslog" pwd="rsyslog" template="nginx") > > > } > > > > > > > > > > > > The question is why rsyslog doesn't work like expected (inserting data > > > into DB), the errror is *INVALID PROPERTY NAME* > > > > > > > > > 4001.894777365:7f578a2b7700: relp engine is dispatching frame with > command > > > 'syslog' > > > 4001.894782968:7f578a2b7700: in 'syslog' command handler > > > 4001.894803276:7f578a2b7700: main Q: qqueueAdd: entry added, > size now log > > > 1, phys 1 entries > > > 4001.894819735:7f578a2b7700: main Q: EnqueueMsg advised worker > start > > > 4001.894842354:7f578a2b7700: in destructor: sendbuf > 0x7f577c0293a0 > > > 4001.894872669:7f578a2b7700: librelp: epoll_set_events sock 11, > target > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
