Supporting dots are back in the latest elasticsearch 5.0 On Fri, Nov 18, 2016 at 6:16 AM, chenlin rao <[email protected]> wrote:
> There is a logstash plugin named logstash-filter-de_dot. > And because logstash has a force queue to send a larger bulk_size to ES > which is better in ES recommandition, I suggest to use > rsyslog-omelasticsearch only for not so high load -- but need to watch it > as quickly-- use case. > > 2016-11-18 18:53 GMT+08:00 Rainer Gerhards <[email protected]>: > > > 2016-11-18 11:51 GMT+01:00 David Lang <[email protected]>: > > > On Fri, 18 Nov 2016, Simon Tiong wrote: > > > > > >> On 18/11/2016 5:39 PM, David Lang wrote: > > >>> > > >>> On Fri, 18 Nov 2016, Peter Viskup wrote: > > >>> > > >>>> Dear all, > > >>>> at the moment our message flow is setup like this: > > >>>> rsyslog remote client->rsyslog server->local > > >>>> file->Logstash->ElasticSearch > > >>>> > > >>>> Logstash does any special message processing, just forwards messages > > to > > >>>> ES. > > >>>> We are considering the use of omelasticsearch module for direct > > >>>> forward to ES (skip local file store and LogStash). > > >>>> > > >>>> What could be the consequences or drawbacks? What are your > experiences > > >>>> you can share? > > >>> > > >>> > > >>> going directly to ES will be MUCH faster :-) > > >>> > > >>> one problem with recent versions of ES is that it won't accept any > json > > >>> that has a '.' in the object name. I don't think Logstash fixes this, > > but I > > >>> don't know what it does in those cases. > > >>> > > >>> David Lang > > >> > > >> > > >> Hi Peter, > > >> > > >> ES not accepting '.', but here is a workaround way. Under > logstash.conf > > >> you might do some converting, for example. Hope it helps. > > > > > > > > > the question was how to not use logstash and what the > > problems/advantages of > > > not using logstash are. > > > > > >> > > >> mutate { > > >> rename => { > > >> "id.orig_h" => "id_orig_h" > > >> "id.orig_p" => "id_orig_p" > > >> "id.resp_h" => "id_resp_h" > > >> "id.resp_p" => "id_resp_p" > > >> } > > >> } > > > > > > > > > this requires that you know all the values that have '.' in their name. > > If > > > you know that you can also manually do the same thing in rsyslog. > > > > recent omelasticsearch handles that itself: > > > > https://github.com/rsyslog/rsyslog/commit/0f5595c0742fce2f5fcfe86fda22bc > > 091330f368 > > > > Rainer > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
