2016-02-23 16:58 GMT+01:00 Joe Blow <[email protected]>: > Sounds good. For the time being though, would you happen to know of any > mechanism i could use to manual switch this characters? I'd like to test > if ES can ingest if i escape those chars.
I think you could use the "replace" function: http://www.rsyslog.com/doc/v8-stable/rainerscript/functions.html ... but I would really like to know what is going on here. > Is there an equivalent to 's/this/that/g' within rsyslog properties? If it is just for confirmation, you could probably send the same message via curl manually and try different tricks to it - just an idea. Rainer > > Cheers, > > JB > > On Tue, Feb 23, 2016 at 10:34 AM, Rainer Gerhards <[email protected]> > wrote: > >> 2016-02-23 16:09 GMT+01:00 Joe Blow <[email protected]>: >> > Correct. I get things like this in my omelasticsearch error log: >> > >> > "error": "MapperParsingException[failed to parse [csuriquery]]; >> > nested: JsonParseException[Invalid UTF-8 start byte 0x80\n at [Source: >> > [B@2210517d; line: 1, column: 450]] >> > >> > Then within the normalized JSON i see my <80> tags at that line. >> > >> > Any ideas? >> >> >> I have checked RFC7159 once again, and "<>" is a perfectly valid JSON >> value. There are also no rules where they recommend to escape "<". >> >> On the other hand, this reminds me of the old style <0a> >> representation of chacracters. Maybe someone from ES got "historical >> feelings" and implemented this as yet another JSON violation? ;) >> >> Anyhow, I think it would be a good idea to ask the ES folks if they >> have an issue with these characters and why. Once we know, we may find >> a way out of it... Please report back in any case. >> >> Rainer >> > >> > Cheers, >> > >> > JB >> > >> > On Tue, Feb 23, 2016 at 9:33 AM, Rainer Gerhards < >> [email protected]> >> > wrote: >> > >> >> 2016-02-23 15:29 GMT+01:00 Joe Blow <[email protected]>: >> >> > Hey all, >> >> > >> >> > I've got some logs which might have different languages in them, and >> it >> >> > appears that things like this are tripping up when i try and send >> them to >> >> > elasticsearch: >> >> > >> >> > KEDANOVA%20FA<80>ANES&sec=08& >> >> > >> >> > Specifically the <80>. What is the best way to escape both the < and >> >> the > >> >> > in the normalized field? I'm already specifying the format as JSON, >> so >> >> > backslashes are being escaped properly. Any ideas? >> >> >> >> I am not aware that <> need to be escaped. Maybe another ES JSON >> >> incompatibility? >> >> >> >> Rainer >> >> > >> >> > Thanks in advance. >> >> > >> >> > Cheers, >> >> > >> >> > JB >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com/professional-services/ >> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. >> >> >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
