Am 18.09.2013 um 20:14 schrieb David Lang <[email protected]>: > On Wed, 18 Sep 2013, Axel Rau wrote: > >>> Ack .. but to do it right, I think we need to know which encoding was used >>> in the first place. Well, ok, to get started something that simply >>> "fixes"/discards invalid UTF-8 may work decently enough and in any case >>> better than what we currently have ;) >> Ack to both. >>>> Axel >>>> PS: I have some experimental code handy, which should do the job. >>> Any place to look at it? ;) >> https://www.chaos1.de/downloads/utf-fix-2.c >>> I think a script function (like utf8fix()) >> Indeed. (-; >>> would probably be a good and fast enough to implement solution. A very >>> basic fix function would probably simply remove those invalid sequences, >>> but if there is more elaborate fixing possible, I am all interested in it >>> (again, I think the ultimate solution must be a conversion based on known >>> charset). >> Yes, but there are cases where it is not known. >> Example: Mail clients, producing headers in local encoding, without using >> the correct escapes. >> (A correct example would be: >> Subject: =?iso-8859-1?Q?Oh_Du_wundersch=F6ner_M=E4rz?= >> > > and there are always going to be cases where the submitter just gets it wrong. > > Since the world seems to be going UTF8 and UTF8 is a strict superset of the > ASCII that rsyslog has traditionally supported, I think there is always going > to be a need to sanitize this. > > I would suggest enhancing the control character escape handling to have a new > option > > EscapeInfvalidUTF8 > > any byte sequences that are not valid UTF8 get changed to #nnn just like > control characters. This would help me a lot. > > This will allow people to paper over many of the problems and deal with > senders that submit invalid UTF8 in the future. > > After this we can talk about conversion routines. > > David Lang > > P.S. If you do muck with the control character handling, please add another > config option to not escape tabs
Axel --- PGP-Key:29E99DD6 ☀ +49 151 2300 9283 ☀ computing @ chaos claudius _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
