Been on the road and will be over the weekend and part of next week (thus
sluggish responses ;)).
> > > For me, this sums up to one question:
> > >
> > > Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> > > non-UTF8 strings? Everyone is trying to be UTF-8 clean these days,
> so it
> > > would be bad if ompgsql could not keep up.
> >
> > I think this is a special case because rsyslog is not the originator
> of
> > those messages. It "just" transports them. And because the syslog-
> Protocol
> > does not define something like encoding in any way the best thing to
> do is
> > just leave those strings "as-is" and make the database behind it do
> so as
> > well with SQL_ASCII.
>
> I like the idea of seeing rsyslog as some kind of transport only. This
> is the
> best argument for switching to SQL_ASCII altogether so far.
>
> Rainer, do you have any thoughts on this?
Let me elaborte a bit: the new IETF syslog standards *do* specify character
encoding and strongly recommend Unicode (UTF-8) to be used. Of course, this
does not solve the issue with original senders that use another, unspecified,
coding. But it helps.
Unfortunately, rsyslog's "old" code is far from being Unicode-aware. As a
side-activity, I am upgrading "old" code to "new" code, which then uses
rsyslog's string classes. While they do not yet support Unicode, it is much
easier to make them support it once all string handling is done consistently.
However, even then I need to have a build time switch to turn this on/off,
because rsyslog in Unicode mode will take not only considerably more space
(especially with larger in-memory queues), it will also considerably affect
its performance (in terms of bytes, the memory transfer rate is effectively
cut in half, as most data in syslog is character-based - also think about the
effects on cache performance).
So moving the whole system to Unicode, while desirable, is far from being a
trivial task. Having seen extremely low demand for that, I have so far opted
to do this at a very low priorty (even though that means I violate RFC5424).
>
> > I thing everythign else will be error prone in some way. The
> Documentation
> > of rsyslog should bring a big fat NOTE that the database must be
> SQL_ASCII
> > as other wise thesrings might not be accepted.
>
> Yes, and the createDB.sql for ompgsql should be changed as well.
>
The doc needs to be written so that I can add this warning ;) Is someone with
actual Postgres knowledge up for this task. Plain text is OK, I can then
copy&paste that into a module doc template.
As for createDB.sql: let me know what I need to change, and I'll apply that
change.
Rainer
> Regards,
> Jakob Haufe (sur5r)
>
>
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAktYxoIACgkQ1YAhDic+adZvugCffdUcjqR/EiQIGojSgEh8A8lU
> m2EAn1AZ1ebx4l+GCFqQLSvg6FqBZFvG
> =1POP
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
