Bastian,
> >> [? [:header_field|Subject] ||, Subject:
> >> [:dquote|[:header_field|Subject]]]#
> >> and get strange encodings on subjects, most german mails with äöü,
> >> sometimes with " or , and the following appears in the amavis logfiles:
> >> Subject:
> >> "=?iso-8859-1?Q?Testmail_mit_""Anf=FChrungszeichen"",_und,_Kommas?=",
> >>
> >> Is there a way to change the encoding of the log entries?
> sorry for asking again, but after some research, we found that at least
> those strange things are in our Logfiles (Amavis->Postfix->Syslog):
>
> # =?ISO-8859-1?Q? , =?iso-8859-1?Q? , =?iso-8859-15?Q? ,
> =?iso-8859-15?Q?
> # =?UTF-8?B? , =?utf-8?Q? , =?Cp1252?Q? , =?windows-1252?Q? ,
> =?koi8-r?B?
>
> You wrote in your mail that logging in utf-8 to syslog would be a bad
> idea, but there log entries are utf-8, or am I totally wrong?
> This seems to be base64, too?
> Subject: "=?utf-8?B?QnJpZWZ0YXViZSBnZWZ1bmRlbiDigJMgd2FzIHR1bj8=?="
>
> This is pretty normal, the problem is an "ü" in für/f=C3=BCr
> Subject: "=?UTF-8?Q?vorabrechnung_f=C3=BCr_Papstpalast?="
>
> My problem is not to handle this utf-8 strings in logfile, not even
> utf-8+base64, decoding those is no problem, but there are so many different
> types (like I wrote above), that writing an decoder for all is a real pita,
> and nobody knows things like =?koi8-r?B? :-)
>
> Would it be possible to decode each string that goes to syslog with utf-8?
> You said that the messages to the DB are utf-8 converted. Why not giving
> out every string to syslog with that?
Here is a patch for amavisd 2.6.4 which implements a macro 'mime2utf8',
which does what you want. Place the following in your $log_templ
(or uncomment corresponding lines in the default log template
near the end of file 'amavisd'):
[? [:header_field|Subject]||, Subject: [:dquote|[:mime2utf8|[:header_field|
Subject]|100]]]#
[? [:header_field|From] ||, From: [:uquote|[:mime2utf8|[:header_field|From]|
100]]]#
The patch will go into 2.7.0-pre8. A release notes entry:
- added a macro 'mime2utf8' which takes a string as its first argument,
and an optional truncation length as the second. The string is decoded
as a MIME-Header string (understands Q or B character set encodings
like =?iso-8859-2?Q?...?=, =?koi8-r?B?...?=) and converted to UTF-8,
optionally truncated to the specified size at clean UTF-8 boundaries,
and returned as a result.
The macro can be useful in a logging template or in other templates
to decode Subject or From header fields, e.g.:
[? [:header_field|Subject]||,\
Subject: [:dquote|[:mime2utf8|[:header_field|Subject]|100]]]#
The patch (mind the long lines if your mail client wrapped them):
--- amavisd~ 2009-06-25 14:39:01.000000000 +0200
+++ amavisd 2010-10-15 17:39:32.617730637 +0200
@@ -8473,4 +8473,5 @@
ll do_log sanitize_str debug_oneshot
am_id add_entropy generate_mail_id
+ safe_decode safe_encode
prolong_timer waiting_for_client
switch_to_my_time switch_to_client_time
@@ -9001,4 +9002,30 @@
uquote => sub {my($nm)=shift; join('', map { s{[ \t]+}{_}g; $_ }
@_)},
b64encode => sub {my($nm)=shift; join(' ', map {encode_base64($_,'')}
@_)},
+ mime2utf8 => sub { # convert to UTF8 octets, truncate to $max_len if
given
+ my($nm,$str,$max_len) = @_;
+ $str = '' if !defined $str;
+ if ($unicode_aware && $str ne '') {
+ eval {
+ my($chars) = safe_decode('MIME-Header',$str); # logical characters
+ my($octets) = safe_encode('UTF-8',$chars); # bytes, UTF-8 encoded
+ if (defined $max_len && $max_len > 0 && length($octets) > $max_len)
{
+ local($1);
+ if ($octets =~ /^(.{0,$max_len})(?=[\x00-\x7F\xC0-\xFF]|\z)/s) {
+ $octets = $1; # cleanly chop a UTF-8 byte sequence, RFC 3629
+ }
+ }
+ $str = $octets; 1;
+ } or do {
+ my($eval_stat) = $@ ne '' ? $@ : "errno=$!"; chomp $eval_stat;
+ do_log(5, "macro mime2utf8: malformed string, keeping raw bytes:
%s",
+ $eval_stat);
+ };
+ }
+ if (defined $max_len && $max_len > 0 && length($str) > $max_len) {
+ # in case of errors or non- unicode_aware
+ $str = substr($str,0,$max_len);
+ }
+ $str;
+ },
# macros f, T, C, B will be defined for each notification as appropriate
# (representing From:, To:, Cc:, and Bcc: respectively)
@@ -24934,5 +24961,7 @@
[remote_mta_smtp_response|[~%x|["queued as ([0-9A-Z]+)$"]|["%1"]|["%0"]]|/]#
#[? [:header_field|Subject]||, Subject: [:dquote|[:header_field|Subject|
100]]]#
+#[? [:header_field|Subject]||, Subject: [:dquote|[:mime2utf8|[:header_field|
Subject]|100]]]#
#[? [:header_field|From] ||, From: [:uquote|[:header_field|From|100]]\
+#[? [:header_field|From] ||, From: [:uquote|[:mime2utf8|[:header_field|
From]|100]]]#
#[? [:dkim|author] || (dkim:AUTHOR)]]#
#[? [:useragent|name] ||, [:useragent|name]: [:uquote|[:useragent|body]]]#
@@ -24962,5 +24991,7 @@
#, smtp_resp: [:smtp_response]#
#[? [:header_field|Subject]||, Subject: [:dquote|[:header_field|Subject|
100]]]#
+#[? [:header_field|Subject]||, Subject: [:dquote|[:mime2utf8|[:header_field|
Subject]|100]]]#
#[? [:header_field|From] ||, From: [:uquote|[:header_field|From|100]]\
+#[? [:header_field|From] ||, From: [:uquote|[:mime2utf8|[:header_field|
From]|100]]]#
#[? [:dkim|author] || (dkim:AUTHOR)]]#
#[? [:useragent|name] ||, [:useragent|name]: [:uquote|[:useragent|body]]]#
Mark
------------------------------------------------------------------------------
Download new Adobe(R) Flash(R) Builder(TM) 4
The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly
Flex(R) Builder(TM)) enable the development of rich applications that run
across multiple browsers and platforms. Download your free trials today!
http://p.sf.net/sfu/adobe-dev2dev
_______________________________________________
AMaViS-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/amavis-user
Please visit http://www.ijs.si/software/amavisd/ regularly
For administrativa requests please send email to rainer at openantivirus dot
org
