A NOTE has been added to this issue. ====================================================================== http://dbmail.org/mantis/view.php?id=805 ====================================================================== Reported By: maximP Assigned To: ====================================================================== Project: DBMail Issue ID: 805 Category: LMTP daemon Reproducibility: have not tried Severity: major Priority: normal Status: new target: ====================================================================== Date Submitted: 09-Sep-09 16:41 CEST Last Modified: 06-Jun-11 23:56 CEST ====================================================================== Summary: LMTPD dies when incorrect utf-8 character is in header Description: It seems that not enough checks are performed when parsing and storing header values (and maybe header names also). When Dbmail tries to store a header with incorrect utf-8 character, the exception from PostgreSQL is thrown:
Error:[message] _header_value_get_id(+1575): SQLException: ERROR: invalid byte sequence for encoding "UTF8": 0xd0 HINT: This error can also happen if the byte sequ And LMTPD just dies. I had to switch to message delivering through dbmail-deliver, it's more reliable when such problems exist. Would it better for LMTPD not to die in case of errors, but be able to deliver next messages? Unfortunately, I can't provide the mail message which caused the problem (it was canceled by MTA after some days). ====================================================================== ---------------------------------------------------------------------- (0002940) maximP (reporter) - 02-Nov-09 17:19 http://dbmail.org/mantis/view.php?id=805#c2940 ---------------------------------------------------------------------- Any thoughts concerning this problem? ---------------------------------------------------------------------- (0002944) paul (administrator) - 10-Nov-09 20:12 http://dbmail.org/mantis/view.php?id=805#c2944 ---------------------------------------------------------------------- Maxim, I looked into this. Plz try: http://git.dbmail.eu/cgit/cgit.cgi/paul/dbmail/commit/?id=82e7d21504044bb4b27d2d9a69d33bb85d043bf9 ---------------------------------------------------------------------- (0002965) Marv-CZ (reporter) - 03-Jan-10 23:13 http://dbmail.org/mantis/view.php?id=805#c2965 ---------------------------------------------------------------------- I have the same problem, but with dbmail-imapd: Jan 2 19:58:16 talita postgres[7855]: [50-2] STATEMENT: INSERT INTO dbmail_header (physmessage_id, headername_id, headervalue_id) VALUES ($1,$2,$3) Jan 2 19:59:37 talita postgres[7855]: [51-1] ERROR: invalid byte sequence for encoding "UTF8": 0xc3 Jan 2 19:59:37 talita postgres[7855]: [51-2] HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled Jan 2 19:59:37 talita postgres[7855]: [51-3] by "client_encoding". Jan 2 19:59:37 talita postgres[7855]: [51-4] STATEMENT: SELECT id FROM dbmail_headervalue WHERE hash=$1 and headervalue=$2 Jan 2 19:59:37 talita kernel: pid 7854 (dbmail-imapd), uid 65534: exited on signal 6 Jan 2 21:34:00 talita postgres[19335]: [2-1] ERROR: duplicate key value violates unique constraint "dbmail_header_pkey" Jan 2 21:34:00 talita postgres[19335]: [2-2] STATEMENT: INSERT INTO dbmail_header (physmessage_id, headername_id, headervalue_id) VALUES ($1,$2,$3) Jan 2 21:34:00 talita postgres[19335]: [3-1] ERROR: invalid byte sequence for encoding "UTF8": 0xc5 Jan 2 21:34:00 talita postgres[19335]: [3-2] HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled Jan 2 21:34:00 talita postgres[19335]: [3-3] by "client_encoding". Jan 2 21:34:00 talita postgres[19335]: [3-4] STATEMENT: SELECT id FROM dbmail_headervalue WHERE hash=$1 and headervalue=$2 Jan 2 21:34:00 talita kernel: pid 19334 (dbmail-imapd), uid 65534: exited on signal 6 I have 2 saved mails which reproduce this crash, but over quota 50kB so I can't post their here. Can I send their anywhere? Paul's patch doesn't solve it. ---------------------------------------------------------------------- (0002966) paul (administrator) - 04-Jan-10 08:00 http://dbmail.org/mantis/view.php?id=805#c2966 ---------------------------------------------------------------------- Maxim, Please send those message to my private address. ---------------------------------------------------------------------- (0002967) Marv-CZ (reporter) - 04-Jan-10 08:06 http://dbmail.org/mantis/view.php?id=805#c2967 ---------------------------------------------------------------------- I'm not Maxim, I don't know your private address. ---------------------------------------------------------------------- (0002968) paul (administrator) - 04-Jan-10 08:57 http://dbmail.org/mantis/view.php?id=805#c2968 ---------------------------------------------------------------------- oops. p...@nfg.nl ---------------------------------------------------------------------- (0003076) Marv-CZ (reporter) - 06-Jul-10 21:59 http://dbmail.org/mantis/view.php?id=805#c3076 ---------------------------------------------------------------------- I have been uploaded here 2 files with messages producing this crash. ---------------------------------------------------------------------- (0003186) fknauf (reporter) - 06-Jun-11 23:56 http://dbmail.org/mantis/view.php?id=805#c3186 ---------------------------------------------------------------------- I've been hit by this bug (in lmtpd) with 3.0.0-rc2. The following subject line reproduces the error: Subject: =?UTF-8?Q?360=C2=B0=20Financial=20Risk=20?= =?UTF-8?Q?Management=20=E2=80=93=20speake?==?UTF-8?Q?rs=20confirmed=20?= Interestingly, the line Subject: =?UTF-8?Q?360=C2=B0=20Financial=20Risk=20?= passes without error. Thus I suspect the reason lmtpd crashes is that there are two rfc2047-encoded words in the header, but I haven't gotten any further than that yet. I'll load up what I think is the relevant part of my dbmail.err with file_logging_levels = 511 - ultimately, assert(*id) in _header_value_get_id fails. I hope this helps. Issue History Date Modified Username Field Change ====================================================================== 09-Sep-09 16:41 maximP New Issue 02-Nov-09 17:19 maximP Note Added: 0002940 10-Nov-09 20:12 paul Note Added: 0002944 03-Jan-10 23:13 Marv-CZ Note Added: 0002965 04-Jan-10 08:00 paul Note Added: 0002966 04-Jan-10 08:06 Marv-CZ Note Added: 0002967 04-Jan-10 08:57 paul Note Added: 0002968 06-Jul-10 21:46 Marv-CZ File Added: dbmail-01.eml 06-Jul-10 21:46 Marv-CZ File Added: dbmail-02.eml 06-Jul-10 21:59 Marv-CZ Note Added: 0003076 06-Jun-11 23:56 fknauf Note Added: 0003186 ====================================================================== _______________________________________________ Dbmail-dev mailing list Dbmail-dev@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail-dev
