The following issue has been UPDATED. ====================================================================== http://www.dbmail.org/mantis/view.php?id=867 ====================================================================== Reported By: motto Assigned To: paul ====================================================================== Project: DBMail Issue ID: 867 Category: Database layer Reproducibility: always Severity: major Priority: normal Status: feedback target: ====================================================================== Date Submitted: 01-Dec-10 16:26 CET Last Modified: 31-Dec-10 16:34 CET ====================================================================== Summary: Jananese subject - non utf8 insert into DB Description: # dbmail-smtp -V This is DBMail version 2.2.17
# cat /etc/dbmail.conf | grep encodi encoding = utf8 default_msg_encoding = utf8 PostgreSQL DB - 8.4.5 Db created with default encoding UTF8. Name | Owner | Encoding | Collation | Ctype | Access privileges -----------+----------+----------+-------------+-------------+----------------------- dbmail_db | adm | UTF8 | en_US.UTF-8 | en_US.UTF-8 | Sending email with next subject cause the issue below: """ 告発外の4億円「犯罪事実」に小沢氏 読売新聞 - ?23 分前? 小沢一郎?元民主党代表(68)の資金管理団体「陸山会」の政治資金規正法違反事件を巡り、東京第5検察審査会が小沢氏を「起訴すべきだ」と """ and here is DB logs: 2010-12-01 09:32:47 ESTLOG: statement: SELECT dbmail_mailboxes.owner_idnr FROM dbmail_mailboxes, dbmail_messages WHERE dbmail_mailboxes.mailbox_idnr = dbmail_messages.mailbox_idnr AND dbmail_messages.message_idnr = 92816 2010-12-01 09:32:47 ESTLOG: statement: SELECT user_idnr FROM dbmail_users WHERE lower(userid) = lower('_...@!internal_delivery_user!@__') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='content-type' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (1,68384,'text/plain; charset="utf-8"') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='date' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (2,68384,'Wed, 01 Dec 2010 14:32:47 +0000') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='from' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (3,68384,'"test11111放上的方式反對薩芬撒方式地方 test1發的薩芬的撒輻射的發" <te...@wildcat.bzcrew.com>') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='message-id' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (4,68384,'<20101201143247.23848.91850117...@wildcat.bzcrew.com>') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='received' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (5,68384,'from localhost ([127.0.0.1]:34260 helo=wildcat.bzcrew.com) by wildcat.bzcrew.com with smtp (Exim 4.72) (envelope-from <te...@wildcat.bzcrew.com>) id 1PNnjj-0003eN-Fi for te...@wildcat.bzcrew.com; Wed, 01 Dec 2010 09:32:47 -0500') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='return-path' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (6,68384,'"test11111放上的方式反對薩芬撒方式地方 test1發的薩芬的撒輻射的發" <te...@wildcat.bzcrew.com>') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='sender' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (7,68384,'te...@wildcat.bzcrew.com') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='subject' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (8,68384,'告発外の4億円「犯罪事実」に小沢氏 読売新聞 - ?23 分前? 小沢一郎?元民主党代表(68)の資金管理団体「陸山会」の政治資金規正法違反事件を巡り、東京第5検察審査会が小沢氏を「起訴すべきだ」と') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='to' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (9,68384,'te...@wildcat.bzcrew.com') 2010-12-01 09:32:47 ESTLOG: statement: SELECT id FROM dbmail_headername WHERE LOWER(headername)='x-viamail-thumbnail-url' 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_headervalue (headername_id, physmessage_id, headervalue) VALUES (10,68384,'http://www.wildcat.bzcrew.com/opus/profile?_a=getresource&id=5002&type=thumbnail&size=320x320&sign=fj51jHMItqu34iO9garc7HR3kuCL1HvG8KLaDPv42ew%3D') 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_tofield (physmessage_id, toname, toaddr) VALUES (68384,'','te...@wildcat.bzcrew.com') 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_fromfield (physmessage_id, fromname, fromaddr) VALUES (68384,'test11111放上的方式反對薩芬撒方式地方 test1發的薩芬的撒輻射的發','te...@wildcat.bzcrew.com') 2010-12-01 09:32:47 ESTLOG: statement: INSERT INTO dbmail_datefield (physmessage_id, datefield) VALUES (68384,'2010-12-01 14:32:47') 2010-12-01 09:32:47 ESTERROR: invalid byte sequence for encoding "UTF8": 0xe82020 2010-12-01 09:32:47 ESTHINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding". ====================================================================== ---------------------------------------------------------------------- (0003126) paul (administrator) - 07-Dec-10 15:08 http://www.dbmail.org/mantis/view.php?id=867#c3126 ---------------------------------------------------------------------- Could you please provide a raw test message to reproduce this? ---------------------------------------------------------------------- (0003127) paul (administrator) - 07-Dec-10 15:09 http://www.dbmail.org/mantis/view.php?id=867#c3127 ---------------------------------------------------------------------- Without a raw message I can't test this. ---------------------------------------------------------------------- (0003129) motto (reporter) - 09-Dec-10 10:04 http://www.dbmail.org/mantis/view.php?id=867#c3129 ---------------------------------------------------------------------- From d...@temple.bzcrew.com Thu Dec 09 11:03:46 2010 Received: from localhost ([127.0.0.1]:55320 helo=temple.bzcrew.com) by temple.bzcrew.com with smtp (Exim 4.72) (envelope-from <d...@temple.bzcrew.com>) id 1PQcPh-00006o-Qf for ca...@temple.bzcrew.com; Thu, 09 Dec 2010 11:03:45 +0200 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Date: Thu, 09 Dec 2010 09:03:45 +0000 Message-ID: <20101209090345.28612.182662938...@temple.bzcrew.com> From: " " <d...@temple.bzcrew.com> To: ca...@temple.bzcrew.com Sender: d...@temple.bzcrew.com Subject: =?utf-8?b?5ZGK55m65aSW44Gu77yU5YSE5YaG44CM54qv572q5LqL5a6f44CN?= =?utf-8?b?44Gr5bCP5rKi5rCPIOiqreWjsuaWsOiBniAtID8yMyDliIbliY0/IA==?= =?utf-8?b?5bCP5rKi5LiA6YOOP+WFg+awkeS4u+WFmuS7o+ihqO+8iO+8lu+8mA==?= =?utf-8?b?77yJ44Gu6LOH6YeR566h55CG5Zuj5L2T44CM6Zm45bGx5Lya44CN44Gu?= =?utf-8?b?5pS/5rK76LOH6YeR6KaP5q2j5rOV6YGV5Y+N5LqL5Lu244KS5beh44KK?= =?utf-8?b?44CB5p2x5Lqs56ys77yV5qSc5a+f5a+p5p+75Lya44GM5bCP5rKi5rCP?= =?utf-8?b?44KS44CM6LW36Ki044GZ44G544GN44Gg44CN44Go5ZGK55m65aSW44Gu?= =?utf-8?b?77yU5YSE5YaG44CM54qv572q5LqL5a6f44CN44Gr5bCP5rKi5rCPIA==?= =?utf-8?b?6Kqt5aOy5paw6IGeIC0gPzIzIOWIhuWJjT8g5bCP5rKi5LiA6YOOPw==?= =?utf-8?b?5YWD5rCR5Li75YWa5Luj6KGo77yI77yW77yY77yJ44Gu6LOH6YeR566h?= =?utf-8?b?55CG5Zuj5L2T44CM6Zm45bGx5Lya44CN44Gu5pS/5rK76LOH6YeR6KaP?= =?utf-8?b?5q2j5rOV6YGV5Y+N5LqL5Lu244KS5beh44KK44CB5p2x5Lqs56ys77yV?= =?utf-8?b?5qSc5a+f5a+p5p+75Lya44GM5bCP5rKi5rCP44KS44CM6LW36Ki044GZ?= =?utf-8?b?44G544GN44Gg44CN44Go?= X-ViaMail-Thumbnail-Url: http://www.temple.bzcrew.com/opus/profile?_a=getresource&id=6828&type=thumbnail&size=320x320&sign=5fmi7HLnQl1CBFE7aoHD8Wv4gd3SICgHJShmGoBQsiw%3D ---------------------------------------------------------------------- (0003132) paul (administrator) - 31-Dec-10 16:34 http://www.dbmail.org/mantis/view.php?id=867#c3132 ---------------------------------------------------------------------- problem confirmed for 2.2 it does not occur on 3.0 please consider upgrading. Issue History Date Modified Username Field Change ====================================================================== 01-Dec-10 16:26 motto New Issue 07-Dec-10 15:08 paul Note Added: 0003126 07-Dec-10 15:09 paul Note Added: 0003127 07-Dec-10 15:09 paul Assigned To => paul 07-Dec-10 15:09 paul Status new => feedback 09-Dec-10 10:04 motto Note Added: 0003129 09-Dec-10 10:05 motto Issue Monitored: motto 31-Dec-10 16:34 paul Note Added: 0003132 ====================================================================== _______________________________________________ Dbmail-dev mailing list Dbmail-dev@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail-dev