The following issue has been ACKNOWLEDGED.
======================================================================
http://www.dbmail.org/mantis/view.php?id=499
======================================================================
Reported By: AntonZ
Assigned To:
======================================================================
Project: DBMail
Issue ID: 499
Category: IMAP daemon
Reproducibility: always
Severity: feature
Priority: low
Status: acknowledged
target:
======================================================================
Date Submitted: 24-Jan-07 13:15 CET
Last Modified: 30-Jan-07 20:29 CET
======================================================================
Summary: incorrect envelope
Description:
locale and mysql charset is utf-8.
Message with not encoded(8 bit) subject have incorrect value in
dbmail_envelope.
dbmail_envelope.envelope value:
("Mon, 22 Jan 2007 19:54:04 +0400" {24}
Outlook express not show that messages. IMAP4.log:
IMAP: 19:44:48 [rx] * 1 FETCH (INTERNALDATE "22-Jan-2007 19:02:59 +0300"
RFC822.SIZE 20418 FLAGS (\Seen) UID 71569 ENVELOPE ("Mon, 22 Jan 2007
11:49:35 -0400" {24}
IMAP: 19:44:48 [rx] Buffer (literal) of length 24
IMAP: 19:44:48 [rx] erences X-Ref X-Priority X-MSMail-Priority X-MSOESRec
Newsgroups)] {44}
IMAP: 19:44:48 [rx] Buffer (literal) of length 44
IMAP: 19:44:48 [rx] )
IMAP: 19:44:48 [db] PARSE ERROR: hr=2148322516
IMAP: 19:44:48 [rx] * 2 FETCH (INTERNALDATE "22-Jan-2007 19:03:03 +0300"
RFC822.SIZE 20423 FLAGS (\Seen) UID 71570 ENVELOPE ("Mon, 22 Jan 2007
11:49:35 -0400" {24}
IMAP: 19:44:48 [rx] Buffer (literal) of length 24
IMAP: 19:44:48 [rx] erences X-Ref X-Priority X-MSMail-Priority X-MSOESRec
Newsgroups)] {44}
IMAP: 19:44:48 [rx] Buffer (literal) of length 44
IMAP: 19:44:48 [rx] )
IMAP: 19:44:48 [db] PARSE ERROR: hr=2148322516
IMAP: 19:44:48 [rx] * 3 FETCH (INTERNALDATE "22-Jan-2007 19:03:07 +0300"
RFC822.SIZE 20457 FLAGS (\Seen) UID 71571 ENVELOPE ("Mon, 22 Jan 2007
19:54:04 +0400" {24}
IMAP: 19:44:48 [rx] Buffer (literal) of length 24
IMAP: 19:44:48 [rx] erences X-Ref X-Priority X-MSMail-Priority X-MSOESRec
Newsgroups)] {44}
IMAP: 19:44:48 [rx] Buffer (literal) of length 44
IMAP: 19:44:48 [rx] )
IMAP: 19:44:48 [db] PARSE ERROR: hr=2148322516
IMAP: 19:44:48 [rx] * 4 FETCH (INTERNALDATE "22-Jan-2007 19:03:09 +0300"
RFC822.SIZE 20442 FLAGS (\Seen) UID 71572 ENVELOPE ("Mon, 22 Jan 2007
19:54:04 +0400" {24}
IMAP: 19:44:48 [rx] Buffer (literal) of length 24
IMAP: 19:44:48 [rx] erences X-Ref X-Priority X-MSMail-Priority X-MSOESRec
Newsgroups)] {44}
IMAP: 19:44:48 [rx] Buffer (literal) of length 44
IMAP: 19:44:48 [rx] )
IMAP: 19:44:48 [db] PARSE ERROR: hr=2148322516
value of subject headervalue for this messages is empty.
also empty fromname for not encoded headers From
======================================================================
----------------------------------------------------------------------
paul - 24-Jan-07 15:03
----------------------------------------------------------------------
Anton, please post trace_level=5 logs.
I don't see anything obviously wrong with the OE logs you posted.
8bit headers work for me.
----------------------------------------------------------------------
AntonZ - 24-Jan-07 15:41
----------------------------------------------------------------------
encoded headers work OK.
if subject have 8bit, dbmail generate and send incorrect:
Debug:[mailbox] dbmail-mailbox.c,dbmail_mailbox_get_set(+1233): [74813:*]
Debug:[misc] misc.c,g_tree_merge(+1219): a[0] [OR] b[1] -> a[1]
Debug:[sql] dbmysql.c,db_query(+286): query [SELECT seen_flag,
answered_flag, deleted_flag, flagged_flag, draft_flag, recent_flag,
DATE_FORMAT(internal_date, '%Y-%m-%d %T'), rfcsize, message_idnr FROM
dbmail_messages msg, dbmail_physmessage pm WHERE pm.id =
msg.physmessage_id AND message_idnr BETWEEN 73322 AND 73322 AND
mailbox_idnr = 1 AND status IN (0,1) ORDER BY message_idnr ASC]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [* 6
FETCH (]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE:
[INTERNALDATE "24-Jan-2007 10:42:34 +0300"]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [ ]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE:
[RFC822.SIZE 15709]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [ ]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [FLAGS
(\Seen)]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [ ]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [UID
73322]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [ ]
Debug:[sql] dbmysql.c,db_query(+286): query [SELECT message_idnr,envelope
FROM dbmail_envelope e JOIN dbmail_messages m ON
m.physmessage_id=e.physmessage_id JOIN dbmail_mailboxes b ON
b.mailbox_idnr=m.mailbox_idnr WHERE m.mailbox_idnr = 1 AND message_idnr
BETWEEN 73322 AND 73322 ]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE:
[ENVELOPE ("Wed, 24 Jan 2007 03:22:54 -0400" {38}
]
Debug:[imapsession] dbmail-imapsession.c,_imap_show_body_section(+1052):
itemtype [3] partspec []
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [ ]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [BODY[]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE:
[HEADER.FIELDS (References X-Ref X-Priority X-MSMail-Priority X-MSOESRec
Newsgroups)] ]
Debug:[sql] dbmysql.c,db_query(+286): query [SELECT
message_idnr,headername,headervalue FROM dbmail_headervalue v JOIN
dbmail_messages m ON v.physmessage_id=m.physmessage_id JOIN
dbmail_mailboxes b ON m.mailbox_idnr=b.mailbox_idnr JOIN dbmail_headername
n ON v.headername_id=n.id WHERE m.mailbox_idnr = 1 AND message_idnr BETWEEN
73322 AND 73322 AND lower(headername) IN
('references','x-ref','x-priority','x-msmail-priority','x-msoesrec','newsgroups')]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [{44}
X-MSMail-Priority: Normal
X-Priority: 3
]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [)
]
Debug:[imapsession]
dbmail-imapsession.c,dbmail_imap_session_printf(+1292): RESPONSE: [mb95 OK
UID FETCH completed
]
Info:[imap] imap4.c,IMAPClientHandler(+336): Finished command uid [0]
----------------------------------------------------------------------
paul - 24-Jan-07 15:52
----------------------------------------------------------------------
Anton,
I've done some minor changes in the envelope code. 8bit subjects are now
encoded in the envelope. Please tell me if it helps.
rev 2428
----------------------------------------------------------------------
paul - 24-Jan-07 15:55
----------------------------------------------------------------------
Anton,
sorry, I meant level=5 logs of message insertion.
----------------------------------------------------------------------
AntonZ - 24-Jan-07 17:08
----------------------------------------------------------------------
Ok. Found [1] missing envelope values.
Debug:[sql] dbmysql.c,db_query(+286): query [SELECT messageblk FROM
dbmail_messageblks WHERE physmessage_id = 33715 AND is_header = '1']
Debug:[message] dbmail-message.c,_set_content_from_stream(+383): parse
message
Debug:[misc] misc.c,imap_get_envelope(+2014): encoding 8bit subject [
----------------------------------------------------------------------
AntonZ - 24-Jan-07 17:19
----------------------------------------------------------------------
logs from "dbmail-utils -by" after delete from database corrupted records
gmime_utils_header_encode not encode subject
field has utf-8 charset and now in table dbmail_envelope 2 records (bug in
dbmail-utils???) with envelope=[
("Wed, 24 Jan 2007 03:22:54 -0400" {29}
acop
]
in dbmail_headervalue subject headervalue is empty
----------------------------------------------------------------------
paul - 24-Jan-07 18:41
----------------------------------------------------------------------
Anton,
UTF8 is *not* a charset. utf8 is an 8bit representation of Unicode. Dbmail
is not (yet) unicode clean.
Please attach the message involved (or a similar one) to this bug.
----------------------------------------------------------------------
AntonZ - 26-Jan-07 10:34
----------------------------------------------------------------------
g_mime_utils_header_encode_text always return not encode header.
mysql on insert truncate string to first unrecognized simbol(in utf8
representation).
I solve problem by convert envelope field from text to blob type.
Problem is keep in headervalue,subjectfield,fromfield,tofield,ccfield.
----------------------------------------------------------------------
paul - 30-Jan-07 20:29
----------------------------------------------------------------------
I'm setting this report to 'suspended - no change required'. Looks
suspiciously like a garbage-in garbage-out situation.
Injecting messages with 8bit headers of unknown (non-utf8) encoding - in
themselves a violation - into a utf8 table will result in unexpected
results. I don't see any solutions other than rejecting such messages
during insertion.
The proposed solution (setting some fields to BLOB, or BYTEA) is not valid
and not supported.
Issue History
Date Modified Username Field Change
======================================================================
24-Jan-07 13:15 AntonZ New Issue
24-Jan-07 15:03 paul Note Added: 0001782
24-Jan-07 15:41 AntonZ Note Added: 0001783
24-Jan-07 15:52 paul Note Added: 0001784
24-Jan-07 15:55 paul Note Added: 0001785
24-Jan-07 17:08 AntonZ Note Added: 0001786
24-Jan-07 17:09 AntonZ File Added: debug.dbmailutil
24-Jan-07 17:19 AntonZ Note Added: 0001787
24-Jan-07 18:41 paul Note Added: 0001791
24-Jan-07 18:54 AntonZ File Added: spam1.msg
26-Jan-07 10:34 AntonZ Note Added: 0001792
30-Jan-07 20:29 paul Note Added: 0001794
30-Jan-07 20:29 paul Priority normal => low
30-Jan-07 20:29 paul Severity major => feature
30-Jan-07 20:29 paul Status new => acknowledged
30-Jan-07 20:29 paul Resolution open => no change
required
30-Jan-07 20:29 paul Category General => IMAP daemon
======================================================================
_______________________________________________
Dbmail-dev mailing list
Dbmail-dev@dbmail.org
http://twister.fastxs.net/mailman/listinfo/dbmail-dev
