Re: UTF7 - UTF8 folder encoding errors

[Eric Liang]

Also, browsing the SVN changes I noticed you fixed the search issue (file: program/steps/mail/search.inc, $imap_charset = 'UTF-8'; ). I have spotted it 3-4 days ago but did not report as I considered it minor in comparison to folders issue.

 

I now report that searching works nicely on non EN charset I've tested with (rev 303 did not). I would advise other people to check also with their own non EN charsets.

 

Thanks again,

Eric

 

On 8/10/06, Eric Liang <[EMAIL PROTECTED]> wrote:

First of all, thank you for this. It made things better. New comments are inline.

On 8/10/06, Thomas Bruederli < [EMAIL PROTECTED]> wrote:

IMP uses the PHP integrated IMAP functions and as far as I could see
do thex handle the charset conversion internally.

 

I don't know - I have just tested IMP and Group-Office lately and they both seem to treat folders nicely from a user's point of view. I am not that familiar with how they do it.

I just committed some changes that should solve these problems. It
works well with my mailbox but I only use ISO characters. Please
checkout the latest revision and test it with your environment.

 

These are the results:

 

1) In folders list the multibyte characters are cut (... is put in the middle of the string) incorrectly. This happened before, I just didn't mention it to the dev list until now. This means that the PHP function should check the length of string as multibyte and cut it as such. Currently I have folders that have AAA?...AAAAA (where AAA=multibyte chars). The question mark (?) is shown because the second byte of the character is cut so it's substituted by ? (therefore I assume that the PHP fuctions do not treat this string as multibyte before checking/converting/minimizing length). There are multibyte folder name with real length of 8 chars that are cut and English folders that are 10 or more and are not cut. So perhaps this behaviour should be examined.

 

2) When I use English GB language the folders work nicely. They used to work nicely on the folder list (ie show), now the create/rename folder works flawlessly too (big thanks).

 

But when I use another language, for example Spanish, things get messed up, just like in the past. Let me know if I am allowed to send you screenshots (via personal email) to show what happens when charset is not ISO-8859-1, or perhaps a login account on such a mailbox.

 

Results (for non English charset):

Create folder -> "error occured while creating folder" or similar error in the translated language

View folder -> strange charset conversion (or no conversion at all?) is shown instead of the normal folder

Rename folder -> "error occured while creating folder" or similar error in the translated language

 

I have noticed that the erratic folder behavior happens when I use specific languages like

Slovak, Polski, Greek, Espanol, Arabic etc but not on Japanese, Russian, English (it works fine on those showing always the correct folder names as intended)

 

I can only assume it has to do with internal PHP charset conversions that only support some charsets and not others? Like what happens with html_entity_decode that supports only _some_ charsets: http://nl2.php.net/manual/en/function.html-entity-decode.php

 

Finally, I am using PHP 4.4.x branch to test.

 

Let me know how I can help further to solve this.

 

Your support is very much appreciated,

 

Eric