Hi Robson, The database was already converted to UTF-8 when 3.0 was installed, if that is what you are asking about. Thanks for the suggestion.
I have done some more digging and i think i have an idea where the problem lies. The Encode object (Kernel/System/Encode.pm) does not seem to correctly fix multi-byte characters when converting from a source that claims to be UTF-8, but contains multibyte characters, unless you explicitly use the *Check* argument to force it to check that the UTF-8 source was valid. I have turned that on in Kernel/Output/HTML/Layout.pm on line 4070, which fixes the subsequent errors, Is this the correct way to handle this? Should not all UTF-8 strings be checked to see if they are valid instead of trusting the sender to be doing this correctly? Regards, David Hess On Tue, Oct 22, 2013 at 11:21 AM, Robson Kobayashi - TRE/MS < robson.kobaya...@tre-ms.jus.br> wrote: > I don't know if it is necessary (with postgres it is), but did you also > upgrade the database? > > <!-------------------------------------> > Robson Massaki Kobayashi > <!-------------------------------------> > > ------------------------------ > *De: *"David Hess" <dahm...@gmail.com> > *Para: *otrs@otrs.org > *Enviadas: *Terça-feira, 22 de Outubro de 2013 11:09:33 > *Assunto: *[otrs] Articles with malformed UTF-8 characters will not > display in the browser > > > Hello, > > I am using OTRS 3.2.11 on CentOS 5.9 (32bit). > Apache 2.2.3 > mysql Ver 14.12 Distrib 5.0.95, for redhat-linux-gnu (i386) using > readline 5.1 > > > I have noticed a problem recently ever since upgrading from OTRS 3.0 (we > upgraded to 3.1 and then immediately to 3.2) that certain emails from users > that contain accent characters will not display in the browser. Instead of > the article text i just see > Internal Server Error > > The server encountered an internal error or misconfiguration and was > unable to complete your request. > > > > I am somewhat at a loss as to know where to start on this issue because i > do not know which OTRS components might be involved. > > I have discovered that if i go and edit the article file stored in > /opt/otrs/var/articles/year/month/day/articleid/ to remove the offending > character, the article displays just fine, but i do not want to have to > keep doing this. > > The httpd error log shows the following when i access the article: > > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/Output/HTML/Layout.pm line 4074. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/Output/HTML/Layout.pm line 4074. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/System/HTMLUtils.pm line 758. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/System/HTMLUtils.pm line 758. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/System/HTMLUtils.pm line 776. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/System/HTMLUtils.pm line 776. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 792. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 792. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 792. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 792. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 792. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 792. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 681. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 681. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/System/HTMLUtils.pm line 685. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/System/HTMLUtils.pm line 685. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/System/HTMLUtils.pm line 692. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution iterator at > /opt/otrs//Kernel/System/HTMLUtils.pm line 692. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 704. > -e: Malformed UTF-8 character (unexpected non-continuation byte 0x6e, > immediately after start byte 0xf3) in substitution (s///) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 953. > [error] Malformed UTF-8 character (fatal) at > /opt/otrs//Kernel/System/HTMLUtils.pm line 953.\n > > > I have also discovered that doing a reply to the unreadable article, > results in the article being correctly loaded into my article response, but > the bad character gets replaced with which is what i would expect to > happen if there is a bad character. > > So the question is, what needs to be changed to get these emails to > display correctly in the agent ticket view? I would expect the malformed > character to display as but for the article to actually be shown. > > Regards, > David Hess > > > > > > --------------------------------------------------------------------- > OTRS mailing list: otrs - Webpage: http://otrs.org/ > Archive: http://lists.otrs.org/pipermail/otrs > To unsubscribe: http://lists.otrs.org/cgi-bin/listinfo/otrs >
--------------------------------------------------------------------- OTRS mailing list: otrs - Webpage: http://otrs.org/ Archive: http://lists.otrs.org/pipermail/otrs To unsubscribe: http://lists.otrs.org/cgi-bin/listinfo/otrs
