Re: UTF-8 POST request results in corrupted data

[Andre-John Mas]

On 7-Oct-08, at 07:02 , Mark Thomas wrote:

Andre-John Mas wrote:
Thanks for the answer on this point. Reading section 3.7.1 of RFC  
2616
indicates that request can specify a character other than the  
default.
For this reason the following should technically be legal:

<form action="" method="post"
enctype="application/x-www-form-urlencoded; charset=utf-8"
accept-charset="utf-8">

What I see, from testing on my Mac, is that Firefox and Safari fail  
to
pass the charset attribute, but Opera does. What I do notice here is
that even though Opera does specify the character set, Tomcat  
ignores it
replacing the submitted Japanese characters by question
marks. This is an indication that UTF-8 was accepted but it was
converted to ISO-8859-1 and no equivalent mapping was available. With
Firefox and Safari I get the same behaviour when I specify:

  request.setCharacterEncoding("UTF-8");

Basically I am not getting the Japanese characters as typed in the  
form.
There is a problem here.

Not that I can see.

The test JSP half way down this page works quite happily for me on any
current Tomcat version, with any input.
http://wiki.apache.org/tomcat/FAQ/CharacterEncoding

If I remove the request.setCharacterEncoding("UTF-8"); line then  
output is
corrupted (as expected).

I then added enctype="application/x-www-form-urlencoded;  
charset=utf-8" to
the form and repeated the test. Again, the output was corrupted with  
IE7
and FF3 as expected since the charset=utf-8 is dropped by the browser.

I then used Fiddler to edit request headers and manually added
charset=utf-8. Everything then worked correctly (as expected).

Are you sure that valves and filters, such as the request dumper  
valve, are
not interfering with your test case?

At this point I am not able to validate this, but from what I can  
remember I was
using a vanilla Tomcat install. I will run further tests when I get a  
chance.

One thing that would be worth seeing is how other application servers  
deal with
this, or if anyone from the w3c has any input on this.

Andre


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]