The URIEncoding parameter of the Tomcat connector does not seem to be taken
into account
----------------------------------------------------------------------------------------
Key: OFBIZ-281
URL: http://issues.apache.org/jira/browse/OFBIZ-281
Project: OFBiz (The Open for Business Project)
Issue Type: Bug
Components: framework
Affects Versions: SVN trunk
Environment: Linux 2.6.x, firefox 1.5
Reporter: Marco Risaliti
When I create an entity value which contains UTF-8 characters in its primary
keys, I'm unable to access it via webtools entity data maintenance or its
corresponding management interface in backend.
For example, if you create a new Security Group from partymgr application with
theses parameters :
id = Securité
description = Test
Then you try to select the newly created Security Group from Security Group
List or you type this url :
https://127.0.0.1:8443/partymgr/control/EditSecurityGroup?groupId=securit%C3%A9
you should obtain an Edit Security Group form with theses parameters :
id = securité -[CommonCannotBeFound: [securité]]-
description =
The symptoms are similar when you try to access to this entity via webtools
https://127.0.0.1:8443/webtools/control/ViewGeneric?entityName=SecurityGroup&groupId=securit%C3%A9
-> Specified SecurityGroup was not found.
The problem is not specific to SecurityGroup entity, it can be reproduced for
all entities.
After some search, it appears that request.getParameter(pkField) doesn't decode
correctly the UTF-8 sequence "%C3%A9" whereas URIEncoding of HTTP(S) connector
is
set to UTF-8.
The patch 'URIEncoding-problem.patch' try to demonstrate that the URIEncoding
specified in base/config/ofbiz-containers.xml (UTF-8) is not set at the
connector level. After having applied the patch, recompiled Ofbiz and
restarted it, the following lines should appear at the end of Ofbiz loading.
32017 (main) [ CatalinaContainer.java:238:INFO ] Connector AJP/1.3 @ 8009 -
not-secure URIEncoding=null [org.apache.jk.server.JkCoyoteHandler] started.
32018 (main) [ CatalinaContainer.java:235:INFO ] Connector HTTP/1.1 @ 8080 -
not-secure URIEncoding=null [org.apache.coyote.http11.Http11Protocol] started.
32018 (main) [ CatalinaContainer.java:235:INFO ] Connector TLS @ 8443 - secure
URIEncoding=null [org.apache.coyote.http11.Http11Protocol] started.
32022 (main) [ CatalinaContainer.java:242:INFO ] Started Apache Tomcat/5.5.9
I've written a small workaround that use setURIEncoding instead of setProperty
Connector's method. After having applied the patch
'URIEncoding-quickfix.patch',
recompiled Ofbiz and restarted it, you should see the following lines at the
end
of ofbiz loading.
20551 (main) [ CatalinaContainer.java:238:INFO ] Connector AJP/1.3 @ 8009 -
not-secure URIEncoding=UTF-8 [org.apache.jk.server.JkCoyoteHandler] started.
20552 (main) [ CatalinaContainer.java:235:INFO ] Connector HTTP/1.1 @ 8080 -
not-secure URIEncoding=UTF-8 [org.apache.coyote.http11.Http11Protocol] started.
20552 (main) [ CatalinaContainer.java:235:INFO ] Connector TLS @ 8443 - secure
URIEncoding=UTF-8 [org.apache.coyote.http11.Http11Protocol] started.
20602 (main) [ CatalinaContainer.java:242:INFO ] Started Apache Tomcat/5.5.9
With this patch the problem disapear but I don't think it is the right
solution.
I think the behavior of the setProperty(String, String) method of Tomcat
Connector class has changed in 5.5.x series. When you look at its source code :
(http://svn.apache.org/repos/asf/tomcat/container/tc5.5.x/catalina/src/share/org/apache/catalina/connector/Connector.java)
it seems this method only set parameters of the protocol handler. And
URIEncoding is a parameter of the connector and not of the protocol handler.
This problem does not appear in the non-embedded version of tomcat because they
use
common-digester to map xml elements and attributes of configuration file to
setters
of Connector object.
Can someone confirm this problem ?
All Comments Work Log Change History Sort Order:
Comment by David E. Jones [24/Apr/06 12:16 AM] [ Permlink ]
This is actually a bigger issue than you might think. Even if the tomcat
setting is in there properly, there are still problems so I HIGHLY recommend
agains't trying to use UTF-8 characters in an HTTP URL. Below is some research
I did on this a while back:
========================================================================
I tried various changes to setting of the character encoding on the request.
With no character encoding neither URI parameters nor the form input (POST) are
decoded properly. When UTF-8 character encoding is used the form input (POST)
parameters are decoded properly, but not the URI parameters.
After playing around and verifying a few things I started to do research in the
Tomcat bug tracking site and found some other things to try there, but the
results are not very encouraging. The clippings below are from the following
URL:
http://issues.apache.org/bugzilla/show_bug.cgi?id=23929
Based on this I have changed the URIEncoding to UTF-8, but it does not seem to
fix the problem. I tried this with the useBodyEncodingForURI with both possible
values, ie true and false. In none of these conditions did it work with Safari
or Firefox (Camino and other Mozilla-based browsers seem to behave the same). I
did not try any of these with IE on Windows because if these browsers
(especially Firefox) do not work, it doesn't really matter, but I suspect IE
will have a similar problem based on what we were seeing before when we did
test it with IE on Windows in the VNC session.
It looks like the best set of values for these is URIEncoding=UTF-8 and
useBodyEncodingForURI=false, but even with those settings it is not working
properly and according to the Tomcat guys, there isn't any way to really get
this working.
So, based on all of this my recommendation is to restrict ID values to the
ISO-8859-1 character set. Actually, anything that is passed as a URI parameter
needs to be this way. Parameters that are passed with forms using input tags
and such can have UTF-8 characters and it appears to work reliably.
BTW, I also tried using our variation on the standard ?= and &= syntax of URI
parameters, which is the /~= syntax (ie /~productId=África instead of
?productId=África), and that did not work either, which is to be expected as it
is also part of the URI string.
The only other thing I can think of to try is doing our own UTF-8 decoding of
URI parameter values. I'm not sure it is feasible or will work reliably (or at
all), but may be worth a try.
-David
===========================================
Sorry, there's no bug. BZ is not there to discuss design decisions. If you want
to do so, post on tomcat-dev. The only standard for URL encoding is to use
UTF-8, but nobody follows the standard. You can also now configure the URI
encoding in the connector. If you insist on using i18n with URL parameters, the
result is that it won't work reliably, but of course, you're free to do what
you
want ;-)
Please do not reopen the report.
===========================================
AND
===========================================
>From Mark:
Character encoding has been the source of quite a bit of debate on the tomcat-
dev list in recent weeks. There have been a few changes (see summary below) as
a result. Essentially some additional configuration options have been
provided. The UTF-8 issue (also reported in bug 22666) has also been fixed.
Character encoding summary:
There are a number of situations where there may be a requirement to use non-
US ASCII characters in a URI. These include:
- Parameters in the query string
- Servlet paths
There is a standard for encoding URIs (http://www.w3.org/International/O-URL-
code.html) but this standard is not consistently followed by clients. This
causes a number of problems.
The functionality provided by Tomcat (4 and 5) to handle this less than ideal
situation is described below.
1. The Coyote HTTP/1.1 connector has a useBodyEncodingForURI attribute which
if set to true will use the request body encoding to decode the URI query
parameters.
- The default value is true for TC4 (breaks spec but gives consistent
behaviour across TC4 versions)
- The default value is false for TC5 (spec compliant but there may be
migration issues for some apps)
2. The Coyote HTTP/1.1 connector has a URIEncoding attribute which defaults to
ISO-8859-1.
3. The parameters class (o.a.t.u.http.Parameters) has a QueryStringEncoding
field which defaults to the URIEncoding. It must be set before the parameters
are parsed to have an effect.
Things to note regarding the servlet API:
1. HttpServletRequest.setCharacterEncoding() normally only applies to the
request body NOT the URI.
2. HttpServletRequest.getPathInfo() is decoded by the web container.
3. HttpServletRequest.getRequestURI() is not decoded by container.
Other tips:
1. Use POST with forms to return parameters as the parameters are then part of
the request body.
===========================================
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
