Re: problem with getParamater in relation to character encoding.

[Stefanos Karasavvidis]

see
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=23929
there is a nice summary on what you can do (coyote connect attribute 
useBodyEncodingForURI)

Stefanos

Deepak Sharma wrote:

I m not able to display japanese characters in the browser because of the
problem with getParamater in tomcat which does not autmatically encode 
the request string in the charset set for the page.

I m on the lookout for a patch related to this or ny pointers to solve this
problem.
I do not want to use this piece of code as it'll need me to do a lot of code
change.\
"decode_str = new String(str.getBytes("8859_1") ,"UTF-8");"
Please help me out with  this.

Regards,
Deepak Sharma.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [Bug 23929] - request.setCharacterEncoding(String) doesn't work

[Stefanos Karasavvidis]

look at these bugs
bug 23929
bug 25360
bug 25231
bug 25235
bug 22666
bug 24557
bug 24345
bug 25848
some keywords I could think of are:
setCharacterEncoding
international characters
not latin characters
special characters
GET method
It is difficult to find a good question in FAQ for this issue. As this a 
tomcat FAQ (and not a general servert/jsp FAQ) I suppose that everyone 
is trying to use the setCharacterEncoding method, which will not work. 
So the FAQ question could be
setCharacterEncoding() does not work for GET method

regards

Stefanos

Tim Funk wrote:

Since this seems a rather populate bug topic ... I'd like to add this to 
the FAQ. Besides linking to this bug report, can anyone provide me:
1) A suggested wording for how to phrase this as a question
2) A suggested wording for the answer (I can figure this out)
3) Any other links (to the archives, or other bug reports) that might 
help any users looking at this.

-Tim

[EMAIL PROTECTED] wrote:

http://nagoya.apache.org/bugzilla/show_bug.cgi?id=23929

request.setCharacterEncoding(String) doesn't work

--- Additional Comments From [EMAIL PROTECTED]  2004-01-14 13:03 
---

From Mark:


Character encoding has been the source of quite a bit of debate on the 
tomcat-
dev list in recent weeks. There have been a few changes (see summary 
below) as a result. Essentially some additional configuration options 
have been provided. The UTF-8 issue (also reported in bug 22666) has 
also been fixed.

Character encoding summary
==
There are a number of situations where there may be a requirement to 
use non-
US ASCII characters in a URI. These include:
- Parameters in the query string
- Servlet paths

There is a standard for encoding URIs 
(http://www.w3.org/International/O-URL-
code.html) but this standard is not consistently followed by clients. 
This causes a number of problems.

The functionality provided by Tomcat (4 and 5) to handle this less 
than ideal situation is described below.

1. The Coyote HTTP/1.1 connector has a useBodyEncodingForURI attribute 
which if set to true will use the request body encoding to decode the 
URI query parameters.
  - The default value is true for TC4 (breaks spec but gives 
consistent behaviour across TC4 versions)
  - The default value is false for TC5 (spec compliant but there may 
be migration issues for some apps)
2. The Coyote HTTP/1.1 connector has a URIEncoding attribute which 
defaults to ISO-8859-1.
3. The parameters class (o.a.t.u.http.Parameters) has a 
QueryStringEncoding field which defaults to the URIEncoding. It must 
be set before the parameters are parsed to have an effect.

Things to note regarding the servlet API:
1. HttpServletRequest.setCharacterEncoding() normally only applies to 
the request body NOT the URI.
2. HttpServletRequest.getPathInfo() is decoded by the web container.
3. HttpServletRequest.getRequestURI() is not decoded by container.

Other tips:
1. Use POST with forms to return parameters as the parameters are then 
part of the request body.

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Fwd: JSR 154 Comments - setCharacterEncoding

[Stefanos Karasavvidis]

As I said in a previous posting, I had sent an email to JSR154 about the 
setCharacterEncoding issue. I got the following answer from Yutaka 
Yoshida who expresses his personal opinion

Stefanos

-
I share the same thought. I agree we need a facility to handle
this case as many people pass non 8859-1 chars on GET params.
Also, no, I don't mind your forwarding my message, but please
make sure that this was my personal opinion.
thank you,
Yutaka Yoshida
Sun Microsystems, Inc.
Stefanos Karasavvidis wrote:

> Hi Yuta,
>
> thank you for the reply.
>
> your point of view matches the one of the tomcat developers, and I 
don't really have anything to object as far as standard conformance is 
concerned.
>
> BUT:
>
> first of all I assume that you agree that passing parameters with the 
GET method is at least useful (IMO it is a necessity). Consider how you 
would pass to someone a URL to a dynamically generated page, based on 
GET parameters.
>
> So there has to be a way to correctly encode text values within the 
URI.  Prior to Servlet/JSP 2.3/1.2 every vendor had it's own way to 
bypass this issue (if there was any) and every developer had to 
"manually" decode the values and hope that the servlet container does 
not have it's own way of doing the decoding (e.g. Sun Web Server).
>
> We "non latin" developers could not use automatic form handling 
applications (they just called getParameter), and EVERY new web 
application introduced a new way of handling these issues.
>
> The introduction of the setCharacterEncoding method was such a BIG 
relieve for us, and now we have again to return to old style coding 
methods (I personally use servlets since 1998).
>
> I also used to teach an undergraduate "web applications" laboratory 
at the Technical University of Crete (trying to use Java), and it was 
always difficult enough to explain to the students that "1 byte is NOT 1 
character" when they tried to pass Greek text values. Now we have to 
explain to them that the way they learnt to deal with encodings within 
the servlet spec has to change again. From a teaching point of view this 
is not a problem (it is actually legitimate). But it is not legitimate 
if the students have to provide so much effort to just get some 
parameter... these are issues that should have been easy to handle a 
long time ago (and they were just until now).
>
> Anyway...
> introducing a new method is IMO the right way to go. And this should 
be done fast because there has been a lot of confusion and even damage. 
Leaving this unaddressed and hoping that every developer will become an 
expert in character encoding issues is IMHO not acceptable.
>
> It is not enough to state that the Java as a language and as a web 
application development framework can handle internationally addressed 
applications. These simple everyday problems should have a consistent 
and unchallengeable way of handling.
>
> Thank you for your time
>
> Do you mind if I forward this message to the tomcat-dev list?
>
> Regards
>
> Stefanos Karasavvidis
>
>
>  > how much it's useful
>  > since what we have to do is just re-creating a String from getBytes
>
>
>
> Yuta Yoshida wrote:
>
>> Hi Stefanos,
>>
>> I personally believe setCharacterEncoding() should only affect
>> the body as stated in javadoc, in other words, POST. Because:
>>   o your second paragraph below
>>   o if this method affected the URI too, it introduces another
>> meaning. As you know there're two mappings in URI. One is
>> from characters in URI to the octet and the other is from
>> the octet to the original character. Set[ting]CharacterEncoding
>> of the POST body is direct - the body is actually encoded in
>> the encoding scheme specified by the method, however, doing so
>> of the GET query param is not direct - it is encoded in ascii
>> but the method is specifying the encoding of the original
>> characters. That's confusing.
>>
>> Considering that the original encoding of GET URI doesn't have to
>> be the same as the one of the POST body, we might need a new method
>> to specify the GET encoding. But I'm not sure how much it's useful
>> since what we have to do is just re-creating a String from getBytes.
>>
>> Anyway, I'll put this into the list we need to address in the next
>> version of the specification. I understand most containers currently
>> implement this method for both POST and GET and we need to take that
>> fact into consideration.
>>
>> Thank you for the comment,
>> Yutaka Yoshida
>> Sun Microsystems, Inc.
>>
>>> There

Re: ServletRequest.setCharacterEncoding() and GET parameters

[Stefanos Karasavvidis]

Jess Holle wrote:

Remy Maucherat wrote:

-1.

The attribute, now that it actually exists, is well documented.


Yes, but the default behavior should be for setCharacterEncoding() to 
work according to general JSP/servlet usage recommendation e.g.: 
http://java.sun.com/developer/technicalArticles/Intl/MultilingualJSP/.

If there are reasons this is unworkable, then these should be worked out 
with Sun so they stop telling everyone that this approach works.  Until 
then obnoxious ignoramouses like me will try to do what Sun says, fail 
with Tomcat, and blame Tomcat.

--
Jess Holle
This is getting ridiculous.
Both sides have strong arguments.
I hadn't thought about this issue until tomcat changed it's behaviour, 
and maybe the Sun folks didn't do it as well.

This should really be clarified within the specification (and I hope 
they come up with a solution that allows us to continue using GET 
parameters as we were used to as this is an absolute necessity... maybe 
by introducing a new method or a second parameter).

Although I do not expect too much, I 've just sent a comment to JSR53 
and JSR154

I just hope that the new tomcat versions will be released soon

Stefanos

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Bug 23929: ServletRequest.setCharacterEncoding()

[Stefanos Karasavvidis]

Remy Maucherat wrote:
Jess Holle wrote:

Remy Maucherat wrote:

This is a good question -- but one which only applies to POST.  My bug 
case was explictly with GET.

If there is an entity body encoding specified in the request, then I 
am not sure which should override.  If there is not, then I would 
presume setCharacterEncoding() should win out.  If the only issue is 
when these differ, then I believe that site designers should simply 
ensure they don't.


I think you should read the HTTP RFC. content-type does not apply to the 
URI or the HTTP header. The fact that setCharacterEncoding would apply 
to (part of) the URI and/or the header violates the RFC on URIs.

Anyway, to put it simply: in the next release, add 
useBodyEncodingForURI="true" on the connector, and you're done.
Please don't complain that it won't do what you want before trying it.
You can also use the URIEncoding attribute to specify the path encoding.

Rémy

my 2 cents on this issue,

Remy is for sure right stating that
a) the HTTP RFC does not cover variable character encoding for query 
parameters for different requests
b) it is (sounds ?) logical to assume that the whole URI (including 
paths, query parameters etc.) should be considered as being encoded with 
the same character encoding

From a developers point of view however, applying the above two points
a) brakes expected behaviour (setCharacterEncoding() method does not 
work the same as before)
b) does not give an acceptable alternative (if all parameter passing 
could be solved with POST method, then the GET method would not be 
needed, would it?)
c) a lot of web apps stopped working when an upgrade of the tomcat 
version was performed

So I think it is legitimate to be upset when first confronted with this 
change of behaviour.

As for how easy it is to NOT file duplicate bugs on this issue, having 
followed this debate, I have collected the following list of somehow 
related bugs
bug 25360
bug 25231
bug 25235
bug 22666
bug 24557
bug 24345
bug 23929
bug 25848
and of course a bunch of messages in the developer list

Speaking for myself and having reread these messages:
Assuming I 've been working for some time with the old behaviour and 
experienced the new one, I would not be able to understand why this 
change was made, EVEN if someone gave me the above list of bugs.

I propose the following:
write a short summary of why this change was necessary and include the 
above list of bugs, as well as links to the related developer list 
threads. Then submit a link to this summary to all the above bugs.
If not already done, port the useBodyEncodingForURI parameter to the 
next 4.1.x release.

I volunteer to write the summary if the list thinks that the proposal is 
reasonable.

Regards

Stefanos

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [5.0] content-type and charset issues

[Stefanos Karasavvidis]

Remy Maucherat wrote:
Kazuhiro Kazama wrote:

Remy,

Tomcat 5.0 always adds a charset=ISO-8859-1 to the content type. 
While this is I think relatively legal, it is rather risky (it causes 
problems with some clients, as I've read on tomcat-user), and very 
dubious when dealing with non text data.


I received the report that the same(?) charset problem exists in
Tomcat 4.1.29 from Japanese developers. They said that Tomcat 4.1.27
is ok.
Could you check whether the same problem exists or not?

If not, I will analyze Tomcat 4.1.29 and send the patch.


This has been fixed already. However, I have to point out that the 
client is not compliant (not specifying a charset is equivalent to 
specifying charset=ISO-8859-1).

Remy
This might be true, but there is the "meta http-equiv" tag in HTML
that does not work as expected when setting the encoding in the HTTP 
response. If both HTTP encoding and "meta" HTML encoding are specified, 
the HTTP encoding takes precedence (according to section 5.2.2 of HTML 
4.01 spec) and the users still need to manually set the encoding from 
the user-agents interface.

For example, imagine having a set of html files served by tomcat, which 
all are encoded with non iso-8859-1 encoding. Even if all of them have a 
meta tag with the correct encoding, tomcat adds by default the 
iso-8859-1 to the HTTP response. The user seens "grabage".

Please note that Apache server does the same in a default installation 
but this is simply unacceptable in a "multi-encoding" installation. I 
have to remove the relevant directive from the httpd.conf file in all 
apache installations because my users are not able to write non 
iso-8859-1 html files.

Please also note from section 5.2.2 of HTML 4.01
"The HTTP protocol ([RFC2616], section 3.7.1) mentions ISO-8859-1 as a 
default character encoding when the "charset" parameter is absent from 
the "Content-Type" header field. In practice, this recommendation has 
proved useless because some servers don't allow a "charset" parameter to 
be sent, and others may not be configured to send the parameter. 
Therefore, user agents must not assume any default value for the 
"charset" parameter."

I would add to this, that servers simply setting a character encoding by 
defualt to ALL responses is at least as bad as not setting at all.

Stefanos Karasavvidis



--
==
Stefanos Karasavvidis
Electronic & Computer Engineer
e-mail : [EMAIL PROTECTED]
Multimedia Systems Center S.A.
Kissamou 178
73100 Chania - Crete - Hellas
http://www.msc.gr
Tel : +30 2821 0 88447
Fax : +30 2821 0 88427
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: AW: AW: [5.0] Connector default configuration + connection timeout

[Stefanos Karasavvidis]

Section 14.45 of HTTP 1.1 states:
The Via general-header field MUST be used by gateways and proxies to 
indicate the intermediate protocols and recipients between the user 
agent and the server on requests, and between the origin server and 
the client on responses..

This means that if a proxy is involved there will be a Via header.

Moreover there is the X-Forwarded-For header (NOT part of the standard) 
used for example by squid proxy, which indicates the original host 
issuing the request.

Of course nothing prohibits a DoS host to insert these headers to fake a 
proxy!!

Stefanos

Fischer, Ilona wrote:

IMHO it's the definition of a proxy to send/recive requests instead of
client... that means only the proxy was talking with the webserver -> in the
HTTP-header was only the IP of the Proxy
Regards :o)
Ilona


-Ursprüngliche Nachricht-
Von: Henri Gomez [mailto:[EMAIL PROTECTED]
Gesendet: Montag, 28. Juli 2003 18:01
An: Tomcat Developers List
Betreff: Re: AW: [5.0] Connector default configuration + connection
timeout
Fischer, Ilona a écrit :


Protection against DOS attack should also have some glues like :

- Max clients from the same IP (ie DOS attack from the same host).
How would you distinguish between users coming from big proxy and a
DOS-attack? Our access.logs shows that approx. 30% acesses 
comes from

t-online (a big ISP in germany) and AOL
That's right (T-ONLINE and AOL) are big hackers ;)

More seriously, in such case there must be the original IP somewhere
in the HTTP header isn't it ?
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

--
==
Stefanos Karasavvidis
Electronic & Computer Engineer
e-mail : [EMAIL PROTECTED]
Multimedia Systems Center S.A.
Kissamou 178
73100 Chania - Crete - Hellas
http://www.msc.gr
Tel : +30 2821 0 88447
Fax : +30 2821 0 88427
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: mod_rewrite & mod_jk2

[Stefanos Karasavvidis]

for mod_jk you have to set
JkOptions ForwardURICompat
into httpd.conf and have mod_rewrite load before mod_jk
I have no idea if this option is present in mod_jk2
Note however that this brakes the spec because the servelts/jsps etc get 
as requested uri the rewritten part and not the original issued by the 
browser!

Nicolas wrote:

hi

first: i did post to tomcat user-first > got reply to check out tomcat-dev

is mod_rewrite supported ?

example:
RewriteEngine  on
RewriteRule^/test$ /examples/servlets/index.html
JkSet uri:/examples/*.group ajp13:unixsocket
the request /test tries to access /examples/servlets/index.html in 
document Root

any solution or not implemented ?

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

--
==
Stefanos Karasavvidis
Electronic & Computer Engineer
e-mail : [EMAIL PROTECTED]
Multimedia Systems Center S.A.
Kissamou 178
73100 Chania - Crete - Hellas
http://www.msc.gr
Tel : +30 2821 0 88447
Fax : +30 2821 0 88427
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Whee!! It's good to be back!

[Stefanos Karasavvidis]

How about calling
out.close()
Stefanos

Steve Burrus wrote:

*It's very good indeed to be back in your "midst" again, and believe 
me, I will try hard to "mind my manners" and try to use this list for 
the most knowledgeable insight that I can posssibly get about how to 
execute certain jsp's/servlets with the Tomcat web container!!*

*Case in point: I am having problems with trying to see this 
"TodayServlet.java" servlet of mine. It just flat doesn't show up in 
my browser when I try to look at it, sad to say. It is just the most 
simple and basic servlet, so it shouldn't be too much of a "big deal" 
to correct whatever problem it has, so I could then see it. I 
naturally have attached it to my posting, and again, it's good to be 
back "in your good graces" again. *

*p.s.: excuse my absolute ignorance about how servlet creation is done 
successfully, but I was wondering if I possibly need a HTML file to go 
along with this servlet as a helper file.*



import java.io.*;
import java.util.Date;
import javax.servlet.*;
import javax.servlet.http.*;
public class TodayServlet extends HttpServlet {

  public void doGet( HttpServletRequest request, 
 HttpServletResponse response )
throws IOException, ServletException {
 response.setContentType( "text/HTML" );
 response.setHeader( "Pragma", "no cache" ); 
 response.setHeader( "Cache-Control", "no cache" );
 response.setHeader( "Expires", "0" );
 PrintWriter out = response.getWriter();
 out.println( "" );
 out.println( "" );
 out.println( "Today" );
 out.println( "" );
 out.println( "" );
out.println("The Date and Time, if u didn't know,is:" );
 Date today = new Date(); 
 out.println( "" + today + "" );
 out.flush();
 out.println( "" );
 out.println( "" );
  }
}
 



http://java.sun.com/j2ee/dtds/web-app_2.2.dtd";>



How to program servlet examples 
  


   this is the web app. in which we show our servlets/jsp's. 



   today
   TodayServlet


   today
   /today


 



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [TC 4.0] How can I obtain the state of the tomcat server?

[Stefanos Karasavvidis]

how about checking a url of the server unitl it gives a 200 response code??

Stefanos

Evans, Terry G wrote:

>I need to know when the server is "available for work".
>
>Our product does not use startup servlets nor context listeners.
>
>We display simple HTML documents on a browser once the Tomcat
>server has started.
>
>My ideal solution would be that my Java application would fire off
>the Tomcat server (we're using Runtime.getRuntime().exec() calls
>to start Tomcat and IE) and then would either check the status of
>the server or receive a callback from the server that it is ready,
>then my Java application would proceed with starting up the browser.
>
>I was hoping to use an internal tomcat feature to implement this.
>
>I've looked at CatalinaManager and CatalinaService, but I'm not
>certain of exactly what they are and what they could do for my
>situation.
>
>Thanks,
>Terry
>
>
>-Original Message-
>From: Shapira, Yoav [mailto:[EMAIL PROTECTED]]
>Sent: Thursday, September 05, 2002 11:22 AM
>To: Tomcat Developers List
>Subject: RE: [TC 4.0] How can I obtain the state of the tomcat server?
>
>
>Hi,
>"The state of the server process" can be many things.  It can be the
>server process is alive, it can be that the host/port respond to
>requests, and it can be that your app is actually available for work.
>All of these can be checked in various ways.
>
>Which one were you interested in?  What would be your ideal solution?
>
>One idea is, if you have any startup servlets or context listeners, have
>them write our a file in a specific location or a message to some log.
>Then have the process that starts the web browser poll, checking for the
>existence of this file or this message in the log.
>
>Another one is to use a 3rd tool, e.g. a JMS server, where you could
>send a message from your server to a queue, and have your java app wait
>for that message before starting the browser...
>
>You get my drift: there are many many ways to do this ;)  Were you
>looking for an internal tomcat feature?
>
>Yoav Shapira
>Millennium ChemInformatics
>
>
>  
>
>>-Original Message-
>>From: Evans, Terry G [mailto:[EMAIL PROTECTED]]
>>Sent: Thursday, September 05, 2002 11:01 AM
>>To: '[EMAIL PROTECTED]'
>>Subject: [TC 4.0] How can I obtain the state of the tomcat server?
>>
>>I am developing a Java application that is designed to start up a
>>
>>
>Tomcat
>  
>
>>server and a web browser.  I don't want the web
>>browser to start up until the Tomcat server is running, but I don't
>>
>>
>want to
>  
>
>>put in an arbitrary sleep time between starting
>>the server and starting the web browser.  Is there a way that I can
>>
>>
>query
>  
>
>>the state of the server process?
>>
>>OS : Windows 2000 Professional
>>JDK/JRE : 1.4.0_01
>>Tomcat : 4.0
>>
>>Thanks for your time,
>>Terry
>>
>>--
>>To unsubscribe, e-mail:   >[EMAIL PROTECTED]>
>>For additional commands, e-mail: >[EMAIL PROTECTED]>
>>
>>
>
>
>--
>To unsubscribe, e-mail:   
>For additional commands, e-mail: 
>
>
>  
>


--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: Proposal draft for Tomcat 5.0

[Stefanos Karasavvidis]

> The major goals for Apache Tomcat 5.0 are to:
> - have simpler/cleaner code, so more people can get involved

Although not Tomcat 5 specific, I would add the following
- have good documentation, so more people can use it without going through the dev list
- have good documentation, so more people can get involved 

Stefanos Karasavvidis



--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>