RE: URI handling in tomcat 3.2.3
DOH! I don't believe I did that. I cutpasted the URI from the wrong spot. Here's the real trace. Note that Apache 1.3.19 does decode the %2f characters but now returns a 404 error instead of sending the decoded string as PATH_INFO. I've attached two traces, one that demonstrates a working PATH_INFO test and a second that uses %2f and fails. --- GET /cgi-bin/dumpenv.bat/pathinfo COMSPEC=C:\WINNT\system32\cmd.exe DOCUMENT_ROOT=c:/program files/apache group/apache1.3.19/htdocs PATH=C:\Apropos\Sybase\Adaptive Server Anywhere 6.0\win32;C:\Apropos\java\bin;C: \Apropos\java\bin\classic;C:\Apropos\bin;C:\Apropos\tcl\bin;c:\java\jdk1.4\b in;c :\java\jdk1.4\bin;D:\Apropos\Sybase\Adaptive Server Anywhere 6.0\win32;C:\Progra m Files\Sybase\Adaptive Server Anywhere 6.0\win32\win32;C:\WINNT\system32;C:\WIN NT;C:\WINNT\System32\Wbem;C:\ClearCase\bin;.;C:\Program Files\Symantec\pcAnywher e;c:\tools\rebol;C:\Program Files\Support Tools\;C:\IMNnq_NT;C:\PROGRA~1\TCL\bin PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.JS;.WS PROMPT=$P$G REMOTE_ADDR=127.0.0.1 REMOTE_PORT=1269 SCRIPT_FILENAME=c:/program files/apache group/apache1.3.19/cgi-bin/dumpenv.bat SERVER_ADDR=127.0.0.1 [EMAIL PROTECTED] SERVER_NAME=msaegesserlpt SERVER_PORT=8081 SERVER_SIGNATURE=ADDRESSApache/1.3.19 Server at msaegesserlpt Port 8081/ADDRE SS SERVER_SOFTWARE=Apache/1.3.19 (Win32) mod_jk SystemRoot=C:\WINNT WINDIR=C:\WINNT GATEWAY_INTERFACE=CGI/1.1 SERVER_PROTOCOL=HTTP/0.9 REQUEST_METHOD=GET QUERY_STRING= REQUEST_URI=/cgi-bin/dumpenv.bat/pathinfo SCRIPT_NAME=/cgi-bin/dumpenv.bat PATH_INFO=/pathinfo PATH_TRANSLATED=c:\program files\apache group\apache1.3.19\htdocs\pathinfo Connection to host lost. --- --- GET /cgi-bin/dumpenv.bat/http:%2f%2ffubar !DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN HTMLHEAD TITLE404 Not Fou nd/TITLE /HEADBODY H1Not Found/H1 The requested URL /cgi-bin/dumpenv.bat/ http://fubar was not found on this server.P HR ADDRESSApache/1.3.19 Server a t msaegesserlpt Port 8081/ADDRESS /BODY/HTML Connection to host lost. --- Marc Saegesser -Original Message- From: Jason Hunter [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 20, 2001 5:13 PM To: [EMAIL PROTECTED] Subject: Re: URI handling in tomcat 3.2.3 You only use http:// in the GET request if you're talking to a proxy server. That's probably why you got the bad request error, not because of the %2f. You should try: GET /cgi-bin/dumpenv.bat/fu%2fbar HTTP/1.1 -jh- Marc Saegesser wrote: Oops, minor correction. Apache actually returns 400 Bad Request Here's the telnet session GET http://localhost:8081/cgi-bin/dumpenv.bat/fu%2fbar HTTP/1.1 server: msaegesserlpt HTTP/1.1 400 Bad Request Date: Thu, 13 Sep 2001 13:51:21 GMT Server: Apache/1.3.19 (Win32) mod_jk Connection: close Transfer-Encoding: chunked Content-Type: text/html; charset=iso-8859-1 18b !DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN HTMLHEAD TITLE400 Bad Req uest/TITLE /HEADBODY H1Bad Request/H1 Your browser sent a request that th is server could not understand.P client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /cgi-bin/dumpenv.bat/fu%2fbarP HR ADDRESSApache /1.3.19 Server at msaegesserlpt Port 8081/ADDRESS /BODY/HTML 0 Connection to host lost. Marc Saegesser -Original Message- From: Marc Saegesser [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 8:48 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: RE: URI handling in tomcat 3.2.3 I agree that this URI handling sucks. I'm the one that committed the change that made it happen and I still think it sucks. However, allowing these encoded characters opens some very large security problems. Also, even if TC 3.2.x allowed these characters, the resulting URL wouldn't be portable because several other web servers impose the same restrictions. In fact we lifted our restriction on encoded special characters straight from Apache HTTPD. In your example URL, if /app/UCB was a CGI script then you would still get a 404 on Apache (at least on Apache 1.3.19 which is what I tested with). If you need to pass this sort of data to a servlet (or CGI) the most portable way
RE: URI handling in tomcat 3.2.3
Jason, Lars, all, I've been investigating the normalized URL, encoded special characters, etc. issues with Tomcat 3.2.3 and I think I have a solution that both maintains the required level of security and allows the kind of thing that Lars and Jason are looking for. I've attached a patch that I would appreciate reviewed by committers to make sure I'm not missing something. What I've done is changed the existing RequestUtil.URLDecode() to prevent the decoding of some additional characters. There was already code there to prevent '/' and '\0' and I added the other protected chars '\\', '.' and '%'. I then removed the draconian test to 404 any URL containing these special chars. If a URI contains encoded special characters from this list then these characters *remain encoded* in the resulting URI. For example /fu/ba+r -- /fu/ba r /fu%2fba+r -- /fu%2fba r I've tested this with all the 'bad' URLs that used to expose JSP source or avoid security constraints and everything works fine. Tomcat will still normalize all URIs as soon as they arrive in the container (sorry Jason). To avoid normalizing data passed in the pathinfo you will need to encode the path characters and then have your servlet URL decode the path info. For example, assuming /fubar/* is the prefix mapping, http://server/fubar/http:%2f%2fURLInPathInfo or http://server/fubar/http:%2f/URLInPathInfo will return path info that can be decoded URLDecoder.decode() into http://URLInPathInfo. Comments? Marc Saegesser -Original Message- From: Marc Saegesser [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 12:49 PM To: [EMAIL PROTECTED] Subject: RE: URI handling in tomcat 3.2.3 Lars, I agree with you. These encoded characters should be allowed in URIs and disallowing them is a hack. Like I said, I think the approach sucks. We were faced with a very serious security problem that had to be addressed very quickly and the decision was made that it was better to forbid certain 'odd' URIs in order to guarantee that resources that the specification *requires* to be protected were indeed protected. We need to look into how to solve the security problems without forbidden valid URIs, but right now I'm about the only one around paying much attention to the tomcat_32 branch so I don't know what kind of time frame wold be involved to get this changed. I do know that the solution will not be trivial. I mentioned Apache httpd only to show that our approach to this problem is in-line with that taken by other industry leading products. We should not (and I think have not) blindly follow httpd (it does a few other things that disagree with). Patches or discussion on how to go about fixing this are certainly welcome! Marc Saegesser -Original Message- From: Lars Oppermann [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 9:23 AM To: [EMAIL PROTECTED] Subject: Re: URI handling in tomcat 3.2.3 Hi Marc, Thanks for you reply... Marc Saegesser wrote: I agree that this URI handling sucks. I'm the one that committed the change that made it happen and I still think it sucks. However, allowing these encoded characters opens some very large security problems. From what I understand, these security problems are all related to mapping URIs to filesystem paths. So how do you feel about doing this processing in the filesystem (default) servlet? Also, even if TC 3.2.x allowed these characters, the resulting URL wouldn't be portable because several other web servers impose the same restrictions. [400 with Apache 1.3.19] I think, if it is possible to do this in a better way while keeping up the protection there is no reason for copying the behaviour of httpd. However, if those security implications can not be handled in the file servlet like I mentioned before, I'd need to do some more thinking on this point. If you need to pass this sort of data to a servlet (or CGI) the most portable way is to simply use a query string. Yes, that sounds like a straight-forward solution. Unfortunatly the service that gets excuted here will access some document and return an html representation. This document contains relative references within the hierarchy represented by the 'wrapped' URI. for this to work with a browser, the request URI has to be used, because the client can not resolve relative references inside a query (why should it) I belive that there are many more use-cases where using URIs to represent hierachical names that do not map to files is desirable, espacialy in a servlet environment. In httpd, which's main work consists of serving file-system resources this might be viewed differently Cheers, Lars -- -- Lars Oppermann [EMAIL PROTECTED] Sun Microsystems Software Engineer - Sun ONE Webtop
Re: URI handling in tomcat 3.2.3
Marc, Thanks for still thinking about this. :-) And thanks to Lars for raising the interesting point from 2396 that a / has a reserved purpose in a URI and if you want to use it for a different purpose you should escape it. That makes it sound like your patch is a highly reasonable solution, and perhaps the best we can do until the spec can provide more guidance. -jh- Marc Saegesser wrote: Jason, Lars, all, I've been investigating the normalized URL, encoded special characters, etc. issues with Tomcat 3.2.3 and I think I have a solution that both maintains the required level of security and allows the kind of thing that Lars and Jason are looking for. I've attached a patch that I would appreciate reviewed by committers to make sure I'm not missing something. What I've done is changed the existing RequestUtil.URLDecode() to prevent the decoding of some additional characters. There was already code there to prevent '/' and '\0' and I added the other protected chars '\\', '.' and '%'. I then removed the draconian test to 404 any URL containing these special chars. If a URI contains encoded special characters from this list then these characters *remain encoded* in the resulting URI. For example /fu/ba+r -- /fu/ba r /fu%2fba+r -- /fu%2fba r I've tested this with all the 'bad' URLs that used to expose JSP source or avoid security constraints and everything works fine. Tomcat will still normalize all URIs as soon as they arrive in the container (sorry Jason). To avoid normalizing data passed in the pathinfo you will need to encode the path characters and then have your servlet URL decode the path info. For example, assuming /fubar/* is the prefix mapping, http://server/fubar/http:%2f%2fURLInPathInfo or http://server/fubar/http:%2f/URLInPathInfo will return path info that can be decoded URLDecoder.decode() into http://URLInPathInfo. Comments? Marc Saegesser -Original Message- From: Marc Saegesser [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 12:49 PM To: [EMAIL PROTECTED] Subject: RE: URI handling in tomcat 3.2.3 Lars, I agree with you. These encoded characters should be allowed in URIs and disallowing them is a hack. Like I said, I think the approach sucks. We were faced with a very serious security problem that had to be addressed very quickly and the decision was made that it was better to forbid certain 'odd' URIs in order to guarantee that resources that the specification *requires* to be protected were indeed protected. We need to look into how to solve the security problems without forbidden valid URIs, but right now I'm about the only one around paying much attention to the tomcat_32 branch so I don't know what kind of time frame wold be involved to get this changed. I do know that the solution will not be trivial. I mentioned Apache httpd only to show that our approach to this problem is in-line with that taken by other industry leading products. We should not (and I think have not) blindly follow httpd (it does a few other things that disagree with). Patches or discussion on how to go about fixing this are certainly welcome! Marc Saegesser -Original Message- From: Lars Oppermann [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 9:23 AM To: [EMAIL PROTECTED] Subject: Re: URI handling in tomcat 3.2.3 Hi Marc, Thanks for you reply... Marc Saegesser wrote: I agree that this URI handling sucks. I'm the one that committed the change that made it happen and I still think it sucks. However, allowing these encoded characters opens some very large security problems. From what I understand, these security problems are all related to mapping URIs to filesystem paths. So how do you feel about doing this processing in the filesystem (default) servlet? Also, even if TC 3.2.x allowed these characters, the resulting URL wouldn't be portable because several other web servers impose the same restrictions. [400 with Apache 1.3.19] I think, if it is possible to do this in a better way while keeping up the protection there is no reason for copying the behaviour of httpd. However, if those security implications can not be handled in the file servlet like I mentioned before, I'd need to do some more thinking on this point. If you need to pass this sort of data to a servlet (or CGI) the most portable way is to simply use a query string. Yes, that sounds like a straight-forward solution. Unfortunatly the service that gets excuted here will access some document and return an html representation. This document contains relative references within the hierarchy represented by the 'wrapped' URI. for this to work with a browser, the request URI has to be used, because the client can not resolve relative references
Re: URI handling in tomcat 3.2.3
You only use http:// in the GET request if you're talking to a proxy server. That's probably why you got the bad request error, not because of the %2f. You should try: GET /cgi-bin/dumpenv.bat/fu%2fbar HTTP/1.1 -jh- Marc Saegesser wrote: Oops, minor correction. Apache actually returns 400 Bad Request Here's the telnet session GET http://localhost:8081/cgi-bin/dumpenv.bat/fu%2fbar HTTP/1.1 server: msaegesserlpt HTTP/1.1 400 Bad Request Date: Thu, 13 Sep 2001 13:51:21 GMT Server: Apache/1.3.19 (Win32) mod_jk Connection: close Transfer-Encoding: chunked Content-Type: text/html; charset=iso-8859-1 18b !DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN HTMLHEAD TITLE400 Bad Req uest/TITLE /HEADBODY H1Bad Request/H1 Your browser sent a request that th is server could not understand.P client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /cgi-bin/dumpenv.bat/fu%2fbarP HR ADDRESSApache /1.3.19 Server at msaegesserlpt Port 8081/ADDRESS /BODY/HTML 0 Connection to host lost. Marc Saegesser -Original Message- From: Marc Saegesser [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 8:48 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: RE: URI handling in tomcat 3.2.3 I agree that this URI handling sucks. I'm the one that committed the change that made it happen and I still think it sucks. However, allowing these encoded characters opens some very large security problems. Also, even if TC 3.2.x allowed these characters, the resulting URL wouldn't be portable because several other web servers impose the same restrictions. In fact we lifted our restriction on encoded special characters straight from Apache HTTPD. In your example URL, if /app/UCB was a CGI script then you would still get a 404 on Apache (at least on Apache 1.3.19 which is what I tested with). If you need to pass this sort of data to a servlet (or CGI) the most portable way is to simply use a query string. Marc Saegesser -Original Message- From: Lars Oppermann [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 5:00 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: URI handling in tomcat 3.2.3 Hi everyone, we were in progress of moving our project to tomcat 3.2.3 when we came accross the new handling of URIs (release-notes sec. 7.2). Since we are using the URI to transport other hierarchical information then filesystem paths, we have the feeling, that this kind of functionality belongs to the default servlet serving filesystem requests. Especialy the fact that %25, %2E, %2F and %5c inside an URI lead to a 404 error seems to somewhat strange. For Example: http://server/app/UCB/vnd.sun.star.hier:%2F/address/myresource would be rejected, before app has teh possibilty to look at the request and ...hier://address/myfile... would be normalized to hier:/address. We are perfectly aware of the security concerns behind these changes. However, they only apply when serving resources from the filesystem. A URL's path-components however are in no way bound to the representaion of filesystem paths.(After all, the U in URL stands for universal :) RFC 2396 states that '/' in an URI has another semantic meaning then %2F in an URI. The '/' seperates path-components, while the %2F means a slash character in a path-component. When such an URI is mapped to a filesystem this would denote a filename that contains a slash. When the system does not allow for such names, it is the responsebilty of the filesystem servlet to report an error (404 since such a file must not exist on unix for example). What are your opinions on this? Cheers -Lars -- -- Lars Oppermann [EMAIL PROTECTED] Sun Microsystems Software Engineer - Sun ONE Webtop Sachsenfeld 4 Phone: +49 40 23646 959D-20097 Hamburg Fax: +49 40 23646 550 http://www.sun.com/webtop
URI handling in tomcat 3.2.3
Hi everyone, we were in progress of moving our project to tomcat 3.2.3 when we came accross the new handling of URIs (release-notes sec. 7.2). Since we are using the URI to transport other hierarchical information then filesystem paths, we have the feeling, that this kind of functionality belongs to the default servlet serving filesystem requests. Especialy the fact that %25, %2E, %2F and %5c inside an URI lead to a 404 error seems to somewhat strange. For Example: http://server/app/UCB/vnd.sun.star.hier:%2F/address/myresource would be rejected, before app has teh possibilty to look at the request and ...hier://address/myfile... would be normalized to hier:/address. We are perfectly aware of the security concerns behind these changes. However, they only apply when serving resources from the filesystem. A URL's path-components however are in no way bound to the representaion of filesystem paths.(After all, the U in URL stands for universal :) RFC 2396 states that '/' in an URI has another semantic meaning then %2F in an URI. The '/' seperates path-components, while the %2F means a slash character in a path-component. When such an URI is mapped to a filesystem this would denote a filename that contains a slash. When the system does not allow for such names, it is the responsebilty of the filesystem servlet to report an error (404 since such a file must not exist on unix for example). What are your opinions on this? Cheers -Lars -- -- Lars Oppermann [EMAIL PROTECTED] Sun Microsystems Software Engineer - Sun ONE Webtop Sachsenfeld 4 Phone: +49 40 23646 959D-20097 Hamburg Fax: +49 40 23646 550 http://www.sun.com/webtop
RE: URI handling in tomcat 3.2.3
I agree that this URI handling sucks. I'm the one that committed the change that made it happen and I still think it sucks. However, allowing these encoded characters opens some very large security problems. Also, even if TC 3.2.x allowed these characters, the resulting URL wouldn't be portable because several other web servers impose the same restrictions. In fact we lifted our restriction on encoded special characters straight from Apache HTTPD. In your example URL, if /app/UCB was a CGI script then you would still get a 404 on Apache (at least on Apache 1.3.19 which is what I tested with). If you need to pass this sort of data to a servlet (or CGI) the most portable way is to simply use a query string. Marc Saegesser -Original Message- From: Lars Oppermann [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 5:00 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: URI handling in tomcat 3.2.3 Hi everyone, we were in progress of moving our project to tomcat 3.2.3 when we came accross the new handling of URIs (release-notes sec. 7.2). Since we are using the URI to transport other hierarchical information then filesystem paths, we have the feeling, that this kind of functionality belongs to the default servlet serving filesystem requests. Especialy the fact that %25, %2E, %2F and %5c inside an URI lead to a 404 error seems to somewhat strange. For Example: http://server/app/UCB/vnd.sun.star.hier:%2F/address/myresource would be rejected, before app has teh possibilty to look at the request and ...hier://address/myfile... would be normalized to hier:/address. We are perfectly aware of the security concerns behind these changes. However, they only apply when serving resources from the filesystem. A URL's path-components however are in no way bound to the representaion of filesystem paths.(After all, the U in URL stands for universal :) RFC 2396 states that '/' in an URI has another semantic meaning then %2F in an URI. The '/' seperates path-components, while the %2F means a slash character in a path-component. When such an URI is mapped to a filesystem this would denote a filename that contains a slash. When the system does not allow for such names, it is the responsebilty of the filesystem servlet to report an error (404 since such a file must not exist on unix for example). What are your opinions on this? Cheers -Lars -- -- Lars Oppermann [EMAIL PROTECTED] Sun Microsystems Software Engineer - Sun ONE Webtop Sachsenfeld 4 Phone: +49 40 23646 959D-20097 Hamburg Fax: +49 40 23646 550 http://www.sun.com/webtop
RE: URI handling in tomcat 3.2.3
Oops, minor correction. Apache actually returns 400 Bad Request Here's the telnet session GET http://localhost:8081/cgi-bin/dumpenv.bat/fu%2fbar HTTP/1.1 server: msaegesserlpt HTTP/1.1 400 Bad Request Date: Thu, 13 Sep 2001 13:51:21 GMT Server: Apache/1.3.19 (Win32) mod_jk Connection: close Transfer-Encoding: chunked Content-Type: text/html; charset=iso-8859-1 18b !DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN HTMLHEAD TITLE400 Bad Req uest/TITLE /HEADBODY H1Bad Request/H1 Your browser sent a request that th is server could not understand.P client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /cgi-bin/dumpenv.bat/fu%2fbarP HR ADDRESSApache /1.3.19 Server at msaegesserlpt Port 8081/ADDRESS /BODY/HTML 0 Connection to host lost. Marc Saegesser -Original Message- From: Marc Saegesser [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 8:48 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: RE: URI handling in tomcat 3.2.3 I agree that this URI handling sucks. I'm the one that committed the change that made it happen and I still think it sucks. However, allowing these encoded characters opens some very large security problems. Also, even if TC 3.2.x allowed these characters, the resulting URL wouldn't be portable because several other web servers impose the same restrictions. In fact we lifted our restriction on encoded special characters straight from Apache HTTPD. In your example URL, if /app/UCB was a CGI script then you would still get a 404 on Apache (at least on Apache 1.3.19 which is what I tested with). If you need to pass this sort of data to a servlet (or CGI) the most portable way is to simply use a query string. Marc Saegesser -Original Message- From: Lars Oppermann [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 5:00 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: URI handling in tomcat 3.2.3 Hi everyone, we were in progress of moving our project to tomcat 3.2.3 when we came accross the new handling of URIs (release-notes sec. 7.2). Since we are using the URI to transport other hierarchical information then filesystem paths, we have the feeling, that this kind of functionality belongs to the default servlet serving filesystem requests. Especialy the fact that %25, %2E, %2F and %5c inside an URI lead to a 404 error seems to somewhat strange. For Example: http://server/app/UCB/vnd.sun.star.hier:%2F/address/myresource would be rejected, before app has teh possibilty to look at the request and ...hier://address/myfile... would be normalized to hier:/address. We are perfectly aware of the security concerns behind these changes. However, they only apply when serving resources from the filesystem. A URL's path-components however are in no way bound to the representaion of filesystem paths.(After all, the U in URL stands for universal :) RFC 2396 states that '/' in an URI has another semantic meaning then %2F in an URI. The '/' seperates path-components, while the %2F means a slash character in a path-component. When such an URI is mapped to a filesystem this would denote a filename that contains a slash. When the system does not allow for such names, it is the responsebilty of the filesystem servlet to report an error (404 since such a file must not exist on unix for example). What are your opinions on this? Cheers -Lars -- -- Lars Oppermann [EMAIL PROTECTED] Sun Microsystems Software Engineer - Sun ONE Webtop Sachsenfeld 4 Phone: +49 40 23646 959D-20097 Hamburg Fax: +49 40 23646 550 http://www.sun.com/webtop
Re: URI handling in tomcat 3.2.3
While 3.3 has this behavior as the default, it can be disabled in the config by: DecodeInterceptor safe=false / Since the release is scheduled to happen by the end of the month, you might consider jumping straight to 3.3. - Original Message - From: Lars Oppermann [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Thursday, September 13, 2001 3:00 AM Subject: URI handling in tomcat 3.2.3 Hi everyone, we were in progress of moving our project to tomcat 3.2.3 when we came accross the new handling of URIs (release-notes sec. 7.2). Since we are using the URI to transport other hierarchical information then filesystem paths, we have the feeling, that this kind of functionality belongs to the default servlet serving filesystem requests. Especialy the fact that %25, %2E, %2F and %5c inside an URI lead to a 404 error seems to somewhat strange. For Example: http://server/app/UCB/vnd.sun.star.hier:%2F/address/myresource would be rejected, before app has teh possibilty to look at the request and ...hier://address/myfile... would be normalized to hier:/address. We are perfectly aware of the security concerns behind these changes. However, they only apply when serving resources from the filesystem. A URL's path-components however are in no way bound to the representaion of filesystem paths.(After all, the U in URL stands for universal :) RFC 2396 states that '/' in an URI has another semantic meaning then %2F in an URI. The '/' seperates path-components, while the %2F means a slash character in a path-component. When such an URI is mapped to a filesystem this would denote a filename that contains a slash. When the system does not allow for such names, it is the responsebilty of the filesystem servlet to report an error (404 since such a file must not exist on unix for example). What are your opinions on this? Cheers -Lars -- -- Lars Oppermann [EMAIL PROTECTED] Sun Microsystems Software Engineer - Sun ONE Webtop Sachsenfeld 4 Phone: +49 40 23646 959D-20097 Hamburg Fax: +49 40 23646 550 http://www.sun.com/webtop ** This message is intended only for the use of the person(s) listed above as the intended recipient(s), and may contain information that is PRIVILEGED and CONFIDENTIAL. If you are not an intended recipient, you may not read, copy, or distribute this message or any attachment. If you received this communication in error, please notify us immediately by e-mail and then delete all copies of this message and any attachments. In addition you should be aware that ordinary (unencrypted) e-mail sent through the Internet is not secure. Do not send confidential or sensitive information, such as social security numbers, account numbers, personal identification numbers and passwords, to us via ordinary (unencrypted) e-mail.
RE: URI handling in tomcat 3.2.3
Lars, I agree with you. These encoded characters should be allowed in URIs and disallowing them is a hack. Like I said, I think the approach sucks. We were faced with a very serious security problem that had to be addressed very quickly and the decision was made that it was better to forbid certain 'odd' URIs in order to guarantee that resources that the specification *requires* to be protected were indeed protected. We need to look into how to solve the security problems without forbidden valid URIs, but right now I'm about the only one around paying much attention to the tomcat_32 branch so I don't know what kind of time frame wold be involved to get this changed. I do know that the solution will not be trivial. I mentioned Apache httpd only to show that our approach to this problem is in-line with that taken by other industry leading products. We should not (and I think have not) blindly follow httpd (it does a few other things that disagree with). Patches or discussion on how to go about fixing this are certainly welcome! Marc Saegesser -Original Message- From: Lars Oppermann [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 9:23 AM To: [EMAIL PROTECTED] Subject: Re: URI handling in tomcat 3.2.3 Hi Marc, Thanks for you reply... Marc Saegesser wrote: I agree that this URI handling sucks. I'm the one that committed the change that made it happen and I still think it sucks. However, allowing these encoded characters opens some very large security problems. From what I understand, these security problems are all related to mapping URIs to filesystem paths. So how do you feel about doing this processing in the filesystem (default) servlet? Also, even if TC 3.2.x allowed these characters, the resulting URL wouldn't be portable because several other web servers impose the same restrictions. [400 with Apache 1.3.19] I think, if it is possible to do this in a better way while keeping up the protection there is no reason for copying the behaviour of httpd. However, if those security implications can not be handled in the file servlet like I mentioned before, I'd need to do some more thinking on this point. If you need to pass this sort of data to a servlet (or CGI) the most portable way is to simply use a query string. Yes, that sounds like a straight-forward solution. Unfortunatly the service that gets excuted here will access some document and return an html representation. This document contains relative references within the hierarchy represented by the 'wrapped' URI. for this to work with a browser, the request URI has to be used, because the client can not resolve relative references inside a query (why should it) I belive that there are many more use-cases where using URIs to represent hierachical names that do not map to files is desirable, espacialy in a servlet environment. In httpd, which's main work consists of serving file-system resources this might be viewed differently Cheers, Lars -- -- Lars Oppermann [EMAIL PROTECTED] Sun Microsystems Software Engineer - Sun ONE Webtop Sachsenfeld 4 Phone: +49 40 23646 959D-20097 Hamburg Fax: +49 40 23646 550 http://www.sun.com/webtop
RE: URI handling in tomcat 3.2.3
Be aware that setting this false will open Tomcat 3.3 to the vulnerability it is intended to prevent. Serving JSP source and bypassing security constraints are among the problems. Detecting potential URL trickery early in the handling provides the most reliable fix. Allowing unsafe URL's past this point means that many modules would need to be updated to detect the trickery. StaticInterceptor, which serves static resources, would only be the first, and perhaps easiest module to fix. Our assessment was that this approach wasn't practical to implement. Cheers, Larry -Original Message- From: Bill Barker [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 12:36 PM To: [EMAIL PROTECTED] Subject: Re: URI handling in tomcat 3.2.3 While 3.3 has this behavior as the default, it can be disabled in the config by: DecodeInterceptor safe=false / Since the release is scheduled to happen by the end of the month, you might consider jumping straight to 3.3. - Original Message - From: Lars Oppermann [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Thursday, September 13, 2001 3:00 AM Subject: URI handling in tomcat 3.2.3 Hi everyone, we were in progress of moving our project to tomcat 3.2.3 when we came accross the new handling of URIs (release-notes sec. 7.2). Since we are using the URI to transport other hierarchical information then filesystem paths, we have the feeling, that this kind of functionality belongs to the default servlet serving filesystem requests. Especialy the fact that %25, %2E, %2F and %5c inside an URI lead to a 404 error seems to somewhat strange. For Example: http://server/app/UCB/vnd.sun.star.hier:%2F/address/myresource would be rejected, before app has teh possibilty to look at the request and ...hier://address/myfile... would be normalized to hier:/address. We are perfectly aware of the security concerns behind these changes. However, they only apply when serving resources from the filesystem. A URL's path-components however are in no way bound to the representaion of filesystem paths.(After all, the U in URL stands for universal :) RFC 2396 states that '/' in an URI has another semantic meaning then %2F in an URI. The '/' seperates path-components, while the %2F means a slash character in a path-component. When such an URI is mapped to a filesystem this would denote a filename that contains a slash. When the system does not allow for such names, it is the responsebilty of the filesystem servlet to report an error (404 since such a file must not exist on unix for example). What are your opinions on this? Cheers -Lars -- -- Lars Oppermann [EMAIL PROTECTED] Sun Microsystems Software Engineer - Sun ONE Webtop Sachsenfeld 4 Phone: +49 40 23646 959 D-20097 Hamburg Fax: +49 40 23646 550 http://www.sun.com/webtop ** This message is intended only for the use of the person(s) listed above as the intended recipient(s), and may contain information that is PRIVILEGED and CONFIDENTIAL. If you are not an intended recipient, you may not read, copy, or distribute this message or any attachment. If you received this communication in error, please notify us immediately by e-mail and then delete all copies of this message and any attachments. In addition you should be aware that ordinary (unencrypted) e-mail sent through the Internet is not secure. Do not send confidential or sensitive information, such as social security numbers, account numbers, personal identification numbers and passwords, to us via ordinary (unencrypted) e-mail.
