Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
Thanks Craig, I'm in the middle of some tricky development with Cocoon at the moment and I don't wont to risk breaking my environment just at this instant. I hadn't consciously had a problem with this on Tomcat (only specifically within the internal ClassLoaders of Cocoon). I only mentioned it because the bug was one of these 'hard to pinpoint' ones that meant that some people's setups seemed to work and others didn't for no apparent reason. It stuck me that Tomcat was likely to be doing similar stuff, and it was worth looking for incase it was a problem for someone else. Hopefully some folk's setups will magically work now. I will update my CVS copy soon. Thanks again for following this up so quickly - I was worried this morning that I might have lead you on a wild goose chase! Festive greetings from Scotland, Stuart. On Friday, December 22, 2000, at 06:07 PM, Craig R. McClanahan wrote: > Stuart Roebuck wrote: > > > Craig, > > > > Here's one step towards what you were asking for. It's a standalone test case of >the > problem. I've written this because, judging from some of the issues that were >occuring > with Cocoon, I have a feeling that this problem is System/Java version dependent. >The > problem has been seen on Solaris, MacOS X and Windows with versions 1.2.2 and 1.3, >but it > may be only in certain combinations. > > > > Thanks for the test case. I've got equivalent behavior to yours (classloader 1 >fails, > classloader2 succeeds) on: > * Linux, Sun JDK 1.2.2 and Sun JDK 1.3.0 > * Win98, Sun JDK 1.2.2 and Sun JDK 1.3.0 > > > If this test case reproduces the problem on your system, then were going in the >right > direction. If not, then we'll never produce a test case for you to be able to test. > > > > Yesterday, I committed the Tomcat 4.0 changes to use getCanonicalPath() instead of > getAbsolutePath() in all the relevant cases. Looking at what we're seeing, I would >bet > you were certain to have problems (on the affected platforms) before if your > CATALINA_HOME had "../" elements, or if the document root of a particular web app >had > them. > > Have you had a chance to try the updated code yet? The nightly build from last >night is > probably the easiest way. > > Craig > - Stuart Roebuck [EMAIL PROTECTED] Lead Developer Mac OS X, Java, XML, etc. ADOLOS http://www.adolos.com/
Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
on 12/22/2000 12:25 PM, "Craig R. McClanahan" <[EMAIL PROTECTED]> wrote: > Are you accessing the properties file via ServletContext.getResource() or > ServletContext.getResourceAsStream()? Neither actually. We are using JServ's old code for loading properties so it is still direct file/io access. Yes, I know this breaks the idea of compressed WAR files, but for now we still require leaving them uncompressed anyway because of log files are stored in directories and such. /me thinks the idea of closed WAR files was lame anyway. > If so, it should be accessed with a > standard context-relative path like > "/WEB-INF/conf/TurbineResources.properties". The path normalization code in > Tomcat does this to your "./" anyway, so the "." should actually be redundant > (unless you depend on it for some application specific reason). Right. Well, it was originally "WEB-INF/." and that worked fine before and now it doesn't. :-) thanks, -jon -- Honk if you love peace and quiet.
Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
Jon Stevens wrote: > fyi, the difference is that I had to add a "./" to the front of a parameter > to a file that I was passing in... > > > properties > > > ./WEB-INF/conf/TurbineResources.properties > > > > I didn't have to have that before. > > -jon Are you accessing the properties file via ServletContext.getResource() or ServletContext.getResourceAsStream()? If so, it should be accessed with a standard context-relative path like "/WEB-INF/conf/TurbineResources.properties". The path normalization code in Tomcat does this to your "./" anyway, so the "." should actually be redundant (unless you depend on it for some application specific reason). Craig
Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
fyi, the difference is that I had to add a "./" to the front of a parameter to a file that I was passing in... properties ./WEB-INF/conf/TurbineResources.properties I didn't have to have that before. -jon
Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
Just so you know, these recent changes to correct the paths has screwed up Turbine. :-) I was depending on a path to have a ./ in it and now that is removed and therefore, Turbine is broken because the paths don't get generated properly. I'm not complaining because I should probably fix it in Turbine as well (and will do so ASAP), but I'm just letting you know that these recent changes do in fact affect other applications. :-) -jon
Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
Stuart Roebuck wrote: > Craig, > > Here's one step towards what you were asking for. It's a standalone test case of >the problem. I've written this because, judging from some of the issues that were >occuring with Cocoon, I have a feeling that this problem is System/Java version >dependent. The problem has been seen on Solaris, MacOS X and Windows with versions >1.2.2 and 1.3, but it may be only in certain combinations. > Thanks for the test case. I've got equivalent behavior to yours (classloader 1 fails, classloader2 succeeds) on: * Linux, Sun JDK 1.2.2 and Sun JDK 1.3.0 * Win98, Sun JDK 1.2.2 and Sun JDK 1.3.0 > If this test case reproduces the problem on your system, then were going in the >right direction. If not, then we'll never produce a test case for you to be able to >test. > Yesterday, I committed the Tomcat 4.0 changes to use getCanonicalPath() instead of getAbsolutePath() in all the relevant cases. Looking at what we're seeing, I would bet you were certain to have problems (on the affected platforms) before if your CATALINA_HOME had "../" elements, or if the document root of a particular web app had them. Have you had a chance to try the updated code yet? The nightly build from last night is probably the easiest way. Craig
Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
Craig,
Here's one step towards what you were asking for. It's a standalone test case of the
problem. I've written this because, judging from some of the issues that were
occuring with Cocoon, I have a feeling that this problem is System/Java version
dependent. The problem has been seen on Solaris, MacOS X and Windows with versions
1.2.2 and 1.3, but it may be only in certain combinations.
If this test case reproduces the problem on your system, then were going in the right
direction. If not, then we'll never produce a test case for you to be able to test.
Here's the code:
import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;
public class Test {
public static void main(String args[]) {
System.out.println("STARTED");
try {
String filename = "jac.zip";
String filepath2 = "/Users/stuart/";
String filepath3 = "Documents/../temp/";
String filepath = filepath2 + filepath3;
System.out.println("filepath = " + filepath);
System.out.println("filepath2 = " + filepath2);
System.out.println("filepath3 = " + filepath3);
System.out.println("filename = " + filename);
File file1 = new File(filepath, filename);
System.out.println("file1 = new File(filepath, filename) = " +
file1.toString());
System.out.println("file1.getAbsolutePath() = " + file1.getAbsolutePath());
System.out.println("file1.getCanonicalPath() = " +
file1.getCanonicalPath());
URL url1 = new URL("file:" + filepath);
URL url2 = new URL("file", null, -1, filepath);
URL url3 = new URL("file", null, filepath);
URL url4 = new URL(new URL("file:"+filepath2), filepath3);
System.out.println("url1 = new URL(\"file:\" + filepath) = " + url1);
System.out.println("url2 = new URL(\"file:\", null, -1, filepath) = " +
url2);
System.out.println("url3 = new URL(\"file:\", null, filepath) = " + url3);
System.out.println("url4 = new URL(new URL(\"file:\"+filepath2),
filepath3) = " + url4);
//URL inUrls [] = {url1, url2, url3, url4};
URL inUrls1 [] = {url2,url3};
URL inUrls2 [] = {url1,url4};
URLClassLoader classLoader1 = new URLClassLoader(inUrls1);
URLClassLoader classLoader2 = new URLClassLoader(inUrls2);
URL outUrls1 [] = classLoader1.getURLs();
URL outUrls2 [] = classLoader2.getURLs();
for (int i = 0; i < outUrls1.length; i++) {
System.out.println("outUrls1["+i+"] = " + outUrls1[i]);
}
for (int i = 0; i < outUrls2.length; i++) {
System.out.println("outUrls2["+i+"] = " + outUrls2[i]);
}
System.out.println("Resource found by classloader1 = " +
classLoader1.findResource(filename));
System.out.println("Resource found by classloader2 = " +
classLoader2.findResource(filename));
} catch (Exception e) {
System.out.println(e);
}
System.exit(0);
}
}
Please substitute in appropriate values for filepath2, filepath3 and filename.
filepath3 should contain a "../" element, filepath + filename should point to a real
file on your system.
My test results are:
STARTED
filepath = /Users/stuart/Documents/../temp/
filepath2 = /Users/stuart/
filepath3 = Documents/../temp/
filename = jac.zip
file1 = new File(filepath, filename) = /Users/stuart/Documents/../temp/jac.zip
file1.getAbsolutePath() = /Users/stuart/Documents/../temp/jac.zip
file1.getCanonicalPath() = /Users/stuart/temp/jac.zip
url1 = new URL("file:" + filepath) = file:/Users/stuart/temp/
url2 = new URL("file:", null, -1, filepath) = file:/Users/stuart/Documents/../temp/
url3 = new URL("file:", null, filepath) = file:/Users/stuart/Documents/../temp/
url4 = new URL(new URL("file:"+filepath2), filepath3) = file:/Users/stuart/temp/
outUrls1[0] = file:/Users/stuart/Documents/../temp/
outUrls1[1] = file:/Users/stuart/Documents/../temp/
outUrls2[0] = file:/Users/stuart/temp/
outUrls2[1] = file:/Users/stuart/temp/
Resource found by classloader1 = null
Resource found by classloader2 = file:/Users/stuart/temp/jac.zip
Stuart.
On Thursday, December 21, 2000, at 06:56 PM, Craig R. McClanahan wrote:
>
>
> Stuart Roebuck wrote:
>
> > In the course of fixing a problem I was having getting Apache Cocoon to run, I
>came across a
> bug in Java in the File.toURL() method. This fault, combined with the use of the
> URLClassLoader resulted in a classloading issue.
> >
>
> Stuart,
>
> I'm trying to create a simple, reproducible test case that causes Tomcat 4.0 to fail
> because of this -- but so far, I have not been able to.
>
> For example, I would assume that the following scenario should fail due to this
>problem:
> * Run on Linux (RH 6.2) + JDK 1.2.2
> * Do *not* set the CA
Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
Stuart Roebuck wrote: > In the course of fixing a problem I was having getting Apache Cocoon to run, I came >across a bug in Java in the File.toURL() method. This fault, combined with the use >of the URLClassLoader resulted in a classloading issue. > Stuart, I'm trying to create a simple, reproducible test case that causes Tomcat 4.0 to fail because of this -- but so far, I have not been able to. For example, I would assume that the following scenario should fail due to this problem: * Run on Linux (RH 6.2) + JDK 1.2.2 * Do *not* set the CATALINA_HOME environment variable * Set current directory to where Tomcat 4.0 is installed * Start it by typing "./bin/catalina.sh start". This would cause CATALINA_HOME (and therefore the "catalina.home" system property) to be set to "./bin/..", which would cause all the URLs in the "system" class loader -- the one created from JAR files in the "bin" directory -- to have absolute pathnames with ".." in them, which should trigger this problem. Yet, it still runs correctly. Can you help me identify a failure case? Craig
Re: [T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
Stuart Roebuck wrote: > In the course of fixing a problem I was having getting Apache Cocoon to run, I came >across a bug in Java in the File.toURL() method. This fault, combined with the use >of the URLClassLoader resulted in a classloading issue. > Thank you Stuart!!! I will be reviewing and fixing the cases you've identified soon. Craig
[T4][Classloaders][PATCH Suggestion] Bug in toURL() and new URL(x,x,x,x)
In the course of fixing a problem I was having getting Apache Cocoon to run, I came
across a bug in Java in the File.toURL() method. This fault, combined with the use of
the URLClassLoader resulted in a classloading issue.
As a result of finding this bug I thought I would check through the Tomcat 4
sourcecode and see if there may be the potential for a similar problem.
Consequently I have found a few points in the code which may be at risk:
catalina/src/share/org/apache/catalina/startup/Bootstrap.java:172:URL
url = new URL("file", null, file.getAbsolutePath());
catalina/src/share/org/apache/catalina/startup/Bootstrap.java:204:URL
url = new URL("file", null,
catalina/src/share/org/apache/catalina/startup/Bootstrap.java:228:URL
url = new URL("file", null, file.getAbsolutePath());
catalina/src/share/org/apache/catalina/startup/Bootstrap.java:269:URL
url = new URL("file", null, file.getAbsolutePath());
catalina/src/share/org/apache/catalina/startup/ContextConfig.java:841:
workURL = new URL("file", null, workDir.getAbsolutePath());
catalina/src/share/org/apache/catalina/startup/HostConfig.java:293:
URL url = new URL("file", null, dir.getAbsolutePath());
catalina/src/share/org/apache/catalina/startup/HostConfig.java:316:
URL url = new URL("file", null, dir.getAbsolutePath());
tester/src/tester/org/apache/tester/TestClient.java:299:URL url = new
URL("http", host, port, request);
In particular the Bootstrap.java file makes a number of calls for classloader
purposes, which could be suffering the same problems as cocoon was.
Now the detail:
The three and four parameter constructors for java.net.URL do *not* normalize
(canonicalize?) filepaths when they generate a URL instance. All other URL
constructors do. So:
new URL("file:/Users/stuart/Documents/../temp/").toString() =
"file:/Users/stuart/temp/"
but
new URL("file", null, "/Users/stuart/Documents/../temp/").toString() =
"file:/Users/stuart/Documents/../temp/"
Presumably one of these versions of the URL constructor is used by
java.io.File.toURL() which also demonstrates the same symptom.
Finally, File.getAbsolutePath() does not normalize the pathname.
File.getCanonicalPath() does, and therefore returns an absolute path at the same time.
The URLClassLoader requires normalized URLs. If you add two identical non-normalized
URLs to a URLClassLoader it accepts them both as different. If you add two identical
normalized URLs, getURLs() returns a single entry. These problems can manifest
themselves in a failure to load requested classes.
In a number of places in Bootstrap.jar it appears that the three parameter form of the
URL() constructor is being used with a non-normalized absolute path parameter. I am
not familiar with the code to be absolutely sure, but I thing these should be changed
so that all the getAbsolutePath() calls are replaced by getCanonicalPath().
I hope this is helpful, please contact me if you require further information.
Stuart.
-
Stuart Roebuck [EMAIL PROTECTED]
Lead Developer Mac OS X, Java, XML, etc.
ADOLOS http://www.adolos.com/
