Were you able to verify that performing similar operations outside of JVM produces better results? What gives me pause in your description is that you're doing this over a network, which has to be slower than local disk access. Note that the bug files with Sun you linked to talks about all file access, not networked access, and it seems from the comments, it mostly comes down to FAT32 file systems (think USB devises).
I ran your algorithm against directory trees both locally and over a network. Predictably, local file traversal was faster than networked one, but it wasn't as though I was getting an unacceptable performance in either case. Also note that tools like Ant and Maven likely use Java's standard IO libraries and are routinely asked to do batch operations on files and I don't often hear people complaining about that being the bottleneck. Not saying you aren't experiencing a performance issue, but maybe you should look at your network performance as well as compare JVM's times against a non-JVM program. Alexey ________________________________ From: John Muir <[email protected]> To: The Java Posse <[email protected]> Sent: Wed, January 6, 2010 8:59:22 AM Subject: [The Java Posse] Bad performance getting file attributes over the network Hi Folks, I have recently run into the problem that when recursively iterating through folders over a network, the performance is terrible. It's 30-50 times slower than when the folders are on the local machine. I've put a small test program at the end of this post - just try it out with a large/deep folder both locally and on a network drive. The culprit in this case is the call to isDirectory(), but any attempt to question any file attribute (e.g. lastModified()) results in the bad performance. And you really must use either isDirectory() or isFile () before doing something with the File[] that listFiles() returned. The problem is not unknown to Sun (http://bugs.sun.com/bugdatabase/ view_bug.do?bug_id=6483858) but it's been ignored for JSR-203 to solve - sometime! Does anyone have a workaround for this problem without invoking JNI or using native code? And does anyone know when JSR-203 will be implemented? Java 7? John <pre> import java.io.*; import java.util.*; public class TstListFiles { static int listPath(File path) { int ct = 0; File[] files = path.listFiles(); for (int i = 0, n = files.length; i < n; i++) { ct++; // long l = files[i].lastModified(); // boolean b = files[i].isDirectory(); boolean b = files[i].isFile(); if (files[i].isDirectory()) { ct += listPath(files[i]); } } return ct; } public static void main(String args[]) { for (int i=0; i < args.length; i++) { long start = System.currentTimeMillis(); int numFiles = listPath(new File(args[i])); System.out.println(args[i] + " has " + numFiles + " files/ folders and took " + (System.currentTimeMillis()-start) + "ms"); } } } </pre> -- You received this message because you are subscribed to the Google Groups "The Java Posse" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.--
You received this message because you are subscribed to the Google Groups "The Java Posse" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to [email protected].
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.
