Github user countmdm commented on the issue: https://github.com/apache/spark/pull/21456 If we don't do normalization ourselves, we may potentially run into the following: path = ... // Produces "foo//bar" path = path.intern(); // Ok, no separate copies of "foo//bar" anymore File f = new Fille(path); // Internally, code in File() constructor looks at the given path. // If path is not in the normalized form, it normalizes it, producing a new stirng "foo/bar" System.out.println(f.getPath()); // Prints "foo/bar" Since the code inside java.io.File doesn't do any string interning, we will keep canonicalizing the "foo//bar" string, but then java.io.File will still generate multiple copies of "foo/bar".
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org