Github user countmdm commented on the issue:
https://github.com/apache/spark/pull/21456
If we don't do normalization ourselves, we may potentially run into the
following:
path = ... // Produces "foo//bar"
path = path.intern(); // Ok, no separate copies of "foo//bar" anymore
File f = new Fille(path);
// Internally, code in File() constructor looks at the given path.
// If path is not in the normalized form, it normalizes it, producing a new
stirng "foo/bar"
System.out.println(f.getPath()); // Prints "foo/bar"
Since the code inside java.io.File doesn't do any string interning, we will
keep canonicalizing the "foo//bar" string, but then java.io.File will still
generate multiple copies of "foo/bar".
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
