Wei Zhong created FLINK-15248:
---------------------------------
Summary: FileUtils#compressDirectory behaves buggy when processing
relative directory path
Key: FLINK-15248
URL: https://issues.apache.org/jira/browse/FLINK-15248
Project: Flink
Issue Type: Bug
Components: FileSystems
Affects Versions: 1.10.0
Reporter: Wei Zhong
Fix For: 1.10.0
_FileUtils#compressDirectory_ behaves buggy when processing relative directory
path. If the path of target directory is a relative path, the relative path
inside the target zip file can not be constructed correctly:
{code:java}
public static Path compressDirectory(Path directory, Path target) throws
IOException {
FileSystem sourceFs = directory.getFileSystem();
FileSystem targetFs = target.getFileSystem();
try (ZipOutputStream out = new ZipOutputStream(targetFs.create(target,
FileSystem.WriteMode.NO_OVERWRITE))) {
addToZip(directory, sourceFs, directory.getParent(), out);
}
return target;
}
private static void addToZip(Path fileOrDirectory, FileSystem fs, Path rootDir,
ZipOutputStream out) throws IOException {
String relativePath = fileOrDirectory.getPath().replace(rootDir.getPath() +
'/', "");
if (fs.getFileStatus(fileOrDirectory).isDir()) {
out.putNextEntry(new ZipEntry(relativePath + '/'));
// The containedFile.getPath() returns an absolute path but the rootDir
// could be a relative path or an empty string (if user only specify the
// directory name as the relative path). In this case when calling this
// method recursively the replacement at the beginning of this method
will
// return a wrong result.
for (FileStatus containedFile : fs.listStatus(fileOrDirectory)) {
addToZip(containedFile.getPath(), fs, rootDir, out);
}
} else {
ZipEntry entry = new ZipEntry(relativePath);
out.putNextEntry(entry);
try (FSDataInputStream in = fs.open(fileOrDirectory)) {
IOUtils.copyBytes(in, out, false);
}
out.closeEntry();
}
}{code}
Currently PyFlink allows users to upload python library directories and
requirements cached directory, which will be compressed by
_FileUtils#compressDirectory_ eventually. If users specify them via relative
paths, this bug will be triggered and causes those features unavailable.
we can fix this bug by converting the directory path to absolute path in
_FileUtils#compressDirectory_ before calling _addToZip_ method_._
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
