[
https://issues.apache.org/jira/browse/FLINK-15248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17002742#comment-17002742
]
Hequn Cheng commented on FLINK-15248:
-------------------------------------
Resolved
in 1.10.0 via f002780091752b97a47e1866b653dbaf4eef802f
in 1.11.0 via 545534e43ed37f518fe59b6ddd8ed56ae82a234b
> FileUtils#compressDirectory behaves buggy when processing relative directory
> path
> ---------------------------------------------------------------------------------
>
> Key: FLINK-15248
> URL: https://issues.apache.org/jira/browse/FLINK-15248
> Project: Flink
> Issue Type: Bug
> Components: FileSystems
> Affects Versions: 1.10.0
> Reporter: Wei Zhong
> Assignee: Wei Zhong
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> _FileUtils#compressDirectory_ behaves buggy when processing relative
> directory path. If the path of target directory is a relative path, the
> relative path inside the target zip file can not be constructed correctly:
>
> {code:java}
> public static Path compressDirectory(Path directory, Path target) throws
> IOException {
> FileSystem sourceFs = directory.getFileSystem();
> FileSystem targetFs = target.getFileSystem();
> try (ZipOutputStream out = new ZipOutputStream(targetFs.create(target,
> FileSystem.WriteMode.NO_OVERWRITE))) {
> addToZip(directory, sourceFs, directory.getParent(), out);
> }
> return target;
> }
> private static void addToZip(Path fileOrDirectory, FileSystem fs, Path
> rootDir, ZipOutputStream out) throws IOException {
> String relativePath = fileOrDirectory.getPath().replace(rootDir.getPath()
> + '/', "");
> if (fs.getFileStatus(fileOrDirectory).isDir()) {
> out.putNextEntry(new ZipEntry(relativePath + '/'));
>
> // The containedFile.getPath() returns an absolute path but the rootDir
> // could be a relative path or an empty string (if user only specify
> the
> // directory name as the relative path). In this case when calling this
> // method recursively the string replacement at the beginning of it will
> // return a wrong result.
> for (FileStatus containedFile : fs.listStatus(fileOrDirectory)) {
> addToZip(containedFile.getPath(), fs, rootDir, out);
> }
> } else {
> ZipEntry entry = new ZipEntry(relativePath);
> out.putNextEntry(entry);
> try (FSDataInputStream in = fs.open(fileOrDirectory)) {
> IOUtils.copyBytes(in, out, false);
> }
> out.closeEntry();
> }
> }{code}
>
> Currently PyFlink allows users to upload python library directories and
> requirements cached directory, which will be compressed by
> _FileUtils#compressDirectory_ eventually. If users specify them via relative
> paths, this bug will be triggered and causes those features unavailable.
> we can fix this bug by converting the directory path to absolute path in
> _FileUtils#compressDirectory_ before calling _addToZip_ method.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
