[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110288#comment-17110288 ] Ayush Saxena commented on HDFS-14762: - Seems this a very old and known one, check HDFS-13 Checking bunch of the related JIRA's all near this. May be HADOOP-3257 is a good place to discuss and get feedback.. > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch, HDFS-14762.004.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110069#comment-17110069 ] Surendra Singh Lilhore commented on HDFS-14762: --- Hi [~ayushtkn], {quote}File Name used IPv6? What is the relation of name & IPv6? {quote} We are trying HDFS with IPv6. Datanode create block pool directory and the name of block pool contain IP of namenode. If the NN started with Ipv6 then this name contain ":" and same problem occur. > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch, HDFS-14762.004.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109947#comment-17109947 ] hemanthboyina commented on HDFS-14762: -- i think this issue also relates to IPV6 > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch, HDFS-14762.004.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17109943#comment-17109943 ] Ayush Saxena commented on HDFS-14762: - bq. some file name used IPv6. File Name used IPv6? What is the relation of name & IPv6? If you want to follow this issue. Please follow at HADOOP-14829 This had same problem, and may be getting a agreement and conclusion at HADOOP-3257 > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch, HDFS-14762.004.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952404#comment-16952404 ] Shixiong Zhu commented on HDFS-14762: - [~ayushtkn] Some file systems allow ":" in the name. For example, in the local file system, the following Scala codes will fail because it cannot create a path for the checksum file ".a:b.crc" on Linux. But "/tmp/a:b" will be created. {code:java} import org.apache.hadoop.fs._ import org.apache.hadoop.conf._ val conf = new Configuration val path = new Path("file:///tmp/a:b") val fs = path.getFileSystem(conf) fs.create(path).close() {code} The same issue happens in S3AFileSystem. IMO, since both FileSystem and Path are generic classes, if ":" should not appear in a valid file name, it should be checked in abstract FileSystem class rather than HDFS. If FileSystem doesn't check it and it's not enforced in all FileSystems, it should be a valid char in a file name, and should be fixed in Path. > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch, HDFS-14762.004.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1693#comment-1693 ] Ayush Saxena commented on HDFS-14762: - Sorry for the goof up, but just realized, name having : isn't a valid name for HDFS. Check here : {code:java} /** * Whether the pathname is valid. Currently prohibits relative paths, * names which contain a ":" or "//", or other non-canonical paths. */ public static boolean isValidName(String src) { // Path must be absolute. if (!src.startsWith(Path.SEPARATOR)) { return false; } // Check for ".." "." ":" "/" String[] components = StringUtils.split(src, '/'); for (int i = 0; i < components.length; i++) { String element = components[i]; if (element.equals(".") || (element.contains(":")) || (element.contains("/"))) { return false; } // ".." is allowed in path starting with /.reserved/.inodes if (element.equals("..")) { if (components.length > 4 && components[1].equals(".reserved") && components[2].equals(".inodes")) { continue; } return false; } // The string may start or end with a /, but not have // "//" in the middle. if (element.isEmpty() && i != components.length - 1 && i != 0) { return false; } } return true; } {code} > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch, HDFS-14762.004.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929777#comment-16929777 ] hemanthboyina commented on HDFS-14762: -- thanks for the new test case condition [~ayushtkn] Updated new patch > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch, HDFS-14762.004.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929764#comment-16929764 ] Hadoop QA commented on HDFS-14762: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 13s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 34s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 16m 23s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 22s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 19m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 12s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 25s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 18s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 55s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}119m 10s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=18.09.7 Server=18.09.7 Image:yetus/hadoop:39e82acc485 | | JIRA Issue | HDFS-14762 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12980323/HDFS-14762.004.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 700d6629710c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / e04b8a4 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_222 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/27874/testReport/ | | Max. process+thread count | 1341 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/27874/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > >
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929678#comment-16929678 ] Ayush Saxena commented on HDFS-14762: - If the path just has colon at end? say a: ? > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929645#comment-16929645 ] hemanthboyina commented on HDFS-14762: -- no [~ayushtkn] that doesn't work we need to update the scheme to > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929575#comment-16929575 ] Ayush Saxena commented on HDFS-14762: - Will just having this work : {code:java} // add "./" in front of Linux relative paths so that a path containing // a colon e.q. "a:b" will not be interpreted as scheme "a". if (!WINDOWS && path.length() > 0 && path.charAt(0) != '/') { path = "./" + path; } {code} > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921182#comment-16921182 ] hemanthboyina commented on HDFS-14762: -- updated the patch , no test failures now > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-14762.001.patch, HDFS-14762.002.patch, > HDFS-14762.003.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921179#comment-16921179 ] Hadoop QA commented on HDFS-14762: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 30s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 16s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 6s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 2s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 9s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 44s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 93m 56s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 | | JIRA Issue | HDFS-14762 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12979147/HDFS-14762.003.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 5705af42c75e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 915cbc9 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_222 | | findbugs | v3.1.0-RC1 | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/27765/artifact/out/whitespace-eol.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/27765/testReport/ | | Max. process+thread count | 1656 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/27765/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920552#comment-16920552 ] Hadoop QA commented on HDFS-14762: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 36s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 54s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 54s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 3s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 45s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 94m 52s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.fs.TestLocalFileSystem | | | hadoop.fs.TestPath | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 | | JIRA Issue | HDFS-14762 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12979082/HDFS-14762.002.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux be7e63a4ab8e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 751b5a1 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_222 | | findbugs | v3.1.0-RC1 | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/27758/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/27758/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/27758/testReport/ | | Max. process+thread count | 1516 (vs. ulimit of 5500) | | modules | C:
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920452#comment-16920452 ] Ayush Saxena commented on HDFS-14762: - Test failures seems related. Please check!!! > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-14762.001.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920435#comment-16920435 ] hemanthboyina commented on HDFS-14762: -- Updated the patch , pls review [~ayushtkn] > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-14762.001.patch > > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920336#comment-16920336 ] Hadoop QA commented on HDFS-14762: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 35s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 54s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 11s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m 11s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 43s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 35s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 46s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}104m 6s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.fs.TestAfsCheckPath | | | hadoop.fs.TestFileUtil | | | hadoop.fs.TestDelegateToFsCheckPath | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 | | JIRA Issue | HDFS-14762 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12979062/HDFS-14762.001.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux c6bcf321d609 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / c7ef4fb | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_222 | | findbugs | v3.1.0-RC1 | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/27752/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/27752/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/27752/testReport/ | | Max. process+thread count | 1497 (vs. ulimit of
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920303#comment-16920303 ] hemanthboyina commented on HDFS-14762: -- {code:java} // add "./" in front of Linux relative paths so that a path containing // a colon e.q. "a:b" will not be interpreted as scheme "a". if (!WINDOWS && path.charAt(0) != '/') { path = "./" + path; } {code} for linux , if the path given was a:b , it should be as ./a:b adding a patch shortly > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919769#comment-16919769 ] Shixiong Zhu commented on HDFS-14762: - I don't have a proposal right now. Fixing the Path constructor is definitely better if that's possible :) > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919244#comment-16919244 ] hemanthboyina commented on HDFS-14762: -- hi [~ayushtkn] [~zsxwing] written a unit test with "a:b" , was passed in windows and failed in linux , ideally it should be success at both I suggest we should fix that at root level i.e at Path constructor . > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919207#comment-16919207 ] Ayush Saxena commented on HDFS-14762: - [~zsxwing] do you propose any other way of fixing this, Ideally I was inclined towards ensuring the Path constructor supports the : symbol in name, Anyway if you have a better solution, we can check that too. > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918897#comment-16918897 ] hemanthboyina commented on HDFS-14762: -- fixing the issue at constructor level will make sure Path a:b will be accepted a:b is not ambigious then ? > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918865#comment-16918865 ] Ayush Saxena commented on HDFS-14762: - IMO we should try fixing {{"Path(Path/String parent, String child)}}, It should resolve properly without exception. > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918827#comment-16918827 ] hemanthboyina commented on HDFS-14762: -- [~zsxwing] root cause fix should be at Path constructor , i think fixing at these two places only wont resolve the problem [~ayushtkn] any thoughts ? > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916118#comment-16916118 ] Shixiong Zhu commented on HDFS-14762: - [~hemanthboyina] I'm not concerned about the behavior of this Path constructor. What I hope can be fixed are the following two places: https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 Right now they throw an exception when hitting a file name that contains ":". > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915242#comment-16915242 ] hemanthboyina commented on HDFS-14762: -- [~zsxwing] , then we need to handle the exception with a proper message ? whats your expecting point on the issue ? > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915241#comment-16915241 ] hemanthboyina commented on HDFS-14762: -- [~zsxwing] , then we need to handle the exception with a proper message ? whats your expecting point on this ? > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914685#comment-16914685 ] Shixiong Zhu commented on HDFS-14762: - IMO, "a:b" is ambiguous. It can be a relative path or a path with a scheme. It doesn't carry enough information. > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914575#comment-16914575 ] hemanthboyina commented on HDFS-14762: -- {code:java} // parse uri scheme, if any int colon = pathString.indexOf(':'); int slash = pathString.indexOf('/'); if ((colon != -1) && ((slash == -1) || (colon < slash))) { // has a scheme scheme = pathString.substring(0, colon); start = colon+1; } // parse uri authority, if any // uri path is the rest of the string -- query & fragment not supported String path = pathString.substring(start, pathString.length()); initialize(scheme, authority, path, null); {code} a colon with slash as next character *:/* will be success , but colon without a slash will fail *:* from the initilaize (URI will be constructed here) a:/b if the scheme was not null and the path starts with slash - Success a:b but if the scheme was not null and path starts without slash - Fails [~ayushtkn] any suggestions for implementing a:b success ? > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Assignee: hemanthboyina >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913583#comment-16913583 ] Ayush Saxena commented on HDFS-14762: - Thanx [~zsxwing] and [~hemanthboyina]. Ideally it shoudln't happen, I think this should be fair to try fixing this. > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14762) "Path(Path/String parent, String child)" will fail when "child" contains ":"
[ https://issues.apache.org/jira/browse/HDFS-14762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913340#comment-16913340 ] hemanthboyina commented on HDFS-14762: -- written a junit , the exception was thrown if the child string containing":" [~ayushtkn] need to fix with a check , else let the exception be thrown ? > "Path(Path/String parent, String child)" will fail when "child" contains ":" > > > Key: HDFS-14762 > URL: https://issues.apache.org/jira/browse/HDFS-14762 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shixiong Zhu >Priority: Major > > When the "child" parameter contains ":", "Path(Path/String parent, String > child)" will throw the following exception: > {code} > java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative > path in absolute URI: ... > {code} > Not sure if this is a legit bug. But the following places will hit this error > when seeing a Path with a file name containing ":": > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L101 > https://github.com/apache/hadoop/blob/f9029c4070e8eb046b403f5cb6d0a132c5d58448/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L270 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org