[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390109#comment-16390109 ] Hanisha Koneru commented on HDFS-13109: --- Thanks [~xyao] for committing the patch and all for the reviews. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 2.8.4, 3.0.2, 3.2.0 > > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch, HDFS-13109.005.patch, > HDFS-13109.006.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388964#comment-16388964 ] Hudson commented on HDFS-13109: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13784 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13784/]) HDFS-13109. Support fully qualified hdfs path in EZ commands. (xyao: rev edf9445708ffb7a9e59cb933e049b540f99add1e) * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 2.8.4, 3.0.2, 3.2.0 > > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch, HDFS-13109.005.patch, > HDFS-13109.006.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388688#comment-16388688 ] Xiaoyu Yao commented on HDFS-13109: --- Agree with [~shahrs87] that we can remove the unused import at commit to save the Jenkins cycle. In the meanwhile, I will post a new patch that matches with the commit using git format-patch in case cherry-pick is not available. Thanks! > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch, HDFS-13109.005.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388664#comment-16388664 ] Daniel Templeton commented on HDFS-13109: - I'd really rather we fix that in the patch. It's very useful to have the patches on the JIRAs line up with the commits. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch, HDFS-13109.005.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388659#comment-16388659 ] Rushabh S Shah commented on HDFS-13109: --- I am sorry to point out that there is 1 checkstyle warning in latest patch. Lets not waste build resource to fix 1 checkstyle warning. [~xyao] can edit that file while committing. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch, HDFS-13109.005.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388644#comment-16388644 ] genericqa commented on HDFS-13109: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 31s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 1s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 46s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 55s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 57s{color} | {color:orange} hadoop-hdfs-project: The patch generated 1 new + 65 unchanged - 0 fixed = 66 total (was 65) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 33s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 23s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 33s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}119m 56s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 25s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}187m 42s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterQuota | | | hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting | | | hadoop.hdfs.server.balancer.TestBalancerRPCDelay | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-13109 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12913247/HDFS-13109.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux bfc78705f3ec 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388547#comment-16388547 ] Xiaoyu Yao commented on HDFS-13109: --- +1 pending Jenkins. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch, HDFS-13109.005.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388377#comment-16388377 ] Rushabh S Shah commented on HDFS-13109: --- Thanks [~hanishakoneru] +1 non-binding pending jenkins for v5 patch. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch, HDFS-13109.005.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388372#comment-16388372 ] Hanisha Koneru commented on HDFS-13109: --- Thank you [~xyao] and [~shahrs87] for the reviews and discussion. I have updated patch v05 addressing Rushabh's comments on TestEncryptionZones. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch, HDFS-13109.005.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386710#comment-16386710 ] Rushabh S Shah commented on HDFS-13109: --- Please ignore my last comment. bq. However, the path parameter of HdfsAdmin#provisionEncryptionZoneTrash has never been fully resolved by itself or the caller from CryptoAdmin. I thought {{provisionTrash}} is also a {{superuser-only}} command but it doesn't seems like that. So I agree with the fix in v4 of the patch. Other than test file changes, I am +1 for the patch. Sorry for the noise. Thanks [~hanishakoneru] for the contribution. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386687#comment-16386687 ] Rushabh S Shah commented on HDFS-13109: --- bq. we will still miss the relative path and path resolution handling The {{createEncryptionZone}} command is run as superuser so you have to specify the whole path and not relative path. It doesn't make sense to create EZ for superuser owned directory. Regarding path resolution, we don't support symlinks on client side. If we are considering clusters where we enable symlinks on client side, then resolving the user supplied path makes sense. [~xyao]: Let me know what you think. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386577#comment-16386577 ] Xiaoyu Yao commented on HDFS-13109: --- Thanks [~shahrs87] for the review. Agree with the suggestions on additional verification testEZwithFullyQualifiedPath. For suggested change in HdfsAdmin, I still think we should move the change to DistributedFileSystem. Below are my thoughts: HdfsAdmin#provisionEncryptionZoneTrash has a public API that does not resolve the input path completely. Removing the scheme/authority before compare does not fully address the problem. In the private version of HdfsAdmin#provisionEncryptionZoneTrash, the encryption zone path returned from getEZForPath() has been fully resolved during DistributedFileSystem#createEncryptionZone. However, the path parameter of HdfsAdmin#provisionEncryptionZoneTrash has never been fully resolved by itself or the caller from CryptoAdmin. Correct me if I'm wrong: If we work around the path compare logic inside HdfsAdmin using Path#getPathWithoutSchemeAndAuthority or DFS#getPathNames, we will still miss the relative path and path resolution handling, etc that added with [~hanishakoneru] 's patch in DistributedFileSystem. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386353#comment-16386353 ] Rushabh S Shah commented on HDFS-13109: --- +HdfsAdmin.java+ If I understand the jira correctly, this check is the problem. {code:title=HdfsAdmin.java|borderStyle=solid} private void provisionEZTrash(Path path) throws IOException { ... String ezPath = ez.getPath(); if (!path.toString().equals(ezPath)) { throw new IllegalArgumentException(path + " is not the root of an " + "encryption zone. Do you mean " + ez.getPath() + "?"); } ... } {code} This if condition is comparing the path including scheme and authority with the path that doesn't contain scheme and authority. If you use the {{Path#getPathWithoutSchemeAndAuthority}}, then it will just compare the path component of the uri. Something like this. {noformat} String ezPath = ez.getPath(); if (!Path.getPathWithoutSchemeAndAuthority(path).toString().equals(ezPath)) { throw new IllegalArgumentException(path + " is not the root of an " + "encryption zone. Do you mean " + ez.getPath() + "?"); } {noformat} This will get rid of all the change in {{DistributedFileSystem}} and {{HdfsAdmin}} related code in the patch. +TestEncryptionZones+ 1. bq.assertZonePresent(null, zone1.toString()); For both the {{assertZonePresent}} checks, we should pass the key string (TEST_KEY) and verify that EZ was created with that specific key. 2. In first case i.e {{Create EZ with Trash}}, I would like to see a check that {{zone1's}} path contains a trash directory. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386176#comment-16386176 ] Rushabh S Shah commented on HDFS-13109: --- Taking a look now. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384286#comment-16384286 ] Xiaoyu Yao commented on HDFS-13109: --- Thanks [~hanishakoneru] for the update. +1 for the v4 patch. I will hold off the commit for next Monday in case [~shahrs87] has additional comments. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379591#comment-16379591 ] genericqa commented on HDFS-13109: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 33s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 42s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 24s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 24s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}147m 45s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}203m 45s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure | | | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.web.TestWebHdfsTimeouts | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting | | | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | HDFS-13109 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12912329/HDFS-13109.004.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux fc5da1da2f86 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool |
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379287#comment-16379287 ] Hanisha Koneru commented on HDFS-13109: --- Thanks for the review, [~xyao]. bq. You have already resolved the path in the calling function public void provisionEZTrash. You can just pass the resolved path to the private method provisionEZTrash instead of getPathName. [~shahrs87], we would have to call {{getPathName()}} as the {{FileSystemLinkResolver.resolve}} function in the calling {{public void provisionEZTrash}} doesn't verify that the path belongs to the correct filesystem. Please let me know if I am missing something here. I have reverted {{p.toUri().getPath()}} to {{getPathName(p)}} in patch v04. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch, HDFS-13109.004.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377855#comment-16377855 ] Xiaoyu Yao commented on HDFS-13109: --- [~hanishakoneru], the latest patch looks good to me. Just have one more NIT DistributedFileSystem.java Line 2505: we should use getPathName(p) to replace the p.toUri().getPath() inside doCall(). You can find similar usage in the doCall() and next() override in other places of this class. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370741#comment-16370741 ] genericqa commented on HDFS-13109: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 44s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 16s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 43s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 22s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}118m 46s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}179m 17s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | HDFS-13109 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12911273/HDFS-13109.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 720859b6050a 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 9028cca | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | unit |
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370572#comment-16370572 ] Hanisha Koneru commented on HDFS-13109: --- Thanks for reviewing the patch, [~shahrs87]. {quote}The variable {{dfs}} in {{HdfsAdmin}} referred to {{DistributedFileSystem}} whereas in {{DistributedFileSystem}}, {{dfs}} refers to {{DFSClient}}.{{DistributedFileSystem#getEZForPath}} resolves the path and calls {{DFSClient#getEZForPath}}. Whereas after the patch, it won't resolve the path. {quote} I did not resolve the path as the input path to {{private void provisionEZTrash}} is already resolved by the calling method. So we can skip {{DistributedFileSystem#getEZForPath}} and directly call {{DFSClient#getEZForPath}}. Please correct me if I understood wrongly. {quote}You have already resolved the path in the calling function public void provisionEZTrash. You can just pass the resolved path to the private method provisionEZTrash instead of getPathName. {quote} Yes, thanks for catching this. We can skip the {{getPathName}} and directly pass the resolved path component. Addressed other review comments and checkstyle issues in patch v03. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch, > HDFS-13109.003.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369370#comment-16369370 ] Rushabh S Shah commented on HDFS-13109: --- Thanks [~hanishakoneru] for the patch. +DistributedFileSystem.java+ 1. In next method of {{provisionEZTrash}}, need to check if fs instanceof DistributedFileSystem. If not, then we need to throw UnsupportedOperationException. Check {{recoverLease}} method for similar implementation. 2. {code} private void provisionEZTrash(String path, FsPermission trashPermission) throws IOException { // make sure the path is an EZ EncryptionZone ez = dfs.getEZForPath(path); {code} I understand that you just moved this method from {{HdfsAdmin}} to {{DistributedFileSystem}}. The variable {{dfs}} in {{HdfsAdmin}} referred to {{DistributedFileSystem}} whereas in {{DistributedFileSystem}}, {{dfs}} refers to {{DFSClient}}. {{DistributedFileSystem#getEZForPath}} resolves the path and calls {{DFSClient#getEZForPath}}. Whereas after the patch, it won't resolve the path. You have already resolved the path in the calling function {{public void provisionEZTrash}}. You can just pass the resolved path to the private method {{provisionEZTrash}} instead of {{getPathName}}. Hope it makes sense. Mnior 3. The patch contains diff of imports for the parts of code which you have not changed. e.g.{{SnapshotDiffReportListing.DiffReportListingEntry}} Please reset it. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363244#comment-16363244 ] genericqa commented on HDFS-13109: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 45s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 14s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 37s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 44s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 23s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 35s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 41s{color} | {color:orange} hadoop-hdfs-project: The patch generated 3 new + 65 unchanged - 0 fixed = 68 total (was 65) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 10s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 35s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 51s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}131m 3s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}196m 42s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.server.namenode.TestDecommissioningStatus | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | HDFS-13109 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12910432/HDFS-13109.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b25a4463 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 332269d | | maven | version: Apache Maven
[jira] [Commented] (HDFS-13109) Support fully qualified hdfs path in EZ commands
[ https://issues.apache.org/jira/browse/HDFS-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362993#comment-16362993 ] Hanisha Koneru commented on HDFS-13109: --- Thanks for the review, [~xyao]. Uploaded patch v02 with a unit test and fixed checkstyle. > Support fully qualified hdfs path in EZ commands > > > Key: HDFS-13109 > URL: https://issues.apache.org/jira/browse/HDFS-13109 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: HDFS-13109.001.patch, HDFS-13109.002.patch > > > When creating an Encryption Zone, if the fully qualified path is specified in > the path argument, it throws the following error. > {code:java} > ~$ hdfs crypto -createZone -keyName mykey1 -path hdfs://ns1/zone1 > IllegalArgumentException: hdfs://ns1/zone1 is not the root of an encryption > zone. Do you mean /zone1? > ~$ hdfs crypto -createZone -keyName mykey1 -path "hdfs://namenode:9000/zone2" > IllegalArgumentException: hdfs://namenode:9000/zone2 is not the root of an > encryption zone. Do you mean /zone2? > {code} > The EZ creation succeeds as the path is resolved in > DFS#createEncryptionZone(). But while creating the Trash directory, the path > is not resolved and it throws the above error. > A fully qualified path should be supported by {{crypto}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org