[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586806#comment-15586806 ] Hudson commented on HDFS-10906: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10633 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10633/]) HDFS-10906. Add unit tests for Trash with HDFS encryption zones. (xyao: rev c62ae7107f025091652e79db3edfca5c4dc84e4a) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestTrashWithSecureEncryptionZones.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestTrashWithEncryptionZones.java > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Hanisha Koneru > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HDFS-10906.000.patch, HDFS-10906.001.patch, > HDFS-10906.002.patch, HDFS-10906.003.patch, HDFS-10906.004.patch, > HDFS-10906.005.patch > > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586750#comment-15586750 ] Hanisha Koneru commented on HDFS-10906: --- Thank you [~xyao] for reviewing and committing the patch. > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Hanisha Koneru > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HDFS-10906.000.patch, HDFS-10906.001.patch, > HDFS-10906.002.patch, HDFS-10906.003.patch, HDFS-10906.004.patch, > HDFS-10906.005.patch > > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586618#comment-15586618 ] Hadoop QA commented on HDFS-10906: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 10s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 25s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 102 unchanged - 1 fixed = 103 total (was 103) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 9s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 72m 19s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 19s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 91m 24s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Issue | HDFS-10906 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12834014/HDFS-10906.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux dfaa5de2b0c1 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / b733a6f | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/17206/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/17206/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17206/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/17206/console | | Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 >
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583442#comment-15583442 ] Xiaoyu Yao commented on HDFS-10906: --- [~hanishakoneru], thanks for the update. Patch v4 looks good to me. +1 pending Jenkins. > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Hanisha Koneru > Attachments: HDFS-10906.000.patch, HDFS-10906.001.patch, > HDFS-10906.002.patch, HDFS-10906.003.patch, HDFS-10906.004.patch > > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583295#comment-15583295 ] Hadoop QA commented on HDFS-10906: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 25s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 11 new + 102 unchanged - 1 fixed = 113 total (was 103) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 58m 21s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 22s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 78m 45s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality | | | hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Issue | HDFS-10906 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12833780/HDFS-10906.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux ccc73ef4f436 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / ed9fcbe | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/17185/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/17185/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17185/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/17185/console | | Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Add unit tests for Trash with HDFS encryption zones > --
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15575948#comment-15575948 ] Xiaoyu Yao commented on HDFS-10906: --- Thanks for the update [~hanishakoneru]. Here are some additional minor issues. 1. Fix the checkstyle issue 2. DFSTestUtil#verifyDelete(), since we are dealing with HDFS path, the following can be simplified from {code} Path trashPath = Path.mergePaths(shell.getCurrentTrashDir(path), path); {code} to {code} Path trashPath = new Path(shell.getCurrentTrashDir(path), path); {code} TestTrashWithSecureEncryptionZones.java 3. NIT: Can we put all static imports in a single block before the non-static imports? 4. The following two counters should be AtomicInteger {code} private static int zoneCounter = 1; private static int fileCounter = 1; {code} > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Hanisha Koneru > Attachments: HDFS-10906.000.patch, HDFS-10906.001.patch, > HDFS-10906.002.patch > > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573648#comment-15573648 ] Hadoop QA commented on HDFS-10906: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 26s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 7 new + 102 unchanged - 1 fixed = 109 total (was 103) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 11s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 66m 42s{color} | {color:green} hadoop-hdfs in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 88m 9s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Issue | HDFS-10906 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12833229/HDFS-10906.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 1581c8308a65 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 0a85d07 | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/17144/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17144/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/17144/console | | Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570278#comment-15570278 ] Xiaoyu Yao commented on HDFS-10906: --- Thanks [~hanishakoneru] for working on this to improve the unit test coverage. The patch looks good to me overall. I just have a few comments. *TestTrashWithEncryptionZones.java* 1. Unused imports import org.apache.hadoop.fs.FileStatus; import org.junit.rules.Timeout; import org.junit.runners.MethodSorters; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.junit.AfterClass; import org.junit.BeforeClass; import org.junit.FixMethodOrder; 2. We can remove the EncryptionFaultInjector if it is not used in the test. {code} @After 119 public void teardown() { 120 if (cluster != null) { 121 cluster.shutdown(); 122 cluster = null; 123 } 124 EncryptionFaultInjector.instance = new EncryptionFaultInjector(); 125 ... {code} 3. verifyDelete() We can use the Path constructor that takes two parameters to avoid concat the path with "/" {code} 203 final Path trashFile = new Path(shell.getCurrentTrashDir(path) + "/" + 204 path); {code} 4. testDeleteEZWithMultipleUsers() Some of the verification of positive cases can be simplified by verifyDelete(). Combining with the verifyDelete() in TestTrashWithSecureEncryptionZones, I think we can add it to test utilities such as DFSTestUtil for better reuse. NIT: extra empty lines around 156 *TestTrashWithSecureEncryptionZones.java* 5. Unused imports import org.slf4j.Logger; import org.slf4j.LoggerFactory; 6. NIT: many instances of local variable len can be a static member of the class {code}final int len = 8192; {code} 7. testTrashExpunge Can we add verification of case where expunge will delete checkpoint inside the encryption zone when the whole encryption zone has been deleted and moved to trash directory under /user/$USER/.Trash? > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Hanisha Koneru > Attachments: HDFS-10906.000.patch, HDFS-10906.001.patch > > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566337#comment-15566337 ] Hadoop QA commented on HDFS-10906: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 23s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 27 new + 0 unchanged - 0 fixed = 27 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 10s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 60m 40s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 22s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 80m 23s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestFileAppend | | | hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Issue | HDFS-10906 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12832717/HDFS-10906.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux ceab961b0513 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 809cfd2 | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/17103/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/17103/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17103/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/17103/console | | Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Add unit tests for Trash with HDFS encryption zones > --- > >
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564335#comment-15564335 ] Hadoop QA commented on HDFS-10906: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 3s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 36s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 598 new + 0 unchanged - 0 fixed = 598 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 11s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch 468 line(s) with tabs. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 78m 5s{color} | {color:green} hadoop-hdfs in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 25s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}101m 56s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Issue | HDFS-10906 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12832592/HDFS-10906.000.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux ca54550115a4 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 96b1266 | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/17095/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/17095/artifact/patchprocess/whitespace-tabs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17095/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/17095/console | | Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task >
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564152#comment-15564152 ] Hanisha Koneru commented on HDFS-10906: --- Thank you [~xyao]. I have added two tests to cover the list of test cases. > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao >Assignee: Hanisha Koneru > Attachments: HDFS-10906.000.patch > > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15530300#comment-15530300 ] Xiaoyu Yao commented on HDFS-10906: --- Agree. Regrading adding Kerberos enabled trash related unit test with KMS/EncryptionZone, we can refactor TestSecureEncryptionZoneWithKMS to share the basic setup for Kerberos (Kerby based MiniKDC) + KMS (MiniKMS) + MiniDFSCluster. > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15530249#comment-15530249 ] Mingliang Liu commented on HDFS-10906: -- Thanks for the to do list, [~xyao]. +1 for the proposal. It merits a lot to cover them in unit tests. > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10906) Add unit tests for Trash with HDFS encryption zones
[ https://issues.apache.org/jira/browse/HDFS-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525143#comment-15525143 ] Xiaoyu Yao commented on HDFS-10906: --- Below is a proposed test plan for list of test cases to add: # Trash Checkpoint within the encryption zone # Trash Checkpoint outside the encryption zone when the whole encryption is deleted and moved. # Trash expunge within the encryption zone # Trash expunge outside the encryption zone # Delete file/directory within the encryption zone (Kerberos off) # Delete file/directory within the encryption zone (Kerberos on) # Delete directory with/wo -skipTrash option # Delete empty directory without -skipTrash option # Delete entire encryption zone with different test users # Delete file/directory from Trash within the encryption zone > Add unit tests for Trash with HDFS encryption zones > --- > > Key: HDFS-10906 > URL: https://issues.apache.org/jira/browse/HDFS-10906 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: encryption >Affects Versions: 2.8.0 >Reporter: Xiaoyu Yao > > The goal is to improve unit test coverage for HDFS trash with encryption zone > especially under Kerberos environment. The current unit test > TestEncryptionZones#testEncryptionZonewithTrash() has limited coverage on > non-Kerberos case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org