[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege
[ https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978574#comment-14978574 ] Yong Zhang commented on HDFS-8333: -- Hi [~szetszwo], currently we have user cases of multi-tenancy on hdfs, if only superuser/supergroup has right to set ec zone, it will be so limitation. In my opinion, it is better for each tenant to have its own administrator, and manage everything belong to him. > Create EC zone should not need superuser privilege > -- > > Key: HDFS-8333 > URL: https://issues.apache.org/jira/browse/HDFS-8333 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Yong Zhang >Assignee: Yong Zhang > Attachments: HDFS-8333-HDFS-7285.000.patch > > > create EC zone should not need superuser privilege, for example, in multiple > tenant scenario, common users only manage their own directory and > subdirectory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege
[ https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979440#comment-14979440 ] Zhe Zhang commented on HDFS-8333: - bq. createEncryptionZone requires superuser privilege. So we should do the same for ec zone. [~andrew.wang] It'd be helpful if you can share some thoughts here. Thanks. > Create EC zone should not need superuser privilege > -- > > Key: HDFS-8333 > URL: https://issues.apache.org/jira/browse/HDFS-8333 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Yong Zhang >Assignee: Yong Zhang > Attachments: HDFS-8333-HDFS-7285.000.patch > > > create EC zone should not need superuser privilege, for example, in multiple > tenant scenario, common users only manage their own directory and > subdirectory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege
[ https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979473#comment-14979473 ] Andrew Wang commented on HDFS-8333: --- Regarding createEncryptionZone, we kept it superuser-only since a new zone typically requires creating a new key, and creating a new key normally requires admin-level permissions. Plus admins will want to lock down what keys are in use on the cluster, so there's another security angle there too. Nicholas is right that we can always relax the permissions later, so that's the most conservative choice. However, I see EC as like setting the replication factor. I think we should let users use their disk quota however they want, be it replication or EC. I'm also still hoping for a unification of EC with the StoragePolicy APIs too, which are not admin-only. We had a long discussion about this on HDFS-8833, but I don't think any progress has been made towards it yet. > Create EC zone should not need superuser privilege > -- > > Key: HDFS-8333 > URL: https://issues.apache.org/jira/browse/HDFS-8333 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Yong Zhang >Assignee: Yong Zhang > Attachments: HDFS-8333-HDFS-7285.000.patch > > > create EC zone should not need superuser privilege, for example, in multiple > tenant scenario, common users only manage their own directory and > subdirectory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege
[ https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550012#comment-14550012 ] Walter Su commented on HDFS-8333: - Patch looks good. I'm +1 for this idea. Hi, [~drankye], and [~zhz]. How do you think about it? Create EC zone should not need superuser privilege -- Key: HDFS-8333 URL: https://issues.apache.org/jira/browse/HDFS-8333 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Yong Zhang Assignee: Yong Zhang Attachments: HDFS-8333-HDFS-7285.000.patch create EC zone should not need superuser privilege, for example, in multiple tenant scenario, common users only manage their own directory and subdirectory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege
[ https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550147#comment-14550147 ] Kai Zheng commented on HDFS-8333: - It looks good to me. As I expressed in HDFS-8112, the superuser privilege might be too restricted to operations for both EC zone and schemas. I thought [~zhangyongxyz] raised a reasonable case here. [~szetszwo] should we consider it for now? Thanks. Create EC zone should not need superuser privilege -- Key: HDFS-8333 URL: https://issues.apache.org/jira/browse/HDFS-8333 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Yong Zhang Assignee: Yong Zhang Attachments: HDFS-8333-HDFS-7285.000.patch create EC zone should not need superuser privilege, for example, in multiple tenant scenario, common users only manage their own directory and subdirectory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege
[ https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551133#comment-14551133 ] Tsz Wo Nicholas Sze commented on HDFS-8333: --- [~drankye], thanks for checking with me. createEncryptionZone requires superuser privilege. So we should do the same for ec zone. Also, since this is a new feature. It is good to start with a more strict permission requirement. We may relax it later on if necessary. Create EC zone should not need superuser privilege -- Key: HDFS-8333 URL: https://issues.apache.org/jira/browse/HDFS-8333 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Yong Zhang Assignee: Yong Zhang Attachments: HDFS-8333-HDFS-7285.000.patch create EC zone should not need superuser privilege, for example, in multiple tenant scenario, common users only manage their own directory and subdirectory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege
[ https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551296#comment-14551296 ] Zhe Zhang commented on HDFS-8333: - Thanks for the discussion Walter, Kai, and Nicholas. Shall we move this to HDFS-8031? Create EC zone should not need superuser privilege -- Key: HDFS-8333 URL: https://issues.apache.org/jira/browse/HDFS-8333 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Yong Zhang Assignee: Yong Zhang Attachments: HDFS-8333-HDFS-7285.000.patch create EC zone should not need superuser privilege, for example, in multiple tenant scenario, common users only manage their own directory and subdirectory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege
[
https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532933#comment-14532933
]
Hadoop QA commented on HDFS-8333:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 31s | Pre-patch HDFS-7285 compilation
is healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 26s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 4m 48s | The applied patch generated
41 release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 39s | The applied patch generated
529 new checkstyle issues (total was 0, now 525). |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 4 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 15s | The patch appears to introduce 8
new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 181m 41s | Tests failed in hadoop-hdfs. |
| | | 227m 26s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| | Inconsistent synchronization of
org.apache.hadoop.hdfs.DFSOutputStream.streamer; locked 89% of time
Unsynchronized access at DFSOutputStream.java:89% of time Unsynchronized
access at DFSOutputStream.java:[line 146] |
| | Possible null pointer dereference of arr$ in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStripedUnderConstruction.initializeBlockRecovery(long)
Dereferenced at BlockInfoStripedUnderConstruction.java:arr$ in
org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStripedUnderConstruction.initializeBlockRecovery(long)
Dereferenced at BlockInfoStripedUnderConstruction.java:[line 206] |
| | Unread field:field be static? At ErasureCodingWorker.java:[line 251] |
| | Should
org.apache.hadoop.hdfs.server.datanode.erasurecode.ErasureCodingWorker$StripedReader
be a _static_ inner class? At ErasureCodingWorker.java:inner class? At
ErasureCodingWorker.java:[lines 910-912] |
| | Found reliance on default encoding in
org.apache.hadoop.hdfs.server.namenode.ErasureCodingZoneManager.createErasureCodingZone(String,
ECSchema):in
org.apache.hadoop.hdfs.server.namenode.ErasureCodingZoneManager.createErasureCodingZone(String,
ECSchema): String.getBytes() At ErasureCodingZoneManager.java:[line 117] |
| | Found reliance on default encoding in
org.apache.hadoop.hdfs.server.namenode.ErasureCodingZoneManager.getECZoneInfo(INodesInPath):in
org.apache.hadoop.hdfs.server.namenode.ErasureCodingZoneManager.getECZoneInfo(INodesInPath):
new String(byte[]) At ErasureCodingZoneManager.java:[line 81] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.constructInternalBlock(LocatedStripedBlock,
int, int, int, int) At StripedBlockUtil.java:to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.constructInternalBlock(LocatedStripedBlock,
int, int, int, int) At StripedBlockUtil.java:[line 84] |
| | Result of integer multiplication cast to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.planReadPortions(int, int, long,
int, int) At StripedBlockUtil.java:to long in
org.apache.hadoop.hdfs.util.StripedBlockUtil.planReadPortions(int, int, long,
int, int) At StripedBlockUtil.java:[line 204] |
| Failed unit tests | hadoop.hdfs.TestRecoverStripedFile |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731165/HDFS-8333-HDFS-7285.000.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / c61c9c8 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/10844/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10844/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/10844/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
