[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[ https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455058#comment-16455058 ] Hudson commented on HDFS-11768: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14070 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14070/]) HDFS-11768. Ozone: KSM: Create Key Space manager service. Contributed by (omalley: rev edff6c6a24948577552ab4109e0b0e0ac962e6c6) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/OzoneClientUtils.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerData.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/KeyData.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/storage/OzoneContainerTranslation.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/TestOzoneClientUtils.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/protocolPB/KeyspaceManagerProtocolServerSideTranslatorPB.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/cli/SQLCLI.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/KeySpaceManagerProtocol.proto * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/StorageContainerLocationProtocol.proto * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/scm/storage/ChunkOutputStream.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/DatanodeContainerProtocol.proto * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/Ozone.proto * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KeySpaceManager.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/package-info.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/helpers/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/proto/StorageContainerDatanodeProtocol.proto * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KSMConfigKeys.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ContainerTestHelper.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/helpers/VolumeArgs.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocol/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/proto/CBlockClientServerProtocol.proto * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocol/KeyspaceManagerProtocol.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/container/ContainerMapping.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/scm/ScmConfigKeys.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocolPB/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/protocol/VersionResponse.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocolPB/KeySpaceManagerProtocolPB.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/scm/container/common/helpers/Pipeline.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ChunkInfo.java > Ozone: KSM: Create Key Space manager service > > > Key: HDFS-11768 > URL: https://issues.apache.org/jira/browse/HDFS-11768 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-11768-HDFS-7240.001.patch, > HDFS-11768-HDFS-7240.002.patch, ozone-key-space.pdf > > > KSM is the namespace manager for Ozone. KSM relies on SCM to achieve block > functions. Ozone handler -- The rest protocol frontend talks to KSM and SCM > to get datanode addresses. > This JIRA will add the service as well as add the protobuf definitions needed > to work with KSM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[ https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450649#comment-16450649 ] Hudson commented on HDFS-11768: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14057/]) HDFS-11768. Ozone: KSM: Create Key Space manager service. Contributed by (aengineer: rev 2796b3435e07eac0782da74bf636052d27f58222) * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/KeySpaceManagerProtocol.proto * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/scm/ScmConfigKeys.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/scm/container/common/helpers/Pipeline.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerData.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/storage/OzoneContainerTranslation.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocolPB/KeySpaceManagerProtocolPB.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/StorageContainerLocationProtocol.proto * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/container/ContainerMapping.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/protocolPB/KeyspaceManagerProtocolServerSideTranslatorPB.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/helpers/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/pom.xml * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ContainerTestHelper.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KSMConfigKeys.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/protocol/VersionResponse.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/proto/StorageContainerDatanodeProtocol.proto * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/proto/CBlockClientServerProtocol.proto * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocolPB/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/scm/cli/SQLCLI.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/Ozone.proto * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/helpers/VolumeArgs.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/ksm/KeySpaceManager.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/DatanodeContainerProtocol.proto * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ChunkInfo.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocol/KeyspaceManagerProtocol.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/KeyData.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/scm/storage/ChunkOutputStream.java * (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/ksm/protocol/package-info.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/OzoneClientUtils.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/TestOzoneClientUtils.java > Ozone: KSM: Create Key Space manager service > > > Key: HDFS-11768 > URL: https://issues.apache.org/jira/browse/HDFS-11768 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-11768-HDFS-7240.001.patch, > HDFS-11768-HDFS-7240.002.patch, ozone-key-space.pdf > > > KSM is the namespace manager for Ozone. KSM relies on SCM to achieve block > functions. Ozone handler -- The rest protocol frontend talks to KSM and SCM > to get datanode addresses. > This JIRA will add the service as well as add the protobuf definitions needed > to work with KSM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail:
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[
https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007462#comment-16007462
]
Hadoop QA commented on HDFS-11768:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
3s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
40s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
39s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
35s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
56s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 10
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 2m
38s{color} | {color:red} The patch generated 1 new + 75 unchanged - 1 fixed =
76 total (was 76) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
26s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 44s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}122m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation |
| | hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[ https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007382#comment-16007382 ] Xiaoyu Yao commented on HDFS-11768: --- Thanks [~anu] for updating the patch. Patch v2 looks good to me. +1 pending Jenkins. I only have one minor issue that you can fixed at commit if Jenkins and other reviewers are OK. ScmConfigKeys.java has an empty line change after move all the KSM keys to KsmConfigKeys.java. > Ozone: KSM: Create Key Space manager service > > > Key: HDFS-11768 > URL: https://issues.apache.org/jira/browse/HDFS-11768 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11768-HDFS-7240.001.patch, > HDFS-11768-HDFS-7240.002.patch, ozone-key-space.pdf > > > KSM is the namespace manager for Ozone. KSM relies on SCM to achieve block > functions. Ozone handler -- The rest protocol frontend talks to KSM and SCM > to get datanode addresses. > This JIRA will add the service as well as add the protobuf definitions needed > to work with KSM. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[ https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005379#comment-16005379 ] Anu Engineer commented on HDFS-11768: - Great Question. This relates to how RPC systems work generally. The identity of who the caller is generally an orthogonal parameter to the RPC. This is to make it harder to spoof the identity. So while we don't have a security system wired up yet, when we do we will be able to return an ACCESS_DENIED error since the assumption is that RPC layer provides some kind of identity support. > Ozone: KSM: Create Key Space manager service > > > Key: HDFS-11768 > URL: https://issues.apache.org/jira/browse/HDFS-11768 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11768-HDFS-7240.001.patch, ozone-key-space.pdf > > > KSM is the namespace manager for Ozone. KSM relies on SCM to achieve block > functions. Ozone handler -- The rest protocol frontend talks to KSM and SCM > to get datanode addresses. > This JIRA will add the service as well as add the protobuf definitions needed > to work with KSM. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[
https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005320#comment-16005320
]
Chen Liang commented on HDFS-11768:
---
I have a question about how will the access check work.
Take {{deleteVolume}} for example, there is {{ACCESS_DENIED}} in .proto for
return status of deleteVolume. But the API {{void deleteVolume(String volume)
throws IOException;}} does not seem to have any information to do with the
permission of the user making this call. So when this call gets to the server,
how will the server know if the access should be granted or denied?
> Ozone: KSM: Create Key Space manager service
>
>
> Key: HDFS-11768
> URL: https://issues.apache.org/jira/browse/HDFS-11768
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Attachments: HDFS-11768-HDFS-7240.001.patch, ozone-key-space.pdf
>
>
> KSM is the namespace manager for Ozone. KSM relies on SCM to achieve block
> functions. Ozone handler -- The rest protocol frontend talks to KSM and SCM
> to get datanode addresses.
> This JIRA will add the service as well as add the protobuf definitions needed
> to work with KSM.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[ https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005206#comment-16005206 ] Xiaoyu Yao commented on HDFS-11768: --- Thanks [~anu] for working on this. The patch looks pretty good to me. I just have few minor comments below. *pom.xml* Should we rename KeyspaceManager.proto -> KeySpaceManager.proto *VolumeArgs.java* Line 33: can you add a comment on keyValueMap or rename? Line 41: extra space between param and keyValueMap *KeyspaceManagerProtocol.java* rename to KeySpaceManagerProtocol.java for consistency. Line 89: remove the extra "volume" *ScmConfigKeys.java* Should we have a separate KsmConfigKeys.java for these KSM keys *KeyspaceManager.proto* Can we pull KeyValue from DatanodeContainerProtocol.proto into Ozone.proto as it seems to be the only one used here. The SetVolumePropertyRequest does not support change the attributes via KeyValue Map. This can be added later with HDFS-11770. Can we add a test for KSM address/port similar to testGetScmDataNodeAddress? *KeyspaceManagerProtocolServerSideTranslatorPB.java* KeyspaceManagerProtocolServerSideTranslatorPB -> KeySpaceManagerProtocolServerSideTranslatorPB for consistency > Ozone: KSM: Create Key Space manager service > > > Key: HDFS-11768 > URL: https://issues.apache.org/jira/browse/HDFS-11768 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11768-HDFS-7240.001.patch, ozone-key-space.pdf > > > KSM is the namespace manager for Ozone. KSM relies on SCM to achieve block > functions. Ozone handler -- The rest protocol frontend talks to KSM and SCM > to get datanode addresses. > This JIRA will add the service as well as add the protobuf definitions needed > to work with KSM. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[
https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005024#comment-16005024
]
Weiwei Yang commented on HDFS-11768:
Hi [~anu]
Thanks for the work. Some comments below,
*KeyspaceManager.proto*
# Create volume API: I am a bit confused with {{adminName}} and {{ownerName}},
I thought a volume can be created by any valid user (not just ozone admin) and
the creator should be its owner. I don't understand the point to have an admin
and an owner.
# The response status from all volume operations seem to be similar, does it
make sense to reuse {{enum Status}} for all messages? And volume operations may
fail with I/O errors, does it make sense to add {{IO_ERROR}} to {{Status}}?
# {{VolumeInfo}} seems to need more fields, e.g data created, data modified,
size, volume status and volumeID. Or this might fit to a new message
{{VolumeStatus}} which can be used in {{InfoVolume}}.
# Delete volume API: why it is only allowed to delete a volume without any
bucket within? It seems pretty common for user to delete a volume owned by
themselves.
*ScmConfigKeys*
is it better to place ksm keys in a similar class such as {{KsmConfigKeys}} ?
*KeySpaceManager*
line 111 - java doc should be main entry for starting *KeySpaceManager*
> Ozone: KSM: Create Key Space manager service
>
>
> Key: HDFS-11768
> URL: https://issues.apache.org/jira/browse/HDFS-11768
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Attachments: HDFS-11768-HDFS-7240.001.patch, ozone-key-space.pdf
>
>
> KSM is the namespace manager for Ozone. KSM relies on SCM to achieve block
> functions. Ozone handler -- The rest protocol frontend talks to KSM and SCM
> to get datanode addresses.
> This JIRA will add the service as well as add the protobuf definitions needed
> to work with KSM.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[ https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004808#comment-16004808 ] Mukul Kumar Singh commented on HDFS-11768: -- Thanks for the patch [~anu], I have few minor comments. 1) KeySpaceManager.proro, in most of the proto files, "Proto" is suffixed to message names. 2) ScmConfigKeys, OZONE_KSM_PORT_KEY should be removed as there are no accessors right now. Should we assign this a correct value as part of updating the listen address. > Ozone: KSM: Create Key Space manager service > > > Key: HDFS-11768 > URL: https://issues.apache.org/jira/browse/HDFS-11768 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer > Attachments: HDFS-11768-HDFS-7240.001.patch, ozone-key-space.pdf > > > KSM is the namespace manager for Ozone. KSM relies on SCM to achieve block > functions. Ozone handler -- The rest protocol frontend talks to KSM and SCM > to get datanode addresses. > This JIRA will add the service as well as add the protobuf definitions needed > to work with KSM. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11768) Ozone: KSM: Create Key Space manager service
[
https://issues.apache.org/jira/browse/HDFS-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004269#comment-16004269
]
Hadoop QA commented on HDFS-11768:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
51s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
1s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
15s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
59s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s{color} | {color:green} HDFS-7240 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
14s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in HDFS-7240 has 17
extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
47s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in HDFS-7240
has 2 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} HDFS-7240 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 2m
41s{color} | {color:red} The patch generated 1 new + 75 unchanged - 1 fixed =
76 total (was 76) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
18s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}127m 19s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}178m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.cblock.TestCBlockCLI |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| |
