[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746813#comment-16746813 ] Hudson commented on HDDS-696: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15791 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15791/]) HDDS-696. Bootstrap genesis SCM(CA) with self-signed certificate. (xyao: rev 0c8829a9a1bcbeb017dd3288d4e0c56d4004a117) * (add) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/package-info.java * (add) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/utils/CertificateCodec.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/client/CertificateClient.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsConfigKeys.java * (add) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/CertificateSignRequest.java * (add) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/utils/SelfSignedCertificate.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/SecurityConfig.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/StorageInfo.java * (edit) hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/TestRootCertificate.java * (add) hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/keys/TestKeyCodec.java * (add) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/utils/package-info.java * (delete) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/HDDSKeyPEMWriter.java * (delete) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/SelfSignedCertificate.java * (add) hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/package-info.java * (add) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/DefaultCAServer.java * (edit) hadoop-hdds/common/src/test/java/org/apache/hadoop/utils/db/TestDBStoreBuilder.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestSecureOzoneCluster.java * (edit) hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/TestCertificateSignRequest.java * (delete) hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/keys/TestHDDSKeyPEMWriter.java * (edit) hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificates/package-info.java * (add) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/KeyCodec.java * (delete) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/CertificateSignRequest.java * (add) hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/authority/TestDefaultCAServer.java * (delete) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificates/package-info.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/certificate/authority/CertificateServer.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/security/x509/keys/HDDSKeyGenerator.java * (add) hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/security/x509/certificate/utils/TestCertificateCodec.java > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Anu Engineer >Priority: Major > Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch, > HDDS-696-HDDS-4.003.patch, HDDS-696-HDDS-4.004.patch > > > If security is enabled, SCM will generate the CA certs and bootstrap a CA. If > it is already bootstrapped it the keys and root certificates are read from > the secure store, if not, they are generated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[
https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701144#comment-16701144
]
Hadoop QA commented on HDDS-696:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
|| || || || {color:brown} HDDS-4 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
22s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
52s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
21s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 2s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
9s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} HDDS-4 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
11s{color} | {color:green} root: The patch generated 0 new + 3 unchanged - 3
fixed = 3 total (was 6) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 40s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 47s{color}
| {color:red} common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 42s{color}
| {color:red} integration-test in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}102m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-696 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949753/HDDS-696-HDDS-4.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 7ee9a2e6f2f1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701102#comment-16701102 ] Xiaoyu Yao commented on HDDS-696: - Thanks [~anu] for the update. Patch v4 looks good to me. +1, pending Jenkins. > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Anu Engineer >Priority: Major > Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch, > HDDS-696-HDDS-4.003.patch, HDDS-696-HDDS-4.004.patch > > > If security is enabled, SCM will generate the CA certs and bootstrap a CA. If > it is already bootstrapped it the keys and root certificates are read from > the secure store, if not, they are generated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[
https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701045#comment-16701045
]
Anu Engineer commented on HDDS-696:
---
[~xyao] Thanks for the review and comments. Patch v4 fixes those issues. Please
see more detailed comment below.
{quote}BlockTokenException.java#Line 26: NIT: accidental change can be removed.
{quote}
Fixed.
{quote}CertificateCodec.java - Files.setPosixFilePermissions already have it
coverred.
{quote}
You are absolutely right. Thanks for pointing this out. Removed this code. In
the KeyCodec, this function is used in test cases. I did not repeat the same
test in certificates, even though it was the idea.
{quote}static JcaX509CertificateConverter, This will be useful for CA. Also, we
need to call setProvider() to honor the "BC"
{quote}
Fixed , For the provider we want to use the default JAVA class here. When we
use the BC provider we get a parse error. I can investigate this more.
{quote}Line 201: basePath is not hornored in the code. (Same on Line 248)
{quote}
Fixed.
{quote}Line 255: need to use the getInstance with provider name parameter to
honor "BC" provider from security config.
{quote}
I am sorry, did you mean for the CertificateHolder?, that is a BC class not
from the JCA.
{quote}CertificateServer.java#Line 56: SCMSecurityException can be removed.
{quote}
Fixed.
{quote}CertificateSignRequest.java. The file location does not match the
package declaration
{quote}
Moved all files to certificates.utils.
{quote}DefaultCAServer.java# Line 63: NIT: can we start a new line for "1.
Success…", Line 84: NIT: typo: "success"
{quote}
Fixed.
{quote}Line 227/245: should we remove the securityConfig parameter and use the
member variable config instead if we could
{quote}
Fixed.
{quote}it has been initialized outside the DefaultCAServer anyway?
{quote}
The init call does that. Do you want this to be passed via ctor?
{quote}Line 65-68: NIT: let's be consistent with the order of "final static"
{quote}
Fixed.
{quote}Line 324 will throw if it is not posix, do we still need a separate
check here?
{quote}
I use this in tests to simulate failure as if the file system is not posix.
{quote}SelfSignedCertificate.java# Line 20: file need to be moved under
certificate.utils with the package name change.
{quote}
Fixed.
{quote}I think we should simply use endDate.atTime(LocalTime.MAX) to indicate
proper end time or
{quote}
Thanks, I converted both begin and endDate to use LocalTime.MIN and
LocalTime.MAX respectively.
{quote}Line 216: do we need to +1 considering we allow the certificate to be
valid from the begin
{quote}
Fixed.
> Bootstrap genesis SCM(CA) with self-signed certificate.
> ---
>
> Key: HDDS-696
> URL: https://issues.apache.org/jira/browse/HDDS-696
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Xiaoyu Yao
>Assignee: Anu Engineer
>Priority: Major
> Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch,
> HDDS-696-HDDS-4.003.patch, HDDS-696-HDDS-4.004.patch
>
>
> If security is enabled, SCM will generate the CA certs and bootstrap a CA. If
> it is already bootstrapped it the keys and root certificates are read from
> the secure store, if not, they are generated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1671#comment-1671 ] Anu Engineer commented on HDDS-696: --- Thank you for the comments, I will post a patch soon to address these. > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Anu Engineer >Priority: Major > Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch, > HDDS-696-HDDS-4.003.patch > > > If security is enabled, SCM will generate the CA certs and bootstrap a CA. If > it is already bootstrapped it the keys and root certificates are read from > the secure store, if not, they are generated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1668#comment-1668 ] Xiaoyu Yao commented on HDDS-696: - Thanks [~anu] for the patch. It looks pretty good to me. Here are a few minor comments: *BlockTokenException.java* Line 26: NIT: accidental change can be removed. *CertificateCodec.java* Line 103-106: can we put isPosix() into a util class so that the same code can be shared between CertificateCodec and KeyCodec. After second check, I think we don't need this as it is not being used in the code. Line 221 calls Files.setPosixFilePermissions already have it coverred. Line 117-118: should we have a static JcaX509CertificateConverter so that we don't have to create each time. This will be useful for CA. Also, we need to call setProvider() to honor the "BC" as provider from the Securityconfig. Line 201: basePath is not hornored in the code. (Same on Line 248) Line 203: SCMSecurityException is not needed to be declared here as it is a subclass of IOException. Line 255: need to use the getInstance with provider name parameter to honor "BC" provider from security config. *CertificateServer.java* Line 56: SCMSecurityException can be removed. *CertificateSignRequest.java* The file location does not match the package declaration. *DefaultCAServer.java* Line 63: NIT: can we start a new line for "1. Success…" Line 84: NIT: typo: "success" Line 227/245: should we remove the securityConfig parameter and use the member variable config instead if we could let SecurityConfig passed into DefaultCAServer contstructor (like other class such as KeyCodec/HDDSKeyGenerator) and it has been initialized outside the DefaultCAServer anyway? *KeyCodec.java* Line 65-68: NIT: let's be consistent with the order of "final static" Line 315-319: Line 324 will throw if it is not posix, do we still need a separate check here? *OmMetadataManagerImpl.java* Line 160: NIT: empty line change can be removed. *Package-info.java* Line 22: the package for main/…/x509/certificates should not change its package name to main/…/x509.certificate.utils test/…/x509/certificates should not change its package name to test/…/x509.certificate.utils If they are moved under utils, we might be able to remove these files. *SelfSignedCertificate.java* Line 20: file need to be moved under certificate.utils with the package name change. Line 132-133: I think we should simply use endDate.atTime(LocalTime.MAX) to indicate proper end time or a slightly complex one like endDate.atStartOfDay().plusDays(1).minusSeconds(1).toInstant(zoneOffset); Line 216: do we need to +1 considering we allow the certificate to be valid from the begin of the beginDate to the end of the endDate. Line 219: this should be > 0, i.e., when certDuration > maxDuration we throw. > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Anu Engineer >Priority: Major > Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch, > HDDS-696-HDDS-4.003.patch > > > If security is enabled, SCM will generate the CA certs and bootstrap a CA. If > it is already bootstrapped it the keys and root certificates are read from > the secure store, if not, they are generated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[
https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699845#comment-16699845
]
Hadoop QA commented on HDDS-696:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
|| || || || {color:brown} HDDS-4 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
32s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
54s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
26s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
6s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 53s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} HDDS-4 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
11s{color} | {color:green} root: The patch generated 0 new + 3 unchanged - 3
fixed = 3 total (was 6) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 56s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
37s{color} | {color:red} hadoop-hdds_common generated 1 new + 1 unchanged - 0
fixed = 2 total (was 1) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 38s{color}
| {color:red} common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 31s{color}
| {color:red} ozone-manager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 32s{color}
| {color:red} integration-test in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}102m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-696 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949582/HDDS-696-HDDS-4.003.patch
|
| Optional Tests | asflicense compile javac
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699770#comment-16699770 ] Anu Engineer commented on HDDS-696: --- Patch v3 addresses the rest of the comments. Thanks. > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Anu Engineer >Priority: Major > Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch, > HDDS-696-HDDS-4.003.patch > > > If security is enabled, SCM will generate the CA certs and bootstrap a CA. If > it is already bootstrapped it the keys and root certificates are read from > the secure store, if not, they are generated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699765#comment-16699765 ] Anu Engineer commented on HDDS-696: --- bq. TestSecureOzoneCluster L80 Shall we change it to static final while keeping the name in uppercase. Not sure what this is referring to. L80 with the patch applied does not point to something that can be renamed. > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Anu Engineer >Priority: Major > Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch > > > If security is enabled, SCM will generate the CA certs and bootstrap a CA. If > it is already bootstrapped it the keys and root certificates are read from > the secure store, if not, they are generated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699544#comment-16699544 ] Anu Engineer commented on HDDS-696: --- Thanks for the comments. bq. Shall we move generateKeys, checkIfKeysExist and checkIfCertificatesExist to a util class. They can be used in Tests and few other places? It is a good idea to have a version of these in the client eventually. But I would prefer those functions to actually decode these files. In the CA, we only check if the file exists, since the next function call will decode them. So I when I get to the client functions I will add them in a proper way, that does not only check for file existence, but also decode the objects in question. bq. Are you planning to implement requestCertificate and revokeCertificate separately? Yes, in the next patch. This patch is already too big. bq. getCertificateLocation should include component part as well as same node may have multiple components and hence multiple certs? I agree we do have an overloaded function with component, in this patch. Line 204 after this patch is applied. I will fix the rest of the issues and upload a new patch. > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Anu Engineer >Priority: Major > Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch > > > If security is enabled, SCM will generate the CA certs and bootstrap a CA. If > it is already bootstrapped it the keys and root certificates are read from > the secure store, if not, they are generated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[
https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698359#comment-16698359
]
Ajay Kumar commented on HDDS-696:
-
[~anu] thanks for working on this. Seems patch doesn't apply to branch anymore.
Could you please rebase it.
Had a high level look on patch:
# DefaultCAServer
## Shall we move {{generateKeys, checkIfKeysExist and
checkIfCertificatesExist}} to a util class. They can be used in Tests and few
other places?
## Are you planning to implement requestCertificate and revokeCertificate
separately?
# SecurityConfig
## getCertificateLocation should include component part as well as same node
may have multiple components and hence multiple certs?
## Typo at L179/198: "addition component"
## L210: space in config name "hdds.key .len"
# TestSecureOzoneCluster L80 Shall we change it to static final while keeping
the name in uppercase.
> Bootstrap genesis SCM(CA) with self-signed certificate.
> ---
>
> Key: HDDS-696
> URL: https://issues.apache.org/jira/browse/HDDS-696
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Xiaoyu Yao
>Assignee: Anu Engineer
>Priority: Major
> Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch
>
>
> If security is enabled, SCM will generate the CA certs and bootstrap a CA. If
> it is already bootstrapped it the keys and root certificates are read from
> the secure store, if not, they are generated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[
https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695404#comment-16695404
]
Hadoop QA commented on HDDS-696:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 13 new or modified test
files. {color} |
|| || || || {color:brown} HDDS-4 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 7m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
14s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
2s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
41s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
0s{color} | {color:green} HDDS-4 passed {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 18m
9s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
45s{color} | {color:green} HDDS-4 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
38s{color} | {color:green} root: The patch generated 0 new + 3 unchanged - 4
fixed = 3 total (was 7) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 11m
50s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 46s{color}
| {color:red} common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 35s{color}
| {color:red} ozone-manager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 39s{color}
| {color:red} integration-test in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}116m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-696 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949121/HDDS-696-HDDS-4.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695303#comment-16695303 ] Anu Engineer commented on HDDS-696: --- Patch v2 fixes CheckStyle and findbugs issues. > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Anu Engineer >Priority: Major > Attachments: HDDS-696-HDDS-4.001.patch, HDDS-696-HDDS-4.002.patch > > > If security is enabled, SCM will generate the CA certs and bootstrap a CA. If > it is already bootstrapped it the keys and root certificates are read from > the secure store, if not, they are generated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[
https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695262#comment-16695262
]
Hadoop QA commented on HDDS-696:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 12 new or modified test
files. {color} |
|| || || || {color:brown} HDDS-4 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
53s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
23s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
3s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
36s{color} | {color:green} HDDS-4 passed {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 15m
35s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s{color} | {color:green} HDDS-4 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} HDDS-4 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
48s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 4s{color} | {color:orange} root: The patch generated 8 new + 5 unchanged -
2 fixed = 13 total (was 7) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 9m
59s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
15s{color} | {color:red} hadoop-hdds/common generated 1 new + 0 unchanged - 0
fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 41s{color}
| {color:red} common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 30s{color}
| {color:red} ozone-manager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 34s{color}
| {color:red} integration-test in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 99m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdds/common |
| | Dead store to x in
org.apache.hadoop.hdds.security.x509.certificate.utils.SelfSignedCertificate$Builder.build()
At
[jira] [Commented] (HDDS-696) Bootstrap genesis SCM(CA) with self-signed certificate.
[ https://issues.apache.org/jira/browse/HDDS-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687182#comment-16687182 ] Arpit Agarwal commented on HDDS-696: iiic this blocks HDDS-102, HDDS-134 and HDDS-594. > Bootstrap genesis SCM(CA) with self-signed certificate. > --- > > Key: HDDS-696 > URL: https://issues.apache.org/jira/browse/HDDS-696 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > > This can be done in the following two scenarios: > 1) scm has not been "-init"-ed > If ozone security is enabled, we will bootstrap genesis CA along with "scm > --init". > 2) scm has been "--init"-ed but without security enabled. > Now, we want to enable security on an non-secure scm cluster. This can be > done with > "scm --init -security" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
