[jira] [Commented] (HDDS-363) Faster datanode registration during the first startup
[ https://issues.apache.org/jira/browse/HDDS-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591718#comment-16591718 ] Hudson commented on HDDS-363: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14824 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14824/]) HDDS-363. Faster datanode registration during the first startup. (elek: rev 138b0c1443ae485ba90b953362d6dea89c77a758) * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/StateContext.java * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/datanode/RunningDatanodeState.java * (edit) hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/ozone/container/common/TestEndPoint.java * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeStateMachine.java * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/RegisterEndpointTask.java > Faster datanode registration during the first startup > - > > Key: HDDS-363 > URL: https://issues.apache.org/jira/browse/HDDS-363 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Minor > Fix For: 0.2.1 > > Attachments: HDDS-363.001.patch, HDDS-363.002.patch > > > During the first startup usually we need to wait about 30 s to find the scm > usable. The datanode registration is a multiple step process > (request/response + request/response) and we need to wait the next HB to > finish the registration. > I propose to use a more higher HB frequency at startup (let's say 2 seconds) > and set the configured HB only at the end of the registration. > It also helps for the first users as it could be less confusing (the datanode > can be seen almost immediately on the UI) > Also it would help a lot for me during the testing (yes, I can decrease the > HB frequency but in that case it's harder the follow the later HBs) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-363) Faster datanode registration during the first startup
[ https://issues.apache.org/jira/browse/HDDS-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591677#comment-16591677 ] Elek, Marton commented on HDDS-363: --- Thanks the review [~hanishakoneru]. Will commit it to the trunk soon > Faster datanode registration during the first startup > - > > Key: HDDS-363 > URL: https://issues.apache.org/jira/browse/HDDS-363 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Minor > Fix For: 0.2.1 > > Attachments: HDDS-363.001.patch, HDDS-363.002.patch > > > During the first startup usually we need to wait about 30 s to find the scm > usable. The datanode registration is a multiple step process > (request/response + request/response) and we need to wait the next HB to > finish the registration. > I propose to use a more higher HB frequency at startup (let's say 2 seconds) > and set the configured HB only at the end of the registration. > It also helps for the first users as it could be less confusing (the datanode > can be seen almost immediately on the UI) > Also it would help a lot for me during the testing (yes, I can decrease the > HB frequency but in that case it's harder the follow the later HBs) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-363) Faster datanode registration during the first startup
[ https://issues.apache.org/jira/browse/HDDS-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590833#comment-16590833 ] Hanisha Koneru commented on HDDS-363: - Thanks for the patch, [~elek]. LGTM, +1. > Faster datanode registration during the first startup > - > > Key: HDDS-363 > URL: https://issues.apache.org/jira/browse/HDDS-363 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Minor > Fix For: 0.2.1 > > Attachments: HDDS-363.001.patch, HDDS-363.002.patch > > > During the first startup usually we need to wait about 30 s to find the scm > usable. The datanode registration is a multiple step process > (request/response + request/response) and we need to wait the next HB to > finish the registration. > I propose to use a more higher HB frequency at startup (let's say 2 seconds) > and set the configured HB only at the end of the registration. > It also helps for the first users as it could be less confusing (the datanode > can be seen almost immediately on the UI) > Also it would help a lot for me during the testing (yes, I can decrease the > HB frequency but in that case it's harder the follow the later HBs) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-363) Faster datanode registration during the first startup
[ https://issues.apache.org/jira/browse/HDDS-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589196#comment-16589196 ] genericqa commented on HDDS-363: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 28s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 19s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 37s{color} | {color:green} container-service in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 22s{color} | {color:green} server-scm in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 25s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 59m 24s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | HDDS-363 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12936676/HDDS-363.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 6462af164138 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 8184739 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_171 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-HDDS-Build/812/testReport/ | | Max. process+thread count | 336 (vs. ulimit of 1) | | modules | C:
[jira] [Commented] (HDDS-363) Faster datanode registration during the first startup
[ https://issues.apache.org/jira/browse/HDDS-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589112#comment-16589112 ] Elek, Marton commented on HDDS-363: --- Acceptance tests are passed: {code} == Acceptance == Acceptance.Basic == Acceptance.Basic.Basic :: Smoketest ozone cluster startup == Test rest interface | PASS | -- Check webui static resources | PASS | -- Start freon testing | PASS | -- Acceptance.Basic.Basic :: Smoketest ozone cluster startup | PASS | 3 critical tests, 3 passed, 0 failed 3 tests total, 3 passed, 0 failed == Acceptance.Basic.Ozone-Shell :: Test ozone shell CLI usage == RestClient without http port | PASS | -- RestClient with http port | PASS | -- RestClient without host name | PASS | -- RpcClient with port | PASS | -- RpcClient without host| PASS | -- RpcClient without scheme | PASS | -- Acceptance.Basic.Ozone-Shell :: Test ozone shell CLI usage| PASS | 6 critical tests, 6 passed, 0 failed 6 tests total, 6 passed, 0 failed == Acceptance.Basic | PASS | 9 critical tests, 9 passed, 0 failed 9 tests total, 9 passed, 0 failed == Acceptance.Ozonefs == Acceptance.Ozonefs.Ozonefs :: Ozonefs test == Create volume and bucket | PASS | -- Check volume from ozonefs | PASS | -- Create directory from ozonefs | PASS | -- Acceptance.Ozonefs.Ozonefs :: Ozonefs test| PASS | 3 critical tests, 3 passed, 0 failed 3 tests total, 3 passed, 0 failed == Acceptance.Ozonefs.Ozonesinglenode :: Ozonefs Single Node Test == Create volume and bucket | PASS | -- Check volume from ozonefs | PASS | -- Create directory from ozonefs | PASS | -- Test key handling | PASS | -- Acceptance.Ozonefs.Ozonesinglenode :: Ozonefs Single Node Test| PASS | 4 critical tests, 4
[jira] [Commented] (HDDS-363) Faster datanode registration during the first startup
[ https://issues.apache.org/jira/browse/HDDS-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589106#comment-16589106 ] genericqa commented on HDDS-363: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 39s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 47s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 9s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 26s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 27s{color} | {color:red} hadoop-hdds_container-service generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 54s{color} | {color:green} container-service in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 26s{color} | {color:green} server-scm in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 26s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 63m 38s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 | | JIRA Issue | HDDS-363 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12936668/HDDS-363.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux c30ef0b02427 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 8184739 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_171 | | findbugs | v3.1.0-RC1 | | javadoc |
[jira] [Commented] (HDDS-363) Faster datanode registration during the first startup
[ https://issues.apache.org/jira/browse/HDDS-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584078#comment-16584078 ] Ajay Kumar commented on HDDS-363: - agree, registration can be little more aggressive. Once registered datanode can fallback to configured HB. > Faster datanode registration during the first startup > - > > Key: HDDS-363 > URL: https://issues.apache.org/jira/browse/HDDS-363 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Minor > Fix For: 0.2.1 > > > During the first startup usually we need to wait about 30 s to find the scm > usable. The datanode registration is a multiple step process > (request/response + request/response) and we need to wait the next HB to > finish the registration. > I propose to use a more higher HB frequency at startup (let's say 2 seconds) > and set the configured HB only at the end of the registration. > It also helps for the first users as it could be less confusing (the datanode > can be seen almost immediately on the UI) > Also it would help a lot for me during the testing (yes, I can decrease the > HB frequency but in that case it's harder the follow the later HBs) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org