[jira] [Updated] (HDDS-3762) Intermittent failure in TestDeleteWithSlowFollower

2020-09-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3762: - Labels: pull-request-available (was: ) > Intermittent failure in TestDeleteWithSlowFollower >

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #1376: HDDS-3762. Intermittent failure in TestDeleteWithSlowFollower

2020-09-01 Thread GitBox
adoroszlai opened a new pull request #1376: URL: https://github.com/apache/hadoop-ozone/pull/1376 ## What changes were proposed in this pull request? Intermittent failure in `testDeleteKeyWithSlowFollower` seems to be caused by: * `DeleteBlocksCommandHandler` increments

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481746270 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
adoroszlai commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481709338 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[jira] [Commented] (HDDS-3103) Have multi-raft pipeline calculator to recommend best pipeline number per datanode

2020-09-01 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188980#comment-17188980 ] Rui Wang commented on HDDS-3103: [~timmylicheng] Do you have a plan to work on this JIRA in near term?

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481638257 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1375: HDDS-4189. Change `ozone admin om getserviceroles` to `ozone admin om status`

2020-09-01 Thread GitBox
amaliujia commented on pull request #1375: URL: https://github.com/apache/hadoop-ozone/pull/1375#issuecomment-685278548 It seemed that there is another opinion about what command name should be. So I will convert this PR to a draft now to wait for a consensus. Thanks for all

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1340: HDDS-3188 Enable SCM group with failover proxy for SCM block location.

2020-09-01 Thread GitBox
amaliujia commented on pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#issuecomment-685271344 This PR overall LGTM This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop-ozone] vivekratnavel commented on pull request #1364: HDDS-4165. GitHub Actions cache does not work outside of workspace

2020-09-01 Thread GitBox
vivekratnavel commented on pull request #1364: URL: https://github.com/apache/hadoop-ozone/pull/1364#issuecomment-685272723 @adoroszlai Thanks for fixing this! +1 LGTM This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1375: HDDS-4189. Change `ozone admin om getserviceroles` to `ozone admin om status`

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1375: URL: https://github.com/apache/hadoop-ozone/pull/1375#discussion_r481610070 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/GetServiceRolesSubcommand.java ## @@ -30,10 +30,10 @@ import

[jira] [Assigned] (HDDS-3762) Intermittent failure in TestDeleteWithSlowFollower

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai reassigned HDDS-3762: -- Assignee: Attila Doroszlai > Intermittent failure in TestDeleteWithSlowFollower >

[GitHub] [hadoop-ozone] cxorm commented on pull request #1367: HDDS-4169. Fix some minor errors in StorageContainerManager.md

2020-09-01 Thread GitBox
cxorm commented on pull request #1367: URL: https://github.com/apache/hadoop-ozone/pull/1367#issuecomment-685254966 The failed `build-branch / unit (pull_request)` seems not related the patch. Let's commit it again. This

[jira] [Commented] (HDDS-3762) Intermittent failure in TestDeleteWithSlowFollower

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188941#comment-17188941 ] Attila Doroszlai commented on HDDS-3762: I think the bug from description was fixed by HDDS-3964:

[GitHub] [hadoop-ozone] cxorm commented on a change in pull request #1375: HDDS-4189. Change `ozone admin om getserviceroles` to `ozone admin om status`

2020-09-01 Thread GitBox
cxorm commented on a change in pull request #1375: URL: https://github.com/apache/hadoop-ozone/pull/1375#discussion_r481576976 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/GetServiceRolesSubcommand.java ## @@ -30,10 +30,10 @@ import

[GitHub] [hadoop-ozone] captainzmc commented on pull request #868: HDDS-3457. Fix ozonefs put and mkdir KEY_NOT_FOUND issue when ACL enable

2020-09-01 Thread GitBox
captainzmc commented on pull request #868: URL: https://github.com/apache/hadoop-ozone/pull/868#issuecomment-685244606 > Hi @captainzmc, > I enabled ACL and performed mkdir using o3fs, but 'checkAccess' function is never reached. I have put some System.out.println statements in

[jira] [Updated] (HDDS-4176) Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4176: - Labels: pull-request-available (was: ) > Fix failed UT: test2WayCommitForTimeoutException >

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481557217 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[jira] [Updated] (HDDS-4176) Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4176: --- Labels: (was: pull-request-available) > Fix failed UT: test2WayCommitForTimeoutException >

[jira] [Updated] (HDDS-4176) Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4176: --- Component/s: test > Fix failed UT: test2WayCommitForTimeoutException >

[jira] [Resolved] (HDDS-4176) Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4176. Fix Version/s: 1.1.0 Resolution: Fixed > Fix failed UT:

[GitHub] [hadoop-ozone] adoroszlai merged pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread GitBox
adoroszlai merged pull request #1370: URL: https://github.com/apache/hadoop-ozone/pull/1370 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread GitBox
adoroszlai commented on pull request #1370: URL: https://github.com/apache/hadoop-ozone/pull/1370#issuecomment-685243200 Thanks @runzhiwang for the fix and @amaliujia for the review. This is an automated message from the

[jira] [Commented] (HDDS-4190) Intermittent failure in TestOMAllocateBlockRequest#testValidateAndUpdateCache

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188923#comment-17188923 ] Attila Doroszlai commented on HDDS-4190: Similar assertions (failing intermittently when run

[jira] [Created] (HDDS-4190) Intermittent failure in TestOMAllocateBlockRequest#testValidateAndUpdateCache

2020-09-01 Thread Attila Doroszlai (Jira)
Attila Doroszlai created HDDS-4190: -- Summary: Intermittent failure in TestOMAllocateBlockRequest#testValidateAndUpdateCache Key: HDDS-4190 URL: https://issues.apache.org/jira/browse/HDDS-4190

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1375: HDDS-4189. Change `ozone admin om getserviceroles` to `ozone admin om status`

2020-09-01 Thread GitBox
adoroszlai commented on a change in pull request #1375: URL: https://github.com/apache/hadoop-ozone/pull/1375#discussion_r481521954 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/om/GetServiceRolesSubcommand.java ## @@ -30,10 +30,10 @@ import

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1375: HDDS-4189. Change `ozone admin om getserviceroles` to `ozone admin om status`

2020-09-01 Thread GitBox
amaliujia commented on pull request #1375: URL: https://github.com/apache/hadoop-ozone/pull/1375#issuecomment-685211998 Thank you for your review @cxorm! ``` [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.417 s <<< FAILURE! - in

[GitHub] [hadoop-ozone] github-actions[bot] closed pull request #912: WIP Patch - HDDS-2949: store dir/key entries in separate tables - first patch onl…

2020-09-01 Thread GitBox
github-actions[bot] closed pull request #912: URL: https://github.com/apache/hadoop-ozone/pull/912 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop-ozone] github-actions[bot] commented on pull request #912: WIP Patch - HDDS-2949: store dir/key entries in separate tables - first patch onl…

2020-09-01 Thread GitBox
github-actions[bot] commented on pull request #912: URL: https://github.com/apache/hadoop-ozone/pull/912#issuecomment-685206902 Thank you very much for the patch. I am closing this PR __temporarily__ as there was no activity recently and it is waiting for response from its author.

[GitHub] [hadoop-ozone] github-actions[bot] commented on pull request #1173: HDDS-3880. Improve OM HA Robot test

2020-09-01 Thread GitBox
github-actions[bot] commented on pull request #1173: URL: https://github.com/apache/hadoop-ozone/pull/1173#issuecomment-685206895 Thank you very much for the patch. I am closing this PR __temporarily__ as there was no activity recently and it is waiting for response from its author.

[GitHub] [hadoop-ozone] github-actions[bot] closed pull request #1173: HDDS-3880. Improve OM HA Robot test

2020-09-01 Thread GitBox
github-actions[bot] closed pull request #1173: URL: https://github.com/apache/hadoop-ozone/pull/1173 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread GitBox
runzhiwang commented on a change in pull request #1370: URL: https://github.com/apache/hadoop-ozone/pull/1370#discussion_r481493152 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java ## @@ -297,9 +298,11 @@

[GitHub] [hadoop-ozone] cxorm commented on pull request #1075: HDDS-3369. Cleanup old write-path of volume in OM

2020-09-01 Thread GitBox
cxorm commented on pull request #1075: URL: https://github.com/apache/hadoop-ozone/pull/1075#issuecomment-685191223 Thank you @adoroszlai for the advise. Splitting `OzoneManagerProtocol` interface is huge work IMHO. I think it is improper to change the interface after

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1375: HDDS-4189. Change `ozone admin om getserviceroles` to `ozone admin om status`

2020-09-01 Thread GitBox
amaliujia commented on pull request #1375: URL: https://github.com/apache/hadoop-ozone/pull/1375#issuecomment-685183677 Context: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481380452 This is an automated

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

2020-09-01 Thread GitBox
amaliujia commented on pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#issuecomment-685181887 Thanks @runzhiwang. This is an awesome work! I will also try to help review this PR. This is an

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1375: HDDS-4189. Change `ozone admin om getserviceroles` to `ozone admin om status`

2020-09-01 Thread GitBox
amaliujia commented on pull request #1375: URL: https://github.com/apache/hadoop-ozone/pull/1375#issuecomment-685181181 R: @adoroszlai @timmylicheng This is an automated message from the Apache Git Service. To respond to

[jira] [Updated] (HDDS-4189) Use a unified Cli syntax for both getting OM and SCM status

2020-09-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4189: - Labels: pull-request-available (was: ) > Use a unified Cli syntax for both getting OM and SCM

[GitHub] [hadoop-ozone] amaliujia opened a new pull request #1375: HDDS-4189. Change `ozone admin om getserviceroles` to `ozone admin om status`

2020-09-01 Thread GitBox
amaliujia opened a new pull request #1375: URL: https://github.com/apache/hadoop-ozone/pull/1375 ## What changes were proposed in this pull request? Change `ozone admin om getserviceroles` to `ozone admin om status` ## What is the link to the Apache JIRA

[GitHub] [hadoop-ozone] cxorm edited a comment on pull request #1233: HDDS-3725. Ozone sh volume client support quota option.

2020-09-01 Thread GitBox
cxorm edited a comment on pull request #1233: URL: https://github.com/apache/hadoop-ozone/pull/1233#issuecomment-685179614 Thank you @captainzmc for updating the PR. Could you rebase it with latest master branch ? This PR looks good to me, +1 (Would you be so kind to take a look

[GitHub] [hadoop-ozone] cxorm commented on pull request #1233: HDDS-3725. Ozone sh volume client support quota option.

2020-09-01 Thread GitBox
cxorm commented on pull request #1233: URL: https://github.com/apache/hadoop-ozone/pull/1233#issuecomment-685179614 Thank you @captainzmc for updating the PR. Could you rebase it with latest master branch ? This PR looks good to me, +1 (cc @ChenSammi )

[GitHub] [hadoop-ozone] cxorm commented on a change in pull request #1233: HDDS-3725. Ozone sh volume client support quota option.

2020-09-01 Thread GitBox
cxorm commented on a change in pull request #1233: URL: https://github.com/apache/hadoop-ozone/pull/1233#discussion_r481477852 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/protocol/ClientProtocol.java ## @@ -101,10 +100,11 @@ void

[jira] [Created] (HDDS-4189) Use a unified Cli syntax for both getting OM and SCM status

2020-09-01 Thread Rui Wang (Jira)
Rui Wang created HDDS-4189: -- Summary: Use a unified Cli syntax for both getting OM and SCM status Key: HDDS-4189 URL: https://issues.apache.org/jira/browse/HDDS-4189 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#issuecomment-685173208 Addressed the following comments 1. Merge `getRatisStatus` with `GetScmInfo` 2. Adopt command syntax `ozone admin scm status` 3. added an acceptance test

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481469131 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocol.java ## @@ -230,4 +230,5 @@

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481469412 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestScmAdminHA.java ## @@ -0,0 +1,66 @@ +package

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481414123 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] hanishakoneru commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-01 Thread GitBox
hanishakoneru commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r476042841 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerDataYaml.java ## @@ -280,6

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1353: HDDS-4148. Add servlet to return SCM DB checkpoint.

2020-09-01 Thread GitBox
prashantpogde commented on a change in pull request #1353: URL: https://github.com/apache/hadoop-ozone/pull/1353#discussion_r481431139 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/DBCheckpointServlet.java ## @@ -0,0 +1,197 @@ +/** + *

[jira] [Resolved] (HDDS-3938) Flaky TestWatchForCommit#test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-3938. Resolution: Duplicate > Flaky TestWatchForCommit#test2WayCommitForTimeoutException >

[jira] [Resolved] (HDDS-4012) FLAKY-UT: TestWatchForCommit#test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4012. Resolution: Duplicate > FLAKY-UT: TestWatchForCommit#test2WayCommitForTimeoutException >

[jira] [Updated] (HDDS-4012) FLAKY-UT: TestWatchForCommit#test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4012: --- Labels: (was: pull-request-available) > FLAKY-UT:

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread GitBox
adoroszlai commented on a change in pull request #1370: URL: https://github.com/apache/hadoop-ozone/pull/1370#discussion_r481426506 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java ## @@ -297,9 +298,11 @@

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1353: HDDS-4148. Add servlet to return SCM DB checkpoint.

2020-09-01 Thread GitBox
xiaoyuyao commented on a change in pull request #1353: URL: https://github.com/apache/hadoop-ozone/pull/1353#discussion_r481416948 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/DBCheckpointServlet.java ## @@ -0,0 +1,197 @@ +/** + * Licensed to

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481414123 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
adoroszlai commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481380452 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481378454 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java ## @@ -560,6 +563,13 @@ public

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481371486 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1366: HDDS-4167. Acceptance test logs missing if SCM fails to exit safe mode

2020-09-01 Thread GitBox
adoroszlai commented on pull request #1366: URL: https://github.com/apache/hadoop-ozone/pull/1366#issuecomment-685067401 Thanks @elek for reviewing and committing it. This is an automated message from the Apache Git Service.

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
adoroszlai commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481359383 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[jira] [Assigned] (HDDS-4179) Implement post-finalize SCM logic to allow nodes of only new version to participate in pipelines.

2020-09-01 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-4179: - Assignee: Prashant Pogde > Implement post-finalize SCM logic to allow nodes of only new

[jira] [Assigned] (HDDS-4178) SCM Finalize command implementation.

2020-09-01 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-4178: - Assignee: Istvan Fajth > SCM Finalize command implementation. >

[jira] [Assigned] (HDDS-4174) Add current HDDS layout version to Datanode heartbeat and registration.

2020-09-01 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-4174: - Assignee: Prashant Pogde > Add current HDDS layout version to Datanode heartbeat and

[jira] [Assigned] (HDDS-4175) Implement Datanode Finalization

2020-09-01 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-4175: - Assignee: Istvan Fajth > Implement Datanode Finalization >

[jira] [Assigned] (HDDS-4173) Implement HDDS Version management using the LayoutVersionManager interface.

2020-09-01 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-4173: - Assignee: Prashant Pogde > Implement HDDS Version management using the

[jira] [Commented] (HDDS-4181) Add acceptance tests for upgrade, finalization and downgrade

2020-09-01 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188636#comment-17188636 ] Aravindan Vijayan commented on HDDS-4181: - Thanks [~elek]. I have seen that changes, and planning

[jira] [Comment Edited] (HDDS-4181) Add acceptance tests for upgrade, finalization and downgrade

2020-09-01 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188636#comment-17188636 ] Aravindan Vijayan edited comment on HDDS-4181 at 9/1/20, 4:49 PM: -- Thanks

[jira] [Commented] (HDDS-4097) S3/Ozone Filesystem inter-op

2020-09-01 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188586#comment-17188586 ] Arpit Agarwal commented on HDDS-4097: - If you always create them, then you are basically interpreting

[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

2020-09-01 Thread GitBox
bharatviswa504 edited a comment on pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328#issuecomment-684950217 @elek There is no issue with this, and I believe we have agreed on the part we need this like 100% HCFS with few changes to AWS S3 semantics. I don't see

[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

2020-09-01 Thread GitBox
bharatviswa504 edited a comment on pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328#issuecomment-684950217 @elek There is no issue with this, and I believe we have agreed on the part we need this like 100% HCFS with few changes to AWS S3 semantics. I don't see

[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

2020-09-01 Thread GitBox
bharatviswa504 edited a comment on pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328#issuecomment-684950217 @elek There is no issue with this. As the other argument is when the flag is disabled to support HCFS. And I believe we have agreed on the part we

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

2020-09-01 Thread GitBox
bharatviswa504 commented on pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328#issuecomment-684950217 @elek There is no issue with this. As the other argument is when the flag is disabled to support HCFS. And I believe we have agreed on the part we need

[jira] [Commented] (HDDS-4097) S3/Ozone Filesystem inter-op

2020-09-01 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188544#comment-17188544 ] Marton Elek commented on HDDS-4097: --- > Unfortunately there is no way you can guarantee that. A

[GitHub] [hadoop-ozone] arp7 commented on pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

2020-09-01 Thread GitBox
arp7 commented on pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328#issuecomment-684929618 > What is your opinion about AWS compatibility issues? What exactly is the compatibility issue? If the paths are not interpreted (default behavior), then there is full

[jira] [Comment Edited] (HDDS-4097) S3/Ozone Filesystem inter-op

2020-09-01 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188538#comment-17188538 ] Arpit Agarwal edited comment on HDDS-4097 at 9/1/20, 3:12 PM: -- bq. Using

[jira] [Commented] (HDDS-4097) S3/Ozone Filesystem inter-op

2020-09-01 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188538#comment-17188538 ] Arpit Agarwal commented on HDDS-4097: - bq. Using simple, acceptable key names (/a/b/c, /a/b/c/d) both

[GitHub] [hadoop-ozone] elek commented on pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

2020-09-01 Thread GitBox
elek commented on pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328#issuecomment-684923522 > @elek Any more comments? Looks like we don't have the consensus, yet, in HDDS-4097. It was discussed yesterday during the community meeting, and I felt that everybody

[jira] [Commented] (HDDS-4097) S3/Ozone Filesystem inter-op

2020-09-01 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188536#comment-17188536 ] Marton Elek commented on HDDS-4097: --- [~arp] It was discussed with more details during the community sync

[GitHub] [hadoop-ozone] elek commented on pull request #1363: HDDS-3805. [OFS] Remove usage of OzoneClientAdapter interface

2020-09-01 Thread GitBox
elek commented on pull request #1363: URL: https://github.com/apache/hadoop-ozone/pull/1363#issuecomment-684914342 Thanks for the separation @smengcl, I think it's easier to discuss. I am not sure it's the same patch what I already commented or not. I had a comment in the previous

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481146665 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestScmAdminHA.java ## @@ -0,0 +1,66 @@ +package

[jira] [Updated] (HDDS-4167) Acceptance test logs missing if SCM fails to exit safe mode

2020-09-01 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDDS-4167: -- Fix Version/s: 1.1.0 Resolution: Fixed Status: Resolved (was: Patch Available) >

[GitHub] [hadoop-ozone] elek merged pull request #1366: HDDS-4167. Acceptance test logs missing if SCM fails to exit safe mode

2020-09-01 Thread GitBox
elek merged pull request #1366: URL: https://github.com/apache/hadoop-ozone/pull/1366 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] elek commented on pull request #1366: HDDS-4167. Acceptance test logs missing if SCM fails to exit safe mode

2020-09-01 Thread GitBox
elek commented on pull request #1366: URL: https://github.com/apache/hadoop-ozone/pull/1366#issuecomment-684899079 > Reduce some code duplication between test-all.sh and ozone-mr/test.sh by extracting functions for the shared code being fixed :heart:

[GitHub] [hadoop-ozone] elek commented on pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-01 Thread GitBox
elek commented on pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374#issuecomment-684895095 The key line is this: ``` int effectiveBufferSize = Math.min(bufferSize, maxSize - bufferList.size() * bufferSize); ``` If the block/key is smaller than

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

2020-09-01 Thread GitBox
elek commented on a change in pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336#discussion_r481179724 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/ChunkBuffer.java ## @@ -44,9 +45,6 @@ static ChunkBuffer allocate(int

[jira] [Updated] (HDDS-4185) Remove IncrementalByteBuffer from Ozone client

2020-09-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4185: - Labels: pull-request-available (was: ) > Remove IncrementalByteBuffer from Ozone client >

[GitHub] [hadoop-ozone] elek opened a new pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-01 Thread GitBox
elek opened a new pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374 ## What changes were proposed in this pull request? During the teragen test it was identified that the IncrementalByteBuffer is one of the biggest bottlenecks. In the PR of HDDS-4119

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481142517 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Description: 40 datanodes with 400, 000 containers, start recon with xmx:10G. After several hours,

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481136753 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java ## @@ -560,6 +563,13 @@

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481135941 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java ## @@ -560,6 +563,13 @@

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Description: 40 datanodes with 400, 000 containers, start recon with xmx:10G. After several hours,

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Attachment: screenshot-2.png > Fix memory leak of recon > > >

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Attachment: screenshot-1.png > Fix memory leak of recon > > >

[jira] [Updated] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Huang-Mu updated HDDS-4188: - Description: In Chinese document *Home/概念/概览*, There is a broken link. Look attachment

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481134741 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocolServerSideTranslatorPB.java

[jira] [Updated] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Huang-Mu updated HDDS-4188: - Attachment: (was: brokenLink.png) > Fix Chinese document broken link. >

[jira] [Updated] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Huang-Mu updated HDDS-4188: - Description: In Chinese document *Home/概念/概览*, There is a broken link. Look attachment

[jira] [Updated] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Huang-Mu updated HDDS-4188: - Attachment: brokenLink.png > Fix Chinese document broken link. >

[jira] [Updated] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Huang-Mu updated HDDS-4188: - Description: In Chinese document *Home/概念/概览*, There is a broken link. Look attachment

[jira] [Created] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
Zheng Huang-Mu created HDDS-4188: Summary: Fix Chinese document broken link. Key: HDDS-4188 URL: https://issues.apache.org/jira/browse/HDDS-4188 Project: Hadoop Distributed Data Store Issue

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Description: 40 datanodes with 400, 000 containers, start recon with xmx:10G. After several hours,

  1   2   >