[jira] [Updated] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Huang-Mu updated HDDS-4188: - Attachment: (was: brokenLink.png) > Fix Chinese document broken link. >

[GitHub] [hadoop-ozone] elek commented on pull request #1366: HDDS-4167. Acceptance test logs missing if SCM fails to exit safe mode

2020-09-01 Thread GitBox
elek commented on pull request #1366: URL: https://github.com/apache/hadoop-ozone/pull/1366#issuecomment-684899079 > Reduce some code duplication between test-all.sh and ozone-mr/test.sh by extracting functions for the shared code being fixed :heart:

[jira] [Updated] (HDDS-4167) Acceptance test logs missing if SCM fails to exit safe mode

2020-09-01 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDDS-4167: -- Fix Version/s: 1.1.0 Resolution: Fixed Status: Resolved (was: Patch Available) >

[GitHub] [hadoop-ozone] arp7 commented on pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

2020-09-01 Thread GitBox
arp7 commented on pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328#issuecomment-684929618 > What is your opinion about AWS compatibility issues? What exactly is the compatibility issue? If the paths are not interpreted (default behavior), then there is full

[jira] [Comment Edited] (HDDS-4097) S3/Ozone Filesystem inter-op

2020-09-01 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188538#comment-17188538 ] Arpit Agarwal edited comment on HDDS-4097 at 9/1/20, 3:12 PM: -- bq. Using

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Description: 40 datanodes with 400, 000 containers, start recon with xmx:10G. After several hours,

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481136753 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java ## @@ -560,6 +563,13 @@

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481142517 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] elek opened a new pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-01 Thread GitBox
elek opened a new pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374 ## What changes were proposed in this pull request? During the teragen test it was identified that the IncrementalByteBuffer is one of the biggest bottlenecks. In the PR of HDDS-4119

[GitHub] [hadoop-ozone] elek commented on pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-01 Thread GitBox
elek commented on pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374#issuecomment-684895095 The key line is this: ``` int effectiveBufferSize = Math.min(bufferSize, maxSize - bufferList.size() * bufferSize); ``` If the block/key is smaller than

[GitHub] [hadoop-ozone] elek commented on a change in pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

2020-09-01 Thread GitBox
elek commented on a change in pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336#discussion_r481179724 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/common/ChunkBuffer.java ## @@ -44,9 +45,6 @@ static ChunkBuffer allocate(int

[GitHub] [hadoop-ozone] elek commented on pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

2020-09-01 Thread GitBox
elek commented on pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336#issuecomment-684832504 See HDDS-4186 about the incremental chunk buffer. This is an automated message from the Apache Git Service.

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Attachment: screenshot-1.png > Fix memory leak of recon > > >

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481134741 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocolServerSideTranslatorPB.java

[jira] [Updated] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Huang-Mu updated HDDS-4188: - Description: In Chinese document *Home/概念/概览*, There is a broken link. Look attachment

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Attachment: screenshot-2.png > Fix memory leak of recon > > >

[jira] [Updated] (HDDS-4187) Fix memory leak of recon

2020-09-01 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4187: - Description: 40 datanodes with 400, 000 containers, start recon with xmx:10G. After several hours,

[jira] [Updated] (HDDS-4185) Remove IncrementalByteBuffer from Ozone client

2020-09-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4185: - Labels: pull-request-available (was: ) > Remove IncrementalByteBuffer from Ozone client >

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
timmylicheng commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481146665 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestScmAdminHA.java ## @@ -0,0 +1,66 @@ +package

[jira] [Assigned] (HDDS-3102) ozone getconf command should use the GenericCli parent class

2020-09-01 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek reassigned HDDS-3102: - Assignee: (was: Marton Elek) > ozone getconf command should use the GenericCli parent class

[jira] [Created] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
Zheng Huang-Mu created HDDS-4188: Summary: Fix Chinese document broken link. Key: HDDS-4188 URL: https://issues.apache.org/jira/browse/HDDS-4188 Project: Hadoop Distributed Data Store Issue

[jira] [Updated] (HDDS-4188) Fix Chinese document broken link.

2020-09-01 Thread Zheng Huang-Mu (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Huang-Mu updated HDDS-4188: - Description: In Chinese document *Home/概念/概览*, There is a broken link. Look attachment

[GitHub] [hadoop-ozone] elek merged pull request #1366: HDDS-4167. Acceptance test logs missing if SCM fails to exit safe mode

2020-09-01 Thread GitBox
elek merged pull request #1366: URL: https://github.com/apache/hadoop-ozone/pull/1366 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] elek commented on pull request #1363: HDDS-3805. [OFS] Remove usage of OzoneClientAdapter interface

2020-09-01 Thread GitBox
elek commented on pull request #1363: URL: https://github.com/apache/hadoop-ozone/pull/1363#issuecomment-684914342 Thanks for the separation @smengcl, I think it's easier to discuss. I am not sure it's the same patch what I already commented or not. I had a comment in the previous

[GitHub] [hadoop-ozone] elek commented on pull request #1328: HDDS-4102. Normalize Keypath for lookupKey.

2020-09-01 Thread GitBox
elek commented on pull request #1328: URL: https://github.com/apache/hadoop-ozone/pull/1328#issuecomment-684923522 > @elek Any more comments? Looks like we don't have the consensus, yet, in HDDS-4097. It was discussed yesterday during the community meeting, and I felt that everybody

[jira] [Assigned] (HDDS-4179) Implement post-finalize SCM logic to allow nodes of only new version to participate in pipelines.

2020-09-01 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-4179: - Assignee: Prashant Pogde > Implement post-finalize SCM logic to allow nodes of only new

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1366: HDDS-4167. Acceptance test logs missing if SCM fails to exit safe mode

2020-09-01 Thread GitBox
adoroszlai commented on pull request #1366: URL: https://github.com/apache/hadoop-ozone/pull/1366#issuecomment-685067401 Thanks @elek for reviewing and committing it. This is an automated message from the Apache Git Service.

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
adoroszlai commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481380452 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481371486 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481378454 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java ## @@ -560,6 +563,13 @@ public

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1346: [WIP] HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
adoroszlai commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481359383 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-01 Thread GitBox
amaliujia commented on a change in pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#discussion_r481414123 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/admin/scm/GetScmRatisStatusSubcommand.java ## @@ -0,0 +1,26 @@ +package

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1370: HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException

2020-09-01 Thread GitBox
adoroszlai commented on a change in pull request #1370: URL: https://github.com/apache/hadoop-ozone/pull/1370#discussion_r481426506 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java ## @@ -297,9 +298,11 @@

[jira] [Resolved] (HDDS-3938) Flaky TestWatchForCommit#test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-3938. Resolution: Duplicate > Flaky TestWatchForCommit#test2WayCommitForTimeoutException >

[jira] [Resolved] (HDDS-4012) FLAKY-UT: TestWatchForCommit#test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4012. Resolution: Duplicate > FLAKY-UT: TestWatchForCommit#test2WayCommitForTimeoutException >

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1353: HDDS-4148. Add servlet to return SCM DB checkpoint.

2020-09-01 Thread GitBox
xiaoyuyao commented on a change in pull request #1353: URL: https://github.com/apache/hadoop-ozone/pull/1353#discussion_r481416948 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/DBCheckpointServlet.java ## @@ -0,0 +1,197 @@ +/** + * Licensed to

[jira] [Updated] (HDDS-4012) FLAKY-UT: TestWatchForCommit#test2WayCommitForTimeoutException

2020-09-01 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai updated HDDS-4012: --- Labels: (was: pull-request-available) > FLAKY-UT:

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1353: HDDS-4148. Add servlet to return SCM DB checkpoint.

2020-09-01 Thread GitBox
prashantpogde commented on a change in pull request #1353: URL: https://github.com/apache/hadoop-ozone/pull/1353#discussion_r481431139 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/DBCheckpointServlet.java ## @@ -0,0 +1,197 @@ +/** + *

<    1   2