[GitHub] [hadoop-ozone] rakeshadr commented on pull request #1507: HDDS-4307.Start Background Service for Trash Deletion in Ozone Manager

2020-10-22 Thread GitBox
rakeshadr commented on pull request #1507: URL: https://github.com/apache/hadoop-ozone/pull/1507#issuecomment-714260856 Thanks @sadanand48 for the contribution. I've added few comments, please go through it. This is an

[GitHub] [hadoop-ozone] lokeshj1703 opened a new pull request #1513: HDDS-4366. SCM deletion service should delete configured number of blocks every interval.

2020-10-22 Thread GitBox
lokeshj1703 opened a new pull request #1513: URL: https://github.com/apache/hadoop-ozone/pull/1513 ## What changes were proposed in this pull request? SCM service currently uses datanode's configuration to determine the number of blocks to delete every interval. It should have its

[GitHub] [hadoop-ozone] rakeshadr commented on a change in pull request #1507: HDDS-4307.Start Background Service for Trash Deletion in Ozone Manager

2020-10-22 Thread GitBox
rakeshadr commented on a change in pull request #1507: URL: https://github.com/apache/hadoop-ozone/pull/1507#discussion_r509891414 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java ## @@ -197,6 +197,12 @@ public static final String

[jira] [Created] (HDDS-4366) SCM deletion service should delete configured number of blocks every interval

2020-10-22 Thread Lokesh Jain (Jira)
Lokesh Jain created HDDS-4366: - Summary: SCM deletion service should delete configured number of blocks every interval Key: HDDS-4366 URL: https://issues.apache.org/jira/browse/HDDS-4366 Project: Hadoop

[jira] [Updated] (HDDS-4344) Block Deletion Performance Improvements

2020-10-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-4344: -- Attachment: Block Deletion Performance.pdf > Block Deletion Performance Improvements >

[GitHub] [hadoop-ozone] timmylicheng merged pull request #1512: HDDS-4365: SCMBlockLocationFailoverProxyProvider should use ScmBlockLocationProtocolPB.class in RPC.setProtocolEngine

2020-10-22 Thread GitBox
timmylicheng merged pull request #1512: URL: https://github.com/apache/hadoop-ozone/pull/1512 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1512: HDDS-4365: SCMBlockLocationFailoverProxyProvider should use ScmBlockLocationProtocolPB.class in RPC.setProtocolEngine

2020-10-22 Thread GitBox
timmylicheng commented on pull request #1512: URL: https://github.com/apache/hadoop-ozone/pull/1512#issuecomment-714253674 +1. Thanks Glen for working on this. Merging This is an automated message from the Apache Git

[jira] [Updated] (HDDS-4366) SCM deletion service should delete configured number of blocks every interval

2020-10-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4366: - Labels: pull-request-available (was: ) > SCM deletion service should delete configured number of

[jira] [Created] (HDDS-4367) Configuration for deletion service intervals should be different for OM, SCM and datanodes

2020-10-22 Thread Lokesh Jain (Jira)
Lokesh Jain created HDDS-4367: - Summary: Configuration for deletion service intervals should be different for OM, SCM and datanodes Key: HDDS-4367 URL: https://issues.apache.org/jira/browse/HDDS-4367

[GitHub] [hadoop-ozone] timmylicheng opened a new pull request #1514: HDDS-4191 Add failover proxy for SCM container location.

2020-10-22 Thread GitBox
timmylicheng opened a new pull request #1514: URL: https://github.com/apache/hadoop-ozone/pull/1514 ## What changes were proposed in this pull request? Add failover proxy for SCM container client (Please fill in changes proposed in this fix) ## What is the link to the Apache

[jira] [Commented] (HDDS-4365) SCMBlockLocationFailoverProxyProvider should use ScmBlockLocationProtocolPB.class in RPC.setProtocolEngine

2020-10-22 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218793#comment-17218793 ] Li Cheng commented on HDDS-4365: Merged. Resolving this... > SCMBlockLocationFailoverProxyProvider should

[jira] [Resolved] (HDDS-4365) SCMBlockLocationFailoverProxyProvider should use ScmBlockLocationProtocolPB.class in RPC.setProtocolEngine

2020-10-22 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Cheng resolved HDDS-4365. Resolution: Fixed > SCMBlockLocationFailoverProxyProvider should use > ScmBlockLocationProtocolPB.class in

[jira] [Updated] (HDDS-4366) SCM deletion service should delete configured number of blocks every interval

2020-10-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-4366: -- Status: Patch Available (was: Open) > SCM deletion service should delete configured number of blocks

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-22 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r510017737 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2205,6 +2276,318 @@ private void

[jira] [Created] (HDDS-4371) Datanode should have deletion service thread per volume

2020-10-22 Thread Lokesh Jain (Jira)
Lokesh Jain created HDDS-4371: - Summary: Datanode should have deletion service thread per volume Key: HDDS-4371 URL: https://issues.apache.org/jira/browse/HDDS-4371 Project: Hadoop Distributed Data Store

[jira] [Created] (HDDS-4370) Datanode deletion service can avoid storing deleted blocks

2020-10-22 Thread Lokesh Jain (Jira)
Lokesh Jain created HDDS-4370: - Summary: Datanode deletion service can avoid storing deleted blocks Key: HDDS-4370 URL: https://issues.apache.org/jira/browse/HDDS-4370 Project: Hadoop Distributed Data

[jira] [Created] (HDDS-4373) Add append design documentation

2020-10-22 Thread mingchao zhao (Jira)
mingchao zhao created HDDS-4373: --- Summary: Add append design documentation Key: HDDS-4373 URL: https://issues.apache.org/jira/browse/HDDS-4373 Project: Hadoop Distributed Data Store Issue

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1498: HDDS-4339. Allow AWSSignatureProcessor init when aws signature is absent.

2020-10-22 Thread GitBox
timmylicheng commented on a change in pull request #1498: URL: https://github.com/apache/hadoop-ozone/pull/1498#discussion_r510107592 ## File path: hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/OzoneClientProducer.java ## @@ -116,6 +116,13 @@ private

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1503: HDDS-4332: ListFileStatus - do lookup in directory and file tables

2020-10-22 Thread GitBox
linyiqun commented on a change in pull request #1503: URL: https://github.com/apache/hadoop-ozone/pull/1503#discussion_r510030316 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java ## @@ -2205,6 +2272,167 @@ private void

[jira] [Created] (HDDS-4368) SCM should avoid sending delete transactions for under-replicated containers

2020-10-22 Thread Lokesh Jain (Jira)
Lokesh Jain created HDDS-4368: - Summary: SCM should avoid sending delete transactions for under-replicated containers Key: HDDS-4368 URL: https://issues.apache.org/jira/browse/HDDS-4368 Project: Hadoop

[jira] [Updated] (HDDS-4370) Datanode deletion service can avoid storing deleted blocks

2020-10-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-4370: -- Component/s: Ozone Datanode > Datanode deletion service can avoid storing deleted blocks >

[jira] [Updated] (HDDS-4369) Datanode should store the delete transaction as is in rocksDB

2020-10-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-4369: -- Component/s: Ozone Datanode > Datanode should store the delete transaction as is in rocksDB >

[jira] [Created] (HDDS-4372) Datanode delete operations should not impact rocksDB cache

2020-10-22 Thread Lokesh Jain (Jira)
Lokesh Jain created HDDS-4372: - Summary: Datanode delete operations should not impact rocksDB cache Key: HDDS-4372 URL: https://issues.apache.org/jira/browse/HDDS-4372 Project: Hadoop Distributed Data

[jira] [Updated] (HDDS-4371) Datanode should have deletion service thread per volume

2020-10-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-4371: -- Component/s: Ozone Datanode > Datanode should have deletion service thread per volume >

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #1395: HDDS-4088. Adding Owner info for Authorizer plugin to honor owner access rights

2020-10-22 Thread GitBox
smengcl commented on a change in pull request #1395: URL: https://github.com/apache/hadoop-ozone/pull/1395#discussion_r510108891 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -1642,25 +1644,42 @@ private boolean

[GitHub] [hadoop-ozone] captainzmc opened a new pull request #1515: HDDS-4373. [Design] Ozone support append operation

2020-10-22 Thread GitBox
captainzmc opened a new pull request #1515: URL: https://github.com/apache/hadoop-ozone/pull/1515 ## What changes were proposed in this pull request? This is a design doc, which is moved from google doc to make it easier to track the progress. ## What is the link to the

[jira] [Created] (HDDS-4369) Datanode should store the delete transaction as is in rocksDB

2020-10-22 Thread Lokesh Jain (Jira)
Lokesh Jain created HDDS-4369: - Summary: Datanode should store the delete transaction as is in rocksDB Key: HDDS-4369 URL: https://issues.apache.org/jira/browse/HDDS-4369 Project: Hadoop Distributed Data

[GitHub] [hadoop-ozone] timmylicheng closed pull request #1514: HDDS-4191 Add failover proxy for SCM container location.

2020-10-22 Thread GitBox
timmylicheng closed pull request #1514: URL: https://github.com/apache/hadoop-ozone/pull/1514 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Updated] (HDDS-4373) Add append design documentation

2020-10-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4373: - Labels: pull-request-available (was: ) > Add append design documentation >

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4385) It would be nice if there is a search function using container ID on Recon Missing Container page

2020-10-22 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen updated HDDS-4385: - Summary: It would be nice if there is a search function using container ID on Recon Missing Container

[jira] [Resolved] (HDDS-4384) Container sequence ID is not loaded after SCM restart

2020-10-22 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen resolved HDDS-4384. -- Resolution: Duplicate > Container sequence ID is not loaded after SCM restart >

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Affects Version/s: (was: 1.0.0) 1.1.0 > Each EndpointStateMachine uses its own

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Commented] (HDDS-4339) Ozone S3 gateway throws NPE with goofys

2020-10-22 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219456#comment-17219456 ] Li Cheng commented on HDDS-4339: I created https://issues.apache.org/jira/browse/HDDS-4361 to track s3g

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: In Tencent production environment, after start Recon for a while, we got warnings that all

[GitHub] [hadoop-ozone] GlenGeng closed pull request #1510: [Draft]HDDS-4191: failover proxy for container location protocol

2020-10-22 Thread GitBox
GlenGeng closed pull request #1510: URL: https://github.com/apache/hadoop-ozone/pull/1510 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Updated] (HDDS-4307) Start Background Service for Trash Deletion in Ozone Manager

2020-10-22 Thread Rakesh Radhakrishnan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh Radhakrishnan updated HDDS-4307: --- Status: Patch Available (was: Open) > Start Background Service for Trash Deletion in

[jira] [Created] (HDDS-4384) Container sequence ID is not loaded after SCM restart

2020-10-22 Thread Sammi Chen (Jira)
Sammi Chen created HDDS-4384: Summary: Container sequence ID is not loaded after SCM restart Key: HDDS-4384 URL: https://issues.apache.org/jira/browse/HDDS-4384 Project: Hadoop Distributed Data Store

[jira] [Assigned] (HDDS-4384) Container sequence ID is not loaded after SCM restart

2020-10-22 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen reassigned HDDS-4384: Assignee: (was: Sammi Chen) > Container sequence ID is not loaded after SCM restart >

[jira] [Updated] (HDDS-4384) Container sequence ID is not loaded after SCM restart

2020-10-22 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen updated HDDS-4384: - Description: {noformat} { "state" : "CLOSED", "replicationFactor" : "THREE", "replicationType" :

[jira] [Updated] (HDDS-4386) EndpointStateMachine

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Summary: EndpointStateMachine (was: CLONE - Ozone S3 gateway throws NPE with goofys) >

[jira] [Created] (HDDS-4386) CLONE - Ozone S3 gateway throws NPE with goofys

2020-10-22 Thread Glen Geng (Jira)
Glen Geng created HDDS-4386: --- Summary: CLONE - Ozone S3 gateway throws NPE with goofys Key: HDDS-4386 URL: https://issues.apache.org/jira/browse/HDDS-4386 Project: Hadoop Distributed Data Store

[jira] [Updated] (HDDS-4385) It would be nice if there is a search function using container ID

2020-10-22 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen updated HDDS-4385: - Description: In production cluster, there can be many missing containers to investigate. It would be

[jira] [Assigned] (HDDS-4386) CLONE - Ozone S3 gateway throws NPE with goofys

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng reassigned HDDS-4386: --- Assignee: Glen Geng (was: Li Cheng) > CLONE - Ozone S3 gateway throws NPE with goofys >

[jira] [Created] (HDDS-4385) It would be nice if there is a search function using container ID

2020-10-22 Thread Sammi Chen (Jira)
Sammi Chen created HDDS-4385: Summary: It would be nice if there is a search function using container ID Key: HDDS-4385 URL: https://issues.apache.org/jira/browse/HDDS-4385 Project: Hadoop Distributed

[GitHub] [hadoop-ozone] captainzmc commented on a change in pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
captainzmc commented on a change in pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516#discussion_r510568534 ## File path: hadoop-hdds/docs/content/feature/Quota.md ## @@ -0,0 +1,67 @@ +--- +title: "Quota in Ozone" +date: "2020-October-22" +weight: 4

[GitHub] [hadoop-ozone] captainzmc commented on a change in pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
captainzmc commented on a change in pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516#discussion_r510568737 ## File path: hadoop-hdds/docs/content/feature/Quota.md ## @@ -0,0 +1,67 @@ +--- +title: "Quota in Ozone" +date: "2020-October-22" +weight: 4

[GitHub] [hadoop-ozone] captainzmc commented on a change in pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
captainzmc commented on a change in pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516#discussion_r510568534 ## File path: hadoop-hdds/docs/content/feature/Quota.md ## @@ -0,0 +1,67 @@ +--- +title: "Quota in Ozone" +date: "2020-October-22" +weight: 4

[GitHub] [hadoop-ozone] captainzmc commented on pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
captainzmc commented on pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516#issuecomment-714884328 Thanks @xiaoyuyao 's review. I have submitted a new commit fix the above issues. This is an automated

[jira] [Updated] (HDDS-4384) Container sequence ID is not loaded after SCM restart

2020-10-22 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen updated HDDS-4384: - Labels: newbie (was: ) > Container sequence ID is not loaded after SCM restart >

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1514: HDDS-4191 Add failover proxy for SCM container location.

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1514: URL: https://github.com/apache/hadoop-ozone/pull/1514#discussion_r510581749 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMContainerLocationFailoverProxyProvider.java ## @@ -0,0

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Description: (was: Configured goofys and s3g on different hosts and Fiotest writes files on the goofys

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Summary: Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon (was:

[jira] [Updated] (HDDS-4386) Each EndpointStateMachine uses its own thread pool to talk with SCM/Recon

2020-10-22 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4386: Labels: (was: pull-request-available) > Each EndpointStateMachine uses its own thread pool to talk with

[jira] [Updated] (HDDS-3731) add storage space quota doc

2020-10-22 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mingchao zhao updated HDDS-3731: Summary: add storage space quota doc (was: Relative Unit Test) > add storage space quota doc >

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1480: HDDS-4315. Use Epoch to generate unique ObjectIDs

2020-10-22 Thread GitBox
linyiqun commented on a change in pull request #1480: URL: https://github.com/apache/hadoop-ozone/pull/1480#discussion_r510272665 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -394,6 +403,8 @@ private

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1480: HDDS-4315. Use Epoch to generate unique ObjectIDs

2020-10-22 Thread GitBox
linyiqun commented on a change in pull request #1480: URL: https://github.com/apache/hadoop-ozone/pull/1480#discussion_r510272665 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -394,6 +403,8 @@ private

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1486: HDDS-4296. SCM changes to process Layout Info in heartbeat request/response

2020-10-22 Thread GitBox
avijayanhwx commented on a change in pull request #1486: URL: https://github.com/apache/hadoop-ozone/pull/1486#discussion_r510187543 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java ## @@ -400,6 +405,66 @@ public void

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1480: HDDS-4315. Use Epoch to generate unique ObjectIDs

2020-10-22 Thread GitBox
linyiqun commented on a change in pull request #1480: URL: https://github.com/apache/hadoop-ozone/pull/1480#discussion_r510268648 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java ## @@ -524,6 +535,48 @@ public static OmKeyInfo

[GitHub] [hadoop-ozone] codecov-io commented on pull request #1514: HDDS-4191 Add failover proxy for SCM container location.

2020-10-22 Thread GitBox
codecov-io commented on pull request #1514: URL: https://github.com/apache/hadoop-ozone/pull/1514#issuecomment-714483581 # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1514?src=pr=h1) Report > Merging [#1514](https://codecov.io/gh/apache/hadoop-ozone/pull/1514?src=pr=desc)

[GitHub] [hadoop-ozone] captainzmc opened a new pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
captainzmc opened a new pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516 ## What changes were proposed in this pull request? The space Quota feature is now nearly complete, so add a description of how to use it ## What is the link to the Apache JIRA

[jira] [Updated] (HDDS-3731) add storage space quota doc

2020-10-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3731: - Labels: pull-request-available (was: ) > add storage space quota doc >

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #1395: HDDS-4088. Adding Owner info for Authorizer plugin to honor owner access rights

2020-10-22 Thread GitBox
smengcl commented on a change in pull request #1395: URL: https://github.com/apache/hadoop-ozone/pull/1395#discussion_r510375521 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -1642,25 +1644,42 @@ private boolean

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516#discussion_r510361614 ## File path: hadoop-hdds/docs/content/feature/Quota.md ## @@ -0,0 +1,67 @@ +--- +title: "Quota in Ozone" +date: "2020-October-22" +weight: 4

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516#discussion_r510360994 ## File path: hadoop-hdds/docs/content/feature/Quota.md ## @@ -0,0 +1,67 @@ +--- +title: "Quota in Ozone" +date: "2020-October-22" +weight: 4

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1395: HDDS-4088. Adding Owner info for Authorizer plugin to honor owner access rights

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1395: URL: https://github.com/apache/hadoop-ozone/pull/1395#discussion_r510293188 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -1642,25 +1644,42 @@ private boolean

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516#discussion_r510364763 ## File path: hadoop-hdds/docs/content/feature/Quota.md ## @@ -0,0 +1,67 @@ +--- +title: "Quota in Ozone" +date: "2020-October-22" +weight: 4

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1516: HDDS-3731. [doc]add storage space quota document.

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1516: URL: https://github.com/apache/hadoop-ozone/pull/1516#discussion_r510363980 ## File path: hadoop-hdds/docs/content/feature/Quota.md ## @@ -0,0 +1,67 @@ +--- +title: "Quota in Ozone" +date: "2020-October-22" +weight: 4

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #1395: HDDS-4088. Adding Owner info for Authorizer plugin to honor owner access rights

2020-10-22 Thread GitBox
smengcl commented on a change in pull request #1395: URL: https://github.com/apache/hadoop-ozone/pull/1395#discussion_r510376263 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -1642,25 +1644,42 @@ private boolean

[jira] [Created] (HDDS-4374) Share RocskDB cache in new datanode DB utilities

2020-10-22 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-4374: Summary: Share RocskDB cache in new datanode DB utilities Key: HDDS-4374 URL: https://issues.apache.org/jira/browse/HDDS-4374 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1395: HDDS-4088. Adding Owner info for Authorizer plugin to honor owner access rights

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1395: URL: https://github.com/apache/hadoop-ozone/pull/1395#discussion_r510410249 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -1642,25 +1644,42 @@ private boolean

[jira] [Assigned] (HDDS-4337) Implement RocksDB options cache for new datanode DB utilities

2020-10-22 Thread Ethan Rose (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Rose reassigned HDDS-4337: Assignee: Ethan Rose > Implement RocksDB options cache for new datanode DB utilities >

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1301: HDDS-3882. Update modification time when updating volume/bucket/key ACLs

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1301: URL: https://github.com/apache/hadoop-ozone/pull/1301#discussion_r510486443 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyAddAclRequest.java ## @@ -43,6 +45,19 @@

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1301: HDDS-3882. Update modification time when updating volume/bucket/key ACLs

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1301: URL: https://github.com/apache/hadoop-ozone/pull/1301#discussion_r510486990 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/acl/OMVolumeAddAclRequest.java ## @@ -48,6

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1301: HDDS-3882. Update modification time when updating volume/bucket/key ACLs

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1301: URL: https://github.com/apache/hadoop-ozone/pull/1301#discussion_r510486544 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeyRemoveAclRequest.java ## @@ -43,6 +45,19

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1301: HDDS-3882. Update modification time when updating volume/bucket/key ACLs

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1301: URL: https://github.com/apache/hadoop-ozone/pull/1301#discussion_r510486787 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/acl/OMKeySetAclRequest.java ## @@ -44,6 +46,19 @@

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1514: HDDS-4191 Add failover proxy for SCM container location.

2020-10-22 Thread GitBox
xiaoyuyao commented on a change in pull request #1514: URL: https://github.com/apache/hadoop-ozone/pull/1514#discussion_r510497229 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMContainerLocationFailoverProxyProvider.java ## @@ -0,0

[jira] [Created] (HDDS-4375) Client asks OM to change the block length when truncate

2020-10-22 Thread runzhiwang (Jira)
runzhiwang created HDDS-4375: Summary: Client asks OM to change the block length when truncate Key: HDDS-4375 URL: https://issues.apache.org/jira/browse/HDDS-4375 Project: Hadoop Distributed Data Store

[jira] [Updated] (HDDS-4375) OM changes the block length when receive truncate request

2020-10-22 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4375: - Summary: OM changes the block length when receive truncate request (was: Client asks OM to change the

[jira] [Created] (HDDS-4379) Mark truncated blocks in OM

2020-10-22 Thread runzhiwang (Jira)
runzhiwang created HDDS-4379: Summary: Mark truncated blocks in OM Key: HDDS-4379 URL: https://issues.apache.org/jira/browse/HDDS-4379 Project: Hadoop Distributed Data Store Issue Type: Sub-task

[jira] [Created] (HDDS-4378) Ozone shell support truncate API

2020-10-22 Thread runzhiwang (Jira)
runzhiwang created HDDS-4378: Summary: Ozone shell support truncate API Key: HDDS-4378 URL: https://issues.apache.org/jira/browse/HDDS-4378 Project: Hadoop Distributed Data Store Issue Type:

[jira] [Updated] (HDDS-4375) OM changes the block length when receives truncate request

2020-10-22 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4375: - Summary: OM changes the block length when receives truncate request (was: OM changes the block length

[jira] [Created] (HDDS-4382) SCM send truncate block to datanode

2020-10-22 Thread runzhiwang (Jira)
runzhiwang created HDDS-4382: Summary: SCM send truncate block to datanode Key: HDDS-4382 URL: https://issues.apache.org/jira/browse/HDDS-4382 Project: Hadoop Distributed Data Store Issue Type:

[jira] [Updated] (HDDS-4380) OM stores truncate key in truncateTable

2020-10-22 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4380: - Description: OM stores in truncateTable. The TruncateKey in RepeatedTruncateOmKeyInfo represents one

[jira] [Updated] (HDDS-4376) SCM create transaction for truncated blocks

2020-10-22 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4376: - Description: When scm receives TruncateScmKeyRequestProto, for the deleteBlocks, scm has already

[jira] [Created] (HDDS-4380) OM stores truncate key in truncateTable

2020-10-22 Thread runzhiwang (Jira)
runzhiwang created HDDS-4380: Summary: OM stores truncate key in truncateTable Key: HDDS-4380 URL: https://issues.apache.org/jira/browse/HDDS-4380 Project: Hadoop Distributed Data Store Issue

[jira] [Created] (HDDS-4381) OM send truncate key to SCM

2020-10-22 Thread runzhiwang (Jira)
runzhiwang created HDDS-4381: Summary: OM send truncate key to SCM Key: HDDS-4381 URL: https://issues.apache.org/jira/browse/HDDS-4381 Project: Hadoop Distributed Data Store Issue Type: Sub-task

[jira] [Updated] (HDDS-4381) OM send truncate key to SCM

2020-10-22 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4381: - Description: OM starts a background thread to select a certain number of entries from truncateTable, and

[jira] [Updated] (HDDS-4379) OM marks truncated blocks

2020-10-22 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4379: - Description: Append mean ozone client can append some content to the tail of some key. So if Client1

[jira] [Updated] (HDDS-4382) SCM send truncate block to datanode

2020-10-22 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-4382: - Description: SCM starts a background thread to send TruncateBlocksCommandProto to the datanode. > SCM

  1   2   >