[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1405: HDDS-4143. Implement a factory for OM Requests that returns an instance based on layout version.

2020-09-08 Thread GitBox
bharatviswa504 commented on a change in pull request #1405: URL: https://github.com/apache/hadoop-ozone/pull/1405#discussion_r485345067 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketSetPropertyRequest.java ## @@

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485344203 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSWithObjectStoreCreate.java ## @@ -209,6

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485339793 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSWithObjectStoreCreate.java ## @@ -209,6

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-08 Thread GitBox
timmylicheng commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r485337010 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMBlockLocationFailoverProxyProvider.java ## @@ -0,0

[GitHub] [hadoop-ozone] arp7 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
arp7 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485332430 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSWithObjectStoreCreate.java ## @@ -209,6 +215,94 @@

[jira] [Updated] (HDDS-4115) CLI command to show current SCM leader and follower status

2020-09-08 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated HDDS-4115: --- Status: Patch Available (was: In Progress) > CLI command to show current SCM leader and follower status >

[jira] [Updated] (HDDS-3684) Add tests for replication annotation

2020-09-08 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated HDDS-3684: --- Status: Patch Available (was: In Progress) > Add tests for replication annotation >

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1314: HDDS-3988: DN can distinguish SCMCommand from stale leader SCM

2020-09-08 Thread GitBox
amaliujia commented on pull request #1314: URL: https://github.com/apache/hadoop-ozone/pull/1314#issuecomment-689290907 cc @amaliujia This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Updated] (HDDS-3988) DN can distinguish SCMCommand from stale leader SCM

2020-09-08 Thread Nanda kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-3988: -- Status: Patch Available (was: Open) > DN can distinguish SCMCommand from stale leader SCM >

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
bharatviswa504 commented on pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#issuecomment-689278088 Thank You @arp7 for the review. I have addressed your review comments. This is an automated

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485315969 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java ## @@ -158,6 +159,23

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r485313969 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSLayoutVersionManager.java ## @@ -0,0 +1,92 @@ +/** + *

[jira] [Created] (HDDS-4221) Support extra large storage capacity server as datanode

2020-09-08 Thread Sammi Chen (Jira)
Sammi Chen created HDDS-4221: Summary: Support extra large storage capacity server as datanode Key: HDDS-4221 URL: https://issues.apache.org/jira/browse/HDDS-4221 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485293304 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java ## @@ -158,6 +159,23

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485293304 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java ## @@ -158,6 +159,23

[GitHub] [hadoop-ozone] cxorm commented on a change in pull request #1296: HDDS-4053. Volume space: add quotaUsageInBytes and update it when write and delete key.

2020-09-08 Thread GitBox
cxorm commented on a change in pull request #1296: URL: https://github.com/apache/hadoop-ozone/pull/1296#discussion_r484607389 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java ## @@ -269,6 +269,7 @@ private OzoneConsts() { public

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1387: HDDS-4198. Compile Ozone with multiple Java versions

2020-09-08 Thread GitBox
adoroszlai commented on pull request #1387: URL: https://github.com/apache/hadoop-ozone/pull/1387#issuecomment-689208686 Thanks @vivekratnavel for the review, and @elek for reviewing and committing it. This is an automated

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1388: HDDS-4204. upgrade docker environment does not work with KEEP_RUNNING=true

2020-09-08 Thread GitBox
adoroszlai commented on pull request #1388: URL: https://github.com/apache/hadoop-ozone/pull/1388#issuecomment-689206320 Thanks @amaliujia for the review, and @elek for reviewing and committing it. This is an automated

[GitHub] [hadoop-ozone] adoroszlai commented on pull request #1033: HDDS-3667. If we gracefully stop datanode it would be better to notify scm and r…

2020-09-08 Thread GitBox
adoroszlai commented on pull request #1033: URL: https://github.com/apache/hadoop-ozone/pull/1033#issuecomment-689204311 /pending @nandakumar131: "State change is not properly handled" This is an automated message from the

[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request #1398: HDDS-4210. ResolveBucket during checkAcls fails.

2020-09-08 Thread GitBox
adoroszlai commented on a change in pull request #1398: URL: https://github.com/apache/hadoop-ozone/pull/1398#discussion_r483929995 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerHAWithData.java ## @@ -69,6 +70,38 @@

[jira] [Resolved] (HDDS-4110) Intermittent failure in TestEndpoints#testGetDatanodes

2020-09-08 Thread Attila Doroszlai (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Doroszlai resolved HDDS-4110. Assignee: Vivek Ratnavel Subramanian Resolution: Duplicate > Intermittent failure in

[GitHub] [hadoop-ozone] errose28 commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-08 Thread GitBox
errose28 commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r485223401 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java ## @@ -487,6 +486,7 @@

[GitHub] [hadoop-ozone] errose28 commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-08 Thread GitBox
errose28 commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r485204689 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeTable.java ## @@ -0,0 +1,140 @@ +/* +

[GitHub] [hadoop-ozone] prashantpogde commented on pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#issuecomment-689143317 yes, I will remove NOT_SUPPORTED_OPERATION. This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r485203276 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/NewSCMFeatureUpgradeAction.java ## @@ -0,0 +1,34

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r485202562 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/upgrade/HDDSLayoutVersionManager.java ## @@ -0,0 +1,92 @@ +/** + *

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r485202252 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/exceptions/SCMException.java ## @@ -123,6 +123,7 @@ public

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1346: HDDS-4115. CLI command to show current SCM leader and follower status.

2020-09-08 Thread GitBox
amaliujia commented on pull request #1346: URL: https://github.com/apache/hadoop-ozone/pull/1346#issuecomment-689140982 Also R: @nandakumar131 can you please take a look? This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] errose28 commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-08 Thread GitBox
errose28 commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r485198575 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/helpers/KeyValueContainerUtil.java ## @@

[GitHub] [hadoop-ozone] errose28 commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-08 Thread GitBox
errose28 commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r485198138 ## File path: hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestSchemaOneBackwardsCompatibility.java ## @@

[GitHub] [hadoop-ozone] errose28 commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-08 Thread GitBox
errose28 commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r482287093 ## File path: hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueBlockIterator.java ## @@ -88,196

[GitHub] [hadoop-ozone] errose28 commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-08 Thread GitBox
errose28 commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r485195881 ## File path: hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestContainerCache.java ## @@ -85,17 +86,17 @@

[jira] [Assigned] (HDDS-3102) ozone getconf command should use the GenericCli parent class

2020-09-08 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned HDDS-3102: -- Assignee: Rui Wang > ozone getconf command should use the GenericCli parent class >

[jira] [Updated] (HDDS-3102) ozone getconf command should use the GenericCli parent class

2020-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3102: - Labels: newbie pull-request-available (was: newbie) > ozone getconf command should use the

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

2020-09-08 Thread GitBox
amaliujia commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-689134002 @elek Any suggestion on how to test this change? This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] amaliujia opened a new pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

2020-09-08 Thread GitBox
amaliujia opened a new pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410 ## What changes were proposed in this pull request? org.apache.hadoop.ozone.freon.OzoneGetCOnf implements a tool to print out current configuration values With all the other CLI

[GitHub] [hadoop-ozone] errose28 commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-08 Thread GitBox
errose28 commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r485192234 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/SchemaOneDeletedBlocksTable.java ## @@ -0,0

[GitHub] [hadoop-ozone] errose28 commented on a change in pull request #1298: HDDS-3869. Use different column families for datanode block and metadata

2020-09-08 Thread GitBox
errose28 commented on a change in pull request #1298: URL: https://github.com/apache/hadoop-ozone/pull/1298#discussion_r485189562 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/BlockManagerImpl.java ## @@ -324,14

[jira] [Created] (HDDS-4220) BlockManagerImpl#getBlockByID does unnecessary serialization

2020-09-08 Thread Ethan Rose (Jira)
Ethan Rose created HDDS-4220: Summary: BlockManagerImpl#getBlockByID does unnecessary serialization Key: HDDS-4220 URL: https://issues.apache.org/jira/browse/HDDS-4220 Project: Hadoop Distributed Data

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1399: HDDS-3684. Add tests for replication annotation

2020-09-08 Thread GitBox
amaliujia commented on pull request #1399: URL: https://github.com/apache/hadoop-ozone/pull/1399#issuecomment-689110760 @nandakumar131 thanks! I moved the existing MockRatisServer out and re-use that one. This is an

[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1399: HDDS-3684. Add tests for replication annotation

2020-09-08 Thread GitBox
nandakumar131 commented on pull request #1399: URL: https://github.com/apache/hadoop-ozone/pull/1399#issuecomment-689086961 @amaliujia FYI: We already have a `MockRatisServer` which is inside `MockSCMHAManager.java`, please extract it out if required and reuse the same.

[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1033: HDDS-3667. If we gracefully stop datanode it would be better to notify scm and r…

2020-09-08 Thread GitBox
nandakumar131 commented on pull request #1033: URL: https://github.com/apache/hadoop-ozone/pull/1033#issuecomment-689082204 /pending What advantage do we get by sending a stop signal from the datanode to SCM? This will require additional logic in Node Management code of SCM. If there

[jira] [Commented] (HDDS-4155) Directory and filename can end up with same name in a path

2020-09-08 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192408#comment-17192408 ] Arpit Agarwal commented on HDDS-4155: - {quote}Sorry, I am not sure what does this answer mean exactly.

[jira] [Commented] (HDDS-4155) Directory and filename can end up with same name in a path

2020-09-08 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192405#comment-17192405 ] Marton Elek commented on HDDS-4155: --- > s3a does not handle it. It just displays both if they exist.

[jira] [Commented] (HDDS-3667) If we gracefully stop datanode it would be better to notify scm and recon

2020-09-08 Thread Nanda kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192404#comment-17192404 ] Nanda kumar commented on HDDS-3667: --- [~maobaolong], what advantage do we get by sending a stop signal

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1399: HDDS-3684. Add tests for replication annotation

2020-09-08 Thread GitBox
amaliujia commented on pull request #1399: URL: https://github.com/apache/hadoop-ozone/pull/1399#issuecomment-689076858 Also R: @nandakumar131 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1400: HDDS-4141. Implement Finalize command in Ozone Manager client.

2020-09-08 Thread GitBox
avijayanhwx commented on a change in pull request #1400: URL: https://github.com/apache/hadoop-ozone/pull/1400#discussion_r485130579 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/upgrade/OMFinalizeUpgradeProgressResponse.java ## @@

[jira] [Commented] (HDDS-4170) Fix typo in method description.

2020-09-08 Thread Harinder Singh Bedi (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192400#comment-17192400 ] Harinder Singh Bedi commented on HDDS-4170: --- I have fixed this under [GitHub Pull Request

[GitHub] [hadoop-ozone] nandakumar131 commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

2020-09-08 Thread GitBox
nandakumar131 commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r485108126 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMBlockLocationFailoverProxyProvider.java ## @@ -0,0

[jira] [Resolved] (HDDS-4150) recon.api.TestEndpoints is flaky

2020-09-08 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Ratnavel Subramanian resolved HDDS-4150. -- Resolution: Fixed > recon.api.TestEndpoints is flaky >

[GitHub] [hadoop-ozone] avijayanhwx merged pull request #1396: HDDS-4150. recon.api.TestEndpoints test is flaky

2020-09-08 Thread GitBox
avijayanhwx merged pull request #1396: URL: https://github.com/apache/hadoop-ozone/pull/1396 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1396: HDDS-4150. recon.api.TestEndpoints test is flaky

2020-09-08 Thread GitBox
avijayanhwx commented on pull request #1396: URL: https://github.com/apache/hadoop-ozone/pull/1396#issuecomment-689063141 Thanks for fixing it @vivekratnavel. This is an automated message from the Apache Git Service. To

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1390: HDDS-4196. Add an endpoint in Recon to query Prometheus

2020-09-08 Thread GitBox
avijayanhwx commented on a change in pull request #1390: URL: https://github.com/apache/hadoop-ozone/pull/1390#discussion_r485120523 ## File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/MetricsProxyEndpoint.java ## @@ -0,0 +1,117 @@ +/* + *

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
avijayanhwx commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r485115397 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/exceptions/SCMException.java ## @@ -123,6 +123,7 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1398: HDDS-4210. ResolveBucket during checkAcls fails.

2020-09-08 Thread GitBox
xiaoyuyao commented on a change in pull request #1398: URL: https://github.com/apache/hadoop-ozone/pull/1398#discussion_r485100541 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -3523,6 +3533,47 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao merged pull request #1406: HDDS-4170. Fix typo in method description.

2020-09-08 Thread GitBox
xiaoyuyao merged pull request #1406: URL: https://github.com/apache/hadoop-ozone/pull/1406 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] xiaoyuyao commented on pull request #1290: HDDS-4064. Show container verbose info with verbose option

2020-09-08 Thread GitBox
xiaoyuyao commented on pull request #1290: URL: https://github.com/apache/hadoop-ozone/pull/1290#issuecomment-689013330 LGTM, +1. I will merge it shortly. This is an automated message from the Apache Git Service. To respond

[GitHub] [hadoop-ozone] xiaoyuyao merged pull request #1290: HDDS-4064. Show container verbose info with verbose option

2020-09-08 Thread GitBox
xiaoyuyao merged pull request #1290: URL: https://github.com/apache/hadoop-ozone/pull/1290 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1296: HDDS-4053. Volume space: add quotaUsageInBytes and update it when write and delete key.

2020-09-08 Thread GitBox
xiaoyuyao commented on a change in pull request #1296: URL: https://github.com/apache/hadoop-ozone/pull/1296#discussion_r485051300 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneVolume.java ## @@ -131,6 +133,18 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1397: HDDS-4211. [OFS] Better owner and group display for listing Ozone volumes and buckets

2020-09-08 Thread GitBox
xiaoyuyao commented on a change in pull request #1397: URL: https://github.com/apache/hadoop-ozone/pull/1397#discussion_r485046310 ## File path: hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java ## @@ -972,24 +992,26

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1404: HDDS-2949: mkdir : store directory entries in a separate table

2020-09-08 Thread GitBox
linyiqun commented on a change in pull request #1404: URL: https://github.com/apache/hadoop-ozone/pull/1404#discussion_r485035228 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneDirectory.java ## @@ -0,0 +1,198 @@ +/* + * Licensed

[jira] [Comment Edited] (HDDS-4209) S3A Filesystem does not work with Ozone S3

2020-09-08 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192296#comment-17192296 ] Bharat Viswanadham edited comment on HDDS-4209 at 9/8/20, 4:09 PM: ---

[jira] [Commented] (HDDS-4209) S3A Filesystem does not work with Ozone S3

2020-09-08 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192296#comment-17192296 ] Bharat Viswanadham commented on HDDS-4209: -- [~elek] I am not sure if this is the corner case.

[jira] [Comment Edited] (HDDS-4209) S3A Filesystem does not work with Ozone S3

2020-09-08 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192296#comment-17192296 ] Bharat Viswanadham edited comment on HDDS-4209 at 9/8/20, 4:08 PM: ---

[GitHub] [hadoop-ozone] sodonnel commented on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

2020-09-08 Thread GitBox
sodonnel commented on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-688978976 Thinking some more about what to do with containers in the DELETED state in SCM. If we are not confident to delete them immediately and automatically, maybe we should

[GitHub] [hadoop-ozone] arp7 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
arp7 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485021796 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java ## @@ -158,6 +159,23 @@ public

[GitHub] [hadoop-ozone] arp7 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
arp7 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485022218 ## File path: hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java ## @@ -197,6 +198,18 @@ public Response

[GitHub] [hadoop-ozone] arp7 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

2020-09-08 Thread GitBox
arp7 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r485020859 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java ## @@ -158,6 +159,23 @@ public

[jira] [Updated] (HDDS-4218) Remove test TestRatisManager

2020-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4218: - Labels: pull-request-available (was: ) > Remove test TestRatisManager >

[GitHub] [hadoop-ozone] sadanand48 opened a new pull request #1409: HDDS-4218. Remove test TestRatisManager

2020-09-08 Thread GitBox
sadanand48 opened a new pull request #1409: URL: https://github.com/apache/hadoop-ozone/pull/1409 ## What changes were proposed in this pull request? Delete test TestRatisManager as RatisManager class itself has been deleted and this test has been disabled for long. ## What is

[jira] [Updated] (HDDS-4143) Implement a factory for OM Requests that returns an instance based on layout version.

2020-09-08 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated HDDS-4143: Status: Patch Available (was: In Progress) > Implement a factory for OM Requests that

[jira] [Updated] (HDDS-3881) Add current layout version to OM Ratis Request

2020-09-08 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated HDDS-3881: Status: Patch Available (was: In Progress) > Add current layout version to OM Ratis

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1405: HDDS-4143. Implement a factory for OM Requests that returns an instance based on layout version.

2020-09-08 Thread GitBox
avijayanhwx commented on pull request #1405: URL: https://github.com/apache/hadoop-ozone/pull/1405#issuecomment-688948787 cc @fapifta / @prashantpogde. This is an automated message from the Apache Git Service. To respond to

[jira] [Updated] (HDDS-4219) Revisit 'static' nature of OM Layout Version Manager.

2020-09-08 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated HDDS-4219: Parent: HDDS-3698 Issue Type: Sub-task (was: Bug) > Revisit 'static' nature of OM

[jira] [Updated] (HDDS-4217) Remove test TestOzoneContainerRatis

2020-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4217: - Labels: pull-request-available (was: ) > Remove test TestOzoneContainerRatis >

[GitHub] [hadoop-ozone] sadanand48 opened a new pull request #1408: HDDS-4217.Remove test TestOzoneContainerRatis

2020-09-08 Thread GitBox
sadanand48 opened a new pull request #1408: URL: https://github.com/apache/hadoop-ozone/pull/1408 ## What changes were proposed in this pull request? Delete test class TestOzoneContainerRatis as it has been disabled for a long time and the tests are no longer relevant. ## What is

[jira] [Created] (HDDS-4219) Revisit 'static' nature of OM Layout Version Manager.

2020-09-08 Thread Aravindan Vijayan (Jira)
Aravindan Vijayan created HDDS-4219: --- Summary: Revisit 'static' nature of OM Layout Version Manager. Key: HDDS-4219 URL: https://issues.apache.org/jira/browse/HDDS-4219 Project: Hadoop Distributed

[jira] [Commented] (HDDS-4155) Directory and filename can end up with same name in a path

2020-09-08 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192265#comment-17192265 ] Arpit Agarwal commented on HDDS-4155: - bq. s3a can handle both dir/file with the same name s3a does

[jira] [Assigned] (HDDS-4164) OM client request fails with "failed to commit as key is not found in OpenKey table"

2020-09-08 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal reassigned HDDS-4164: --- Assignee: Bharat Viswanadham > OM client request fails with "failed to commit as key is not

[jira] [Updated] (HDDS-4218) Remove test TestRatisManager

2020-09-08 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy updated HDDS-4218: -- Description: Delete this test as RatisManager is no longer present and this test has been

[jira] [Created] (HDDS-4218) Remove test TestRatisManager

2020-09-08 Thread Sadanand Shenoy (Jira)
Sadanand Shenoy created HDDS-4218: - Summary: Remove test TestRatisManager Key: HDDS-4218 URL: https://issues.apache.org/jira/browse/HDDS-4218 Project: Hadoop Distributed Data Store Issue

[jira] [Created] (HDDS-4217) Remove test TestOzoneContainerRatis

2020-09-08 Thread Sadanand Shenoy (Jira)
Sadanand Shenoy created HDDS-4217: - Summary: Remove test TestOzoneContainerRatis Key: HDDS-4217 URL: https://issues.apache.org/jira/browse/HDDS-4217 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] linyiqun commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
linyiqun commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r484858807 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/exceptions/SCMException.java ## @@ -123,6 +123,7 @@ public ResultCodes

[GitHub] [hadoop-ozone] sodonnel commented on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

2020-09-08 Thread GitBox
sodonnel commented on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-688774840 As I said before, I have a feeling that it *might* be better to have a separate service for deleting empty containers, and it would perhaps naturally be part of a "merge

[GitHub] [hadoop-ozone] sodonnel commented on a change in pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

2020-09-08 Thread GitBox
sodonnel commented on a change in pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#discussion_r484808717 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java ## @@ -312,6 +313,16 @@ private

[GitHub] [hadoop-ozone] bshashikant commented on pull request #1374: HDDS-4185. Remove IncrementalByteBuffer from Ozone client

2020-09-08 Thread GitBox
bshashikant commented on pull request #1374: URL: https://github.com/apache/hadoop-ozone/pull/1374#issuecomment-688760210 > @bshashikant Do you have any feedback? Thanks @elek for putting up the patch together. The patch looks good to me in general. I am just thinking of a scenario

[GitHub] [hadoop-ozone] ChenSammi edited a comment on pull request #1338: HDDS-4023. Delete closed container after all blocks have been deleted.

2020-09-08 Thread GitBox
ChenSammi edited a comment on pull request #1338: URL: https://github.com/apache/hadoop-ozone/pull/1338#issuecomment-687089550 Hi, @sodonnel do you have time for another review? This is an automated message from the Apache

[jira] [Commented] (HDDS-4155) Directory and filename can end up with same name in a path

2020-09-08 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192015#comment-17192015 ] Marton Elek commented on HDDS-4155: --- Thanks the answer [~arp]. > We will not "support" them, they will

[jira] [Commented] (HDDS-3841) FLAKY-UT: TestSecureOzoneRpcClient timeout

2020-09-08 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192009#comment-17192009 ] maobaolong commented on HDDS-3841: -- [~adoroszlai] Ok, we can close this. > FLAKY-UT:

[jira] [Resolved] (HDDS-3841) FLAKY-UT: TestSecureOzoneRpcClient timeout

2020-09-08 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong resolved HDDS-3841. -- Resolution: Cannot Reproduce > FLAKY-UT: TestSecureOzoneRpcClient timeout >

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r484708634 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -2244,6 +2248,14 @@ public

[GitHub] [hadoop-ozone] prashantpogde commented on pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#issuecomment-688677742 Uploaded the changes after addressing comments. Please take a look. This is an automated message

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r484708258 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/storage/ContainerProtocolCalls.java ## @@ -578,4 +581,35 @@ private

[jira] [Updated] (HDDS-4158) Provide a class type for Java based configuration

2020-09-08 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4158: - Labels: pull-request-available (was: ) > Provide a class type for Java based configuration >

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r484708528 ## File path: hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ScmSubcommand.java ## @@ -0,0 +1,43 @@ +/* + * Licensed to the

[GitHub] [hadoop-ozone] maobaolong opened a new pull request #1407: HDDS-4158. Provide a class type for Java based configuration

2020-09-08 Thread GitBox
maobaolong opened a new pull request #1407: URL: https://github.com/apache/hadoop-ozone/pull/1407 ## What changes were proposed in this pull request? Provide a class type for Java based configuration ## What is the link to the Apache JIRA HDDS-4158 ## How was

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r484706946 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/upgrade/SCMLayoutFeatureCatalog.java ## @@ -0,0 +1,93 @@ +/** +

[GitHub] [hadoop-ozone] prashantpogde commented on a change in pull request #1392: HDDS-4173. Implement HDDS Version management using the LayoutVersion…

2020-09-08 Thread GitBox
prashantpogde commented on a change in pull request #1392: URL: https://github.com/apache/hadoop-ozone/pull/1392#discussion_r484706750 ## File path: hadoop-hdds/server-scm/pom.xml ## @@ -29,6 +29,19 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd;> jar + + Review

[jira] [Resolved] (HDDS-4203) Publish docker image for ozone 1.0.0

2020-09-08 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek resolved HDDS-4203. --- Fix Version/s: 1.0.0 Resolution: Fixed > Publish docker image for ozone 1.0.0 >

[GitHub] [hadoop-docker-ozone] elek removed a comment on pull request #16: HDDS-4203. Publish apache/ozone:1.0.0 image

2020-09-08 Thread GitBox
elek removed a comment on pull request #16: URL: https://github.com/apache/hadoop-docker-ozone/pull/16#issuecomment-688664677 Merged. Thanks the review @adoroszlai This is an automated message from the Apache Git Service.

  1   2   >