[jira] [Created] (HDDS-4237) Testing Infrastructure for network partitioning

Rui Wang created HDDS-4237: -- Summary: Testing Infrastructure for network partitioning Key: HDDS-4237 URL: https://issues.apache.org/jira/browse/HDDS-4237 Project: Hadoop Distributed Data Store

[jira] [Assigned] (HDDS-3890) Stop BackgroundPipelineCreator when PipelineManager is closed

[ https://issues.apache.org/jira/browse/HDDS-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned HDDS-3890: -- Assignee: Rui Wang > Stop BackgroundPipelineCreator when PipelineManager is closed >

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1398: HDDS-4210. ResolveBucket during checkAcls fails.

amaliujia commented on a change in pull request #1398: URL: https://github.com/apache/hadoop-ozone/pull/1398#discussion_r487468731 ## File path: hadoop-ozone/dist/src/main/compose/ozonesecure-om-ha/test.sh ## @@ -30,6 +30,8 @@ execute_robot_test scm kinit.robot

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1405: HDDS-4143. Implement a factory for OM Requests that returns an instance based on layout version.

avijayanhwx commented on pull request #1405: URL: https://github.com/apache/hadoop-ozone/pull/1405#issuecomment-691330372 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] runzhiwang commented on pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#issuecomment-690911218 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] lamber-ken removed a comment on pull request #1417: HDDS-4324. Add important comment to ListVolumes logic

lamber-ken removed a comment on pull request #1417: URL: https://github.com/apache/hadoop-ozone/pull/1417#issuecomment-690928333 help This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r487246329 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSWithObjectStoreCreate.java ## @@ -303,6

[GitHub] [hadoop-ozone] bshashikant merged pull request #1408: HDDS-4217. Remove test TestOzoneContainerRatis

bshashikant merged pull request #1408: URL: https://github.com/apache/hadoop-ozone/pull/1408 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

lokeshj1703 commented on pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#issuecomment-690940004 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] lamber-ken commented on pull request #1417: HDDS-4324. Add important comment to ListVolumes logic

lamber-ken commented on pull request #1417: URL: https://github.com/apache/hadoop-ozone/pull/1417#issuecomment-690928333 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

amaliujia commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-691320364 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1413: HDDS-4228: add field 'num' to ALLOCATE_BLOCK of scm audit log.

timmylicheng commented on pull request #1413: URL: https://github.com/apache/hadoop-ozone/pull/1413#issuecomment-690833613 Merging This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop-ozone] ChenSammi merged pull request #1337: HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE.

ChenSammi merged pull request #1337: URL: https://github.com/apache/hadoop-ozone/pull/1337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] bshashikant merged pull request #1409: HDDS-4218. Remove test TestRatisManager

bshashikant merged pull request #1409: URL: https://github.com/apache/hadoop-ozone/pull/1409 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] elek commented on pull request #1396: HDDS-4150. recon.api.TestEndpoints test is flaky

elek commented on pull request #1396: URL: https://github.com/apache/hadoop-ozone/pull/1396#issuecomment-691426288 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1337: HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE.

ChenSammi commented on pull request #1337: URL: https://github.com/apache/hadoop-ozone/pull/1337#issuecomment-690919896 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] elek commented on pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

elek commented on pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336#issuecomment-691125675 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop-ozone] elek commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

elek commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-690929244 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1398: HDDS-4210. ResolveBucket during checkAcls fails.

bharatviswa504 commented on a change in pull request #1398: URL: https://github.com/apache/hadoop-ozone/pull/1398#discussion_r487178258 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -3523,6 +3533,47 @@ public

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

lokeshj1703 commented on a change in pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#discussion_r486905798 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyDeletingService.java ## @@ -66,9 +66,6 @@ private

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1414: HDDS-4231. Background Service blocks on task results.

lokeshj1703 commented on a change in pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#discussion_r486841264 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/BackgroundService.java ## @@ -62,11 +56,11 @@ public

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1414: HDDS-4231. Background Service blocks on task results.

lokeshj1703 commented on pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#issuecomment-690939847 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] elek merged pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

elek merged pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop-ozone] timmylicheng closed pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

timmylicheng closed pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1415: HDDS-4232. Use single thread for KeyDeletingService.

amaliujia commented on a change in pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#discussion_r487185841 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyDeletingService.java ## @@ -66,9 +66,6 @@ private static

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

amaliujia commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r486775970 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java ## @@ -174,6 +184,20 @@ public

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1296: HDDS-4053. Volume space: add quotaUsageInBytes and update it when write and delete key.

xiaoyuyao commented on a change in pull request #1296: URL: https://github.com/apache/hadoop-ozone/pull/1296#discussion_r486775263 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneVolume.java ## @@ -131,6 +133,18 @@ public

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

amaliujia commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-691320364 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] lamber-ken removed a comment on pull request #1417: HDDS-4324. Add important comment to ListVolumes logic

lamber-ken removed a comment on pull request #1417: URL: https://github.com/apache/hadoop-ozone/pull/1417#issuecomment-690928333 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r487246329 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSWithObjectStoreCreate.java ## @@ -303,6

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

timmylicheng commented on pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#issuecomment-690835337 > The client failover logic is based on the suggested leader sent by SCM. The `String` value of suggested leader sent by SCM Server is `RaftPeer#getAddress`, but at

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

timmylicheng commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r486736129 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMBlockLocationFailoverProxyProvider.java ## @@ -0,0

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

lokeshj1703 commented on pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#issuecomment-690940004 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] lamber-ken commented on pull request #1417: HDDS-4324. Add important comment to ListVolumes logic

lamber-ken commented on pull request #1417: URL: https://github.com/apache/hadoop-ozone/pull/1417#issuecomment-690928333 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] bshashikant merged pull request #1408: HDDS-4217. Remove test TestOzoneContainerRatis

bshashikant merged pull request #1408: URL: https://github.com/apache/hadoop-ozone/pull/1408 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] runzhiwang commented on pull request #1371: HDDS-2922. Balance ratis leader distribution in datanodes

runzhiwang commented on pull request #1371: URL: https://github.com/apache/hadoop-ozone/pull/1371#issuecomment-690911218 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1405: HDDS-4143. Implement a factory for OM Requests that returns an instance based on layout version.

avijayanhwx commented on pull request #1405: URL: https://github.com/apache/hadoop-ozone/pull/1405#issuecomment-691330372 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] bshashikant merged pull request #1409: HDDS-4218. Remove test TestRatisManager

bshashikant merged pull request #1409: URL: https://github.com/apache/hadoop-ozone/pull/1409 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] ChenSammi merged pull request #1337: HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE.

ChenSammi merged pull request #1337: URL: https://github.com/apache/hadoop-ozone/pull/1337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] elek commented on pull request #1396: HDDS-4150. recon.api.TestEndpoints test is flaky

elek commented on pull request #1396: URL: https://github.com/apache/hadoop-ozone/pull/1396#issuecomment-691426288 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1337: HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE.

ChenSammi commented on pull request #1337: URL: https://github.com/apache/hadoop-ozone/pull/1337#issuecomment-690919896 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] elek commented on pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

elek commented on pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336#issuecomment-691125675 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop-ozone] timmylicheng merged pull request #1413: HDDS-4228: add field 'num' to ALLOCATE_BLOCK of scm audit log.

timmylicheng merged pull request #1413: URL: https://github.com/apache/hadoop-ozone/pull/1413 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] elek commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

elek commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-690929244 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1398: HDDS-4210. ResolveBucket during checkAcls fails.

bharatviswa504 commented on a change in pull request #1398: URL: https://github.com/apache/hadoop-ozone/pull/1398#discussion_r487178258 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java ## @@ -3523,6 +3533,47 @@ public

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

lokeshj1703 commented on a change in pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#discussion_r486905798 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyDeletingService.java ## @@ -66,9 +66,6 @@ private

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1414: HDDS-4231. Background Service blocks on task results.

lokeshj1703 commented on a change in pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#discussion_r486841264 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/BackgroundService.java ## @@ -62,11 +56,11 @@ public

[GitHub] [hadoop-ozone] elek merged pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

elek merged pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1411: HDDS-4097. [DESIGN] S3/Ozone Filesystem inter-op

xiaoyuyao commented on a change in pull request #1411: URL: https://github.com/apache/hadoop-ozone/pull/1411#discussion_r486757610 ## File path: hadoop-hdds/docs/content/design/s3_hcfs.md ## @@ -0,0 +1,280 @@ +--- +title: S3/Ozone Filesystem inter-op +summary: How to support

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1415: HDDS-4232. Use single thread for KeyDeletingService.

amaliujia commented on a change in pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#discussion_r487185841 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyDeletingService.java ## @@ -66,9 +66,6 @@ private static

[GitHub] [hadoop-ozone] timmylicheng closed pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

timmylicheng closed pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop-ozone] amaliujia commented on a change in pull request #1412: HDDS-3751. Ozone sh client support bucket quota option.

amaliujia commented on a change in pull request #1412: URL: https://github.com/apache/hadoop-ozone/pull/1412#discussion_r486775970 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java ## @@ -174,6 +184,20 @@ public

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1414: HDDS-4231. Background Service blocks on task results.

lokeshj1703 commented on pull request #1414: URL: https://github.com/apache/hadoop-ozone/pull/1414#issuecomment-690939847 Pushed another commit to fix a unit test. This is an automated message from the Apache Git Service. To

[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #1296: HDDS-4053. Volume space: add quotaUsageInBytes and update it when write and delete key.

xiaoyuyao commented on a change in pull request #1296: URL: https://github.com/apache/hadoop-ozone/pull/1296#discussion_r486775263 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneVolume.java ## @@ -131,6 +133,18 @@ public

[GitHub] [hadoop-ozone] amaliujia commented on pull request #1410: HDDS-3102. ozone getconf command should use the GenericCli parent class

amaliujia commented on pull request #1410: URL: https://github.com/apache/hadoop-ozone/pull/1410#issuecomment-691320364 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] lamber-ken removed a comment on pull request #1417: HDDS-4324. Add important comment to ListVolumes logic

lamber-ken removed a comment on pull request #1417: URL: https://github.com/apache/hadoop-ozone/pull/1417#issuecomment-690928333 help This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1361: HDDS-4155. Directory and filename can end up with same name in a path.

bharatviswa504 commented on a change in pull request #1361: URL: https://github.com/apache/hadoop-ozone/pull/1361#discussion_r487246329 ## File path: hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSWithObjectStoreCreate.java ## @@ -303,6

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1413: HDDS-4228: add field 'num' to ALLOCATE_BLOCK of scm audit log.

timmylicheng commented on pull request #1413: URL: https://github.com/apache/hadoop-ozone/pull/1413#issuecomment-690833613 Merging This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

timmylicheng commented on pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#issuecomment-690835337 > The client failover logic is based on the suggested leader sent by SCM. The `String` value of suggested leader sent by SCM Server is `RaftPeer#getAddress`, but at

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #1340: HDDS-3188 Add failover proxy for SCM block location.

timmylicheng commented on a change in pull request #1340: URL: https://github.com/apache/hadoop-ozone/pull/1340#discussion_r486736129 ## File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/proxy/SCMBlockLocationFailoverProxyProvider.java ## @@ -0,0

[GitHub] [hadoop-ozone] lokeshj1703 commented on pull request #1415: HDDS-4232. Use single thread for KeyDeletingService and OpenKeyCleanupService.

lokeshj1703 commented on pull request #1415: URL: https://github.com/apache/hadoop-ozone/pull/1415#issuecomment-690940004 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop-ozone] lamber-ken commented on pull request #1417: HDDS-4324. Add important comment to ListVolumes logic

lamber-ken commented on pull request #1417: URL: https://github.com/apache/hadoop-ozone/pull/1417#issuecomment-690928333 help This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Created] (HDDS-4236) Move "Om*Codec.java" to new project hadoop-ozone/interface-storage

Rui Wang created HDDS-4236: -- Summary: Move "Om*Codec.java" to new project hadoop-ozone/interface-storage Key: HDDS-4236 URL: https://issues.apache.org/jira/browse/HDDS-4236 Project: Hadoop Distributed Data

[GitHub] [hadoop-ozone] elek commented on pull request #1396: HDDS-4150. recon.api.TestEndpoints test is flaky

elek commented on pull request #1396: URL: https://github.com/apache/hadoop-ozone/pull/1396#issuecomment-691426288 Sorry, I am late in the game to review it, but it shouldn't be too late to say: thanks a lot to fix this flakiness.