[jira] [Updated] (HDDS-3683) Ozone fuse support

2020-06-10 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong updated HDDS-3683: - Description: https://github.com/opendataio/hcfsfuse design doc will be updated here.

[GitHub] [hadoop-ozone] codecov-commenter commented on pull request #1054: Hdds 3772. Add LOG to S3ErrorTable for easier problem locating.

2020-06-10 Thread GitBox
codecov-commenter commented on pull request #1054: URL: https://github.com/apache/hadoop-ozone/pull/1054#issuecomment-642411148 # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1054?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base

[jira] [Updated] (HDDS-3481) SCM ask too many datanodes to replicate the same container

2020-06-10 Thread Nanda kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-3481: -- Status: Patch Available (was: Open) > SCM ask too many datanodes to replicate the same container >

[GitHub] [hadoop-ozone] leosunli commented on a change in pull request #1033: HDDS-3667. If we gracefully stop datanode it would be better to notify scm and r…

2020-06-10 Thread GitBox
leosunli commented on a change in pull request #1033: URL: https://github.com/apache/hadoop-ozone/pull/1033#discussion_r438540923 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/UnRegisterEndpointTask.java ##

[jira] [Updated] (HDDS-3481) SCM ask too many datanodes to replicate the same container

2020-06-10 Thread Nanda kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-3481: -- Labels: Triaged pull-request-available (was: TriagePending pull-request-available) > SCM ask too many

[GitHub] [hadoop-ozone] ChenSammi opened a new pull request #1054: Hdds 3772. Add LOG to S3ErrorTable for easier problem locating.

2020-06-10 Thread GitBox
ChenSammi opened a new pull request #1054: URL: https://github.com/apache/hadoop-ozone/pull/1054 https://issues.apache.org/jira/browse/HDDS-3772 This is an automated message from the Apache Git Service. To respond to the

[jira] [Updated] (HDDS-3772) Add LOG to S3ErrorTable for easier problem locating

2020-06-10 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen updated HDDS-3772: - Description: Currently it's hard to directly tell the failure reason when something unexpected happened.

[jira] [Created] (HDDS-3776) Upgrading RocksDB version to avoid java heap issue

2020-06-10 Thread Li Cheng (Jira)
Li Cheng created HDDS-3776: -- Summary: Upgrading RocksDB version to avoid java heap issue Key: HDDS-3776 URL: https://issues.apache.org/jira/browse/HDDS-3776 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1019: HDDS-3679. Add unit tests for PipelineManagerV2.

2020-06-10 Thread GitBox
timmylicheng commented on pull request #1019: URL: https://github.com/apache/hadoop-ozone/pull/1019#issuecomment-642370476 @elek Tests seem passed here. I created https://issues.apache.org/jira/browse/HDDS-3776 to track rocksdb upgrade.

[jira] [Commented] (HDDS-3499) Address compatibility issue by SCM DB instances change

2020-06-10 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132853#comment-17132853 ] Li Cheng commented on HDDS-3499: [~arp] Our internal production deployment is still on schedule. But we

[jira] [Resolved] (HDDS-3499) Address compatibility issue by SCM DB instances change

2020-06-10 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Cheng resolved HDDS-3499. Fix Version/s: 0.6.0 Resolution: Fixed > Address compatibility issue by SCM DB instances change >

[GitHub] [hadoop-ozone] codecov-commenter edited a comment on pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
codecov-commenter edited a comment on pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#issuecomment-642271726 # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/986?src=pr=h1) Report > Merging

[jira] [Updated] (HDDS-3737) Improve OM performance

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3737: - Labels: pull-request-available (was: ) > Improve OM performance > -- > >

[GitHub] [hadoop-ozone] runzhiwang opened a new pull request #1053: HDDS-3737. Avoid serialization between UUID and String

2020-06-10 Thread GitBox
runzhiwang opened a new pull request #1053: URL: https://github.com/apache/hadoop-ozone/pull/1053 ## What changes were proposed in this pull request? **What's the problem ?** Serialization between UUID and String: UUID.toString (I have improved this) and UUID.fromString, not only

[GitHub] [hadoop-ozone] codecov-commenter commented on pull request #1002: HDDS-3642. Stop/Pause Background services while replacing OM DB with checkpoint from Leader

2020-06-10 Thread GitBox
codecov-commenter commented on pull request #1002: URL: https://github.com/apache/hadoop-ozone/pull/1002#issuecomment-642325738 # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/1002?src=pr=h1) Report > Merging

[GitHub] [hadoop-ozone] codecov-commenter edited a comment on pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
codecov-commenter edited a comment on pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#issuecomment-642271726 # [Codecov](https://codecov.io/gh/apache/hadoop-ozone/pull/986?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #1052: HDDS-3749. Addendum: Fix checkstyle issue.

2020-06-10 Thread GitBox
bharatviswa504 merged pull request #1052: URL: https://github.com/apache/hadoop-ozone/pull/1052 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #1034: HDDS-3749. Improve OM performance with 3.7% by avoid stream.collect

2020-06-10 Thread GitBox
bharatviswa504 commented on pull request #1034: URL: https://github.com/apache/hadoop-ozone/pull/1034#issuecomment-642305014 Hi @xiaoyuyao This has caused checkstyle issues and PR's are failing with CI run. Posted a PR to fix this https://github.com/apache/hadoop-ozone/pull/1052

[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #1052: HDDS-3479. Addendum: Fix checkstyle issue.

2020-06-10 Thread GitBox
bharatviswa504 opened a new pull request #1052: URL: https://github.com/apache/hadoop-ozone/pull/1052 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## What is the link to the Apache JIRA (Please create an issue in ASF

[jira] [Resolved] (HDDS-3749) Improve OM performance with 3.7% by avoid stream.collect

2020-06-10 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao resolved HDDS-3749. -- Fix Version/s: 0.6.0 Resolution: Fixed > Improve OM performance with 3.7% by avoid

[GitHub] [hadoop-ozone] xiaoyuyao merged pull request #1034: HDDS-3749. Improve OM performance with 3.7% by avoid stream.collect

2020-06-10 Thread GitBox
xiaoyuyao merged pull request #1034: URL: https://github.com/apache/hadoop-ozone/pull/1034 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
bharatviswa504 commented on a change in pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#discussion_r438435882 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java ## @@ -338,20 +352,19 @@

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
bharatviswa504 commented on a change in pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#discussion_r438435468 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java ## @@ -338,20 +352,19 @@

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
bharatviswa504 commented on a change in pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#discussion_r438435468 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java ## @@ -338,20 +352,19 @@

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
bharatviswa504 commented on a change in pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#discussion_r438434496 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java ## @@ -338,20 +352,19 @@

[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
bharatviswa504 commented on pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#issuecomment-642292921 Thank You @hanishakoneru for the review. I have addressed review comments. This is an automated

[GitHub] [hadoop-ozone] hanishakoneru commented on a change in pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
hanishakoneru commented on a change in pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#discussion_r438432283 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java ## @@ -338,20 +352,19 @@

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
bharatviswa504 commented on a change in pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#discussion_r438431531 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java ## @@ -338,20 +352,19 @@

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
bharatviswa504 commented on a change in pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#discussion_r438407648 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/OzoneManagerSnapshotProvider.java ## @@ -112,16

[GitHub] [hadoop-ozone] hanishakoneru commented on a change in pull request #986: HDDS-3476. Use persisted transaction info during OM startup in OM StateMachine.

2020-06-10 Thread GitBox
hanishakoneru commented on a change in pull request #986: URL: https://github.com/apache/hadoop-ozone/pull/986#discussion_r438353082 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java ## @@ -259,16 +261,25 @@ public

[jira] [Commented] (HDDS-1134) OzoneFileSystem#create should allocate alteast one block for future writes.

2020-06-10 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132701#comment-17132701 ] Bharat Viswanadham commented on HDDS-1134: -- Hi [~msingh] I see this is being handled in

[jira] [Resolved] (HDDS-1134) OzoneFileSystem#create should allocate alteast one block for future writes.

2020-06-10 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham resolved HDDS-1134. -- Fix Version/s: 0.6.0 Resolution: Fixed This has been already fixed. Right now,

[GitHub] [hadoop-ozone] smengcl edited a comment on pull request #1046: HDDS-3767. [OFS] Address merge conflicts after HDDS-3627

2020-06-10 Thread GitBox
smengcl edited a comment on pull request #1046: URL: https://github.com/apache/hadoop-ozone/pull/1046#issuecomment-642120503 > Thanks this patch @smengcl (And sorry for the master changes, we worked paralell.) > > Understanding this PR is really challenging. Finally, I fetched the

[jira] [Updated] (HDDS-3685) Remove replay logic from actual request logic

2020-06-10 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDDS-3685: - Priority: Critical (was: Major) > Remove replay logic from actual request logic >

[jira] [Comment Edited] (HDDS-3707) UUID can be non unique for a huge samples

2020-06-10 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132675#comment-17132675 ] Arpit Agarwal edited comment on HDDS-3707 at 6/10/20, 7:52 PM: --- Hi

[jira] [Commented] (HDDS-3707) UUID can be non unique for a huge samples

2020-06-10 Thread Arpit Agarwal (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132675#comment-17132675 ] Arpit Agarwal commented on HDDS-3707: - Hi [~maobaolong] the probability is so infinitesimal I don't

[jira] [Resolved] (HDDS-3639) Maintain FileHandle Information in OMMetadataManager

2020-06-10 Thread Hanisha Koneru (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanisha Koneru resolved HDDS-3639. -- Resolution: Fixed > Maintain FileHandle Information in OMMetadataManager >

[jira] [Created] (HDDS-3775) Add documentation for flame graph

2020-06-10 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HDDS-3775: - Summary: Add documentation for flame graph Key: HDDS-3775 URL: https://issues.apache.org/jira/browse/HDDS-3775 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] vivekratnavel commented on pull request #1047: HDDS-3726. Upload code coverage data to Codecov and enable checks in …

2020-06-10 Thread GitBox
vivekratnavel commented on pull request #1047: URL: https://github.com/apache/hadoop-ozone/pull/1047#issuecomment-642162869 @elek Thanks for the review and merge! This is an automated message from the Apache Git Service. To

[GitHub] [hadoop-ozone] maobaolong commented on pull request #1051: Redundancy if condition code in ListPipelinesSubcommand

2020-06-10 Thread GitBox
maobaolong commented on pull request #1051: URL: https://github.com/apache/hadoop-ozone/pull/1051#issuecomment-642148224 @bhemanthkumar Thanks for working on this, please fix the style problem. Also, please update the description from the given template. Reference this PR.

[GitHub] [hadoop-ozone] maobaolong commented on a change in pull request #1051: Redundancy if condition code in ListPipelinesSubcommand

2020-06-10 Thread GitBox
maobaolong commented on a change in pull request #1051: URL: https://github.com/apache/hadoop-ozone/pull/1051#discussion_r438285600 ## File path: hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/pipeline/ListPipelinesSubcommand.java ## @@ -54,17 +54,13 @@

[jira] [Updated] (HDDS-3726) Upload code coverage to Codecov and enable checks in PR workflow of Github Actions

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDDS-3726: -- Fix Version/s: 0.6.0 > Upload code coverage to Codecov and enable checks in PR workflow of Github >

[GitHub] [hadoop-ozone] smengcl commented on pull request #1046: HDDS-3767. [OFS] Address merge conflicts after HDDS-3627

2020-06-10 Thread GitBox
smengcl commented on pull request #1046: URL: https://github.com/apache/hadoop-ozone/pull/1046#issuecomment-642120503 > Thanks this patch @smengcl (And sorry for the master changes, we worked paralell.) > > Understanding this PR is really challenging. Finally, I fetched the PR

[jira] [Comment Edited] (HDDS-3747) Remove the redundancy if condition code in ListPipelinesSubcommand

2020-06-10 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130830#comment-17130830 ] hemanthboyina edited comment on HDDS-3747 at 6/10/20, 4:02 PM: --- raised a PR

[jira] [Commented] (HDDS-3747) Remove the redundancy if condition code in ListPipelinesSubcommand

2020-06-10 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130830#comment-17130830 ] hemanthboyina commented on HDDS-3747: - raised a PR : 

[GitHub] [hadoop-ozone] bhemanthkumar opened a new pull request #1051: Redundancy if condition code in ListPipelinesSubcommand

2020-06-10 Thread GitBox
bhemanthkumar opened a new pull request #1051: URL: https://github.com/apache/hadoop-ozone/pull/1051 Remove the redundancy if condition code in ListPipelinesSubcommand ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ##

[jira] [Commented] (HDDS-3512) s3g multi-upload saved content incorrect when client uses aws java sdk 1.11.* jar

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130792#comment-17130792 ] Marton Elek commented on HDDS-3512: --- Is it still a problem? I tried to reproduce it with freon:

[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1048: HDDS-3481. SCM ask too many datanodes to replicate the same container

2020-06-10 Thread GitBox
nandakumar131 commented on pull request #1048: URL: https://github.com/apache/hadoop-ozone/pull/1048#issuecomment-642077756 @runzhiwang was this PR closed by accident? This is an automated message from the Apache Git

[jira] [Commented] (HDDS-3481) SCM ask too many datanodes to replicate the same container

2020-06-10 Thread Nanda kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130772#comment-17130772 ] Nanda kumar commented on HDDS-3481: --- [~yjxxtd], [~xyao], [~elek], [~sodonnell] I completely agree. We

[jira] [Created] (HDDS-3774) Throttle replication commands sent to datanode

2020-06-10 Thread Nanda kumar (Jira)
Nanda kumar created HDDS-3774: - Summary: Throttle replication commands sent to datanode Key: HDDS-3774 URL: https://issues.apache.org/jira/browse/HDDS-3774 Project: Hadoop Distributed Data Store

[jira] [Assigned] (HDDS-3512) s3g multi-upload saved content incorrect when client uses aws java sdk 1.11.* jar

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek reassigned HDDS-3512: - Assignee: Marton Elek > s3g multi-upload saved content incorrect when client uses aws java sdk

[jira] [Updated] (HDDS-2447) Allow datanodes to operate with simulated containers

2020-06-10 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-2447: -- Target Version/s: 0.7.0 (was: 0.6.0) > Allow datanodes to operate with simulated containers >

[jira] [Updated] (HDDS-2449) Delete block command should use a thread pool

2020-06-10 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-2449: -- Target Version/s: 0.7.0 (was: 0.6.0) > Delete block command should use a thread pool >

[GitHub] [hadoop-ozone] elek commented on pull request #1019: HDDS-3679. Add unit tests for PipelineManagerV2.

2020-06-10 Thread GitBox
elek commented on pull request #1019: URL: https://github.com/apache/hadoop-ozone/pull/1019#issuecomment-642035752 > @elek Shall we make a separate commit to upgrade rocksdb version? I am open for both approaches, but seems to be a good idea to do it on `master`, too. I am

[jira] [Assigned] (HDDS-3773) Add OMDBDefinition to define structure of om.db

2020-06-10 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy reassigned HDDS-3773: - Assignee: Sadanand Shenoy > Add OMDBDefinition to define structure of om.db >

[jira] [Created] (HDDS-3773) Add OMDBDefinition to define structure of om.db

2020-06-10 Thread Sadanand Shenoy (Jira)
Sadanand Shenoy created HDDS-3773: - Summary: Add OMDBDefinition to define structure of om.db Key: HDDS-3773 URL: https://issues.apache.org/jira/browse/HDDS-3773 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] elek merged pull request #1043: HDDS-3760. Avoid UUID#toString call in Pipeline#getProtobufMessage

2020-06-10 Thread GitBox
elek merged pull request #1043: URL: https://github.com/apache/hadoop-ozone/pull/1043 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1005: HDDS-3350. Ozone Retry Policy Improvements.

2020-06-10 Thread GitBox
lokeshj1703 commented on a change in pull request #1005: URL: https://github.com/apache/hadoop-ozone/pull/1005#discussion_r438144376 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java ## @@ -281,15 +281,33 @@ public static final

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1005: HDDS-3350. Ozone Retry Policy Improvements.

2020-06-10 Thread GitBox
lokeshj1703 commented on a change in pull request #1005: URL: https://github.com/apache/hadoop-ozone/pull/1005#discussion_r438144281 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java ## @@ -269,23 +282,76 @@ static GrpcTlsConfig

[GitHub] [hadoop-ozone] lokeshj1703 commented on a change in pull request #1005: HDDS-3350. Ozone Retry Policy Improvements.

2020-06-10 Thread GitBox
lokeshj1703 commented on a change in pull request #1005: URL: https://github.com/apache/hadoop-ozone/pull/1005#discussion_r438143919 ## File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java ## @@ -269,23 +282,76 @@ static GrpcTlsConfig

[jira] [Resolved] (HDDS-3622) Implement rocksdb tool to parse scm db

2020-06-10 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy resolved HDDS-3622. --- Target Version/s: 0.6.0 Resolution: Resolved > Implement rocksdb tool to parse scm

[jira] [Updated] (HDDS-3622) Implement rocksdb tool to parse scm db

2020-06-10 Thread Sadanand Shenoy (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadanand Shenoy updated HDDS-3622: -- Component/s: Tools > Implement rocksdb tool to parse scm db >

[GitHub] [hadoop-ozone] sadanand48 closed pull request #864: HDDS-3405. Tool for Listing keys from the OpenKeyTable

2020-06-10 Thread GitBox
sadanand48 closed pull request #864: URL: https://github.com/apache/hadoop-ozone/pull/864 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop-ozone] sadanand48 edited a comment on pull request #864: HDDS-3405. Tool for Listing keys from the OpenKeyTable

2020-06-10 Thread GitBox
sadanand48 edited a comment on pull request #864: URL: https://github.com/apache/hadoop-ozone/pull/864#issuecomment-642019017 > Do we need this patch? It seems to be more easy to extend #945 / [HDDS-3622](https://issues.apache.org/jira/browse/HDDS-3622) with supporting OM... > >

[GitHub] [hadoop-ozone] sadanand48 commented on pull request #864: HDDS-3405. Tool for Listing keys from the OpenKeyTable

2020-06-10 Thread GitBox
sadanand48 commented on pull request #864: URL: https://github.com/apache/hadoop-ozone/pull/864#issuecomment-642019017 > Do we need this patch? It seems to be more easy to extend #945 / [HDDS-3622](https://issues.apache.org/jira/browse/HDDS-3622) with supporting OM... > > What do

[jira] [Updated] (HDDS-3757) Add test coverage of the acceptance tests to overall test coverage

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3757: - Labels: pull-request-available (was: ) > Add test coverage of the acceptance tests to overall

[GitHub] [hadoop-ozone] elek opened a new pull request #1050: HDDS-3757. Add test coverage of the acceptance tests to overall test coverage

2020-06-10 Thread GitBox
elek opened a new pull request #1050: URL: https://github.com/apache/hadoop-ozone/pull/1050 ## What changes were proposed in this pull request? This patch adds the coverage data from the acceptance test to the generic coverage measurement. There was one question during the

[GitHub] [hadoop-ozone] elek commented on pull request #864: HDDS-3405. Tool for Listing keys from the OpenKeyTable

2020-06-10 Thread GitBox
elek commented on pull request #864: URL: https://github.com/apache/hadoop-ozone/pull/864#issuecomment-642015500 Do we need this patch? It seems to be more easy to extend #945 / HDDS-3622 with supporting OM... What do you think?

[jira] [Created] (HDDS-3772) Add LOG to S3ErrorTable for easier problem locating

2020-06-10 Thread Sammi Chen (Jira)
Sammi Chen created HDDS-3772: Summary: Add LOG to S3ErrorTable for easier problem locating Key: HDDS-3772 URL: https://issues.apache.org/jira/browse/HDDS-3772 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] elek commented on pull request #1031: HDDS-3745. Improve OM and SCM performance with 64% by avoid call getServiceInfo in s3g

2020-06-10 Thread GitBox
elek commented on pull request #1031: URL: https://github.com/apache/hadoop-ozone/pull/1031#issuecomment-642010606 Thanks the patch @runzhiwang It looks correct me, but it's also a question about the long-term usage of `getServiceInfo`. Originally it was introduced (as far as I

[jira] [Resolved] (HDDS-3750) Improve SCM performance with 3.2% by avoid stream.collect

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek resolved HDDS-3750. --- Fix Version/s: 0.6.0 Resolution: Fixed > Improve SCM performance with 3.2% by avoid

[GitHub] [hadoop-ozone] elek merged pull request #1035: HDDS-3750. Improve SCM performance with 3.2% by avoid stream.collect

2020-06-10 Thread GitBox
elek merged pull request #1035: URL: https://github.com/apache/hadoop-ozone/pull/1035 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] runzhiwang closed pull request #1048: HDDS-3481. SCM ask too many datanodes to replicate the same container

2020-06-10 Thread GitBox
runzhiwang closed pull request #1048: URL: https://github.com/apache/hadoop-ozone/pull/1048 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Commented] (HDDS-3481) SCM ask too many datanodes to replicate the same container

2020-06-10 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130640#comment-17130640 ] Stephen O'Donnell commented on HDDS-3481: - I saw this problem a long time back in theory from

[jira] [Commented] (HDDS-3755) Storage-class support for Ozone

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130629#comment-17130629 ] Marton Elek commented on HDDS-3755: --- [~maobaolong] Would be great to provide an example configuration

[jira] [Commented] (HDDS-3755) Storage-class support for Ozone

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130626#comment-17130626 ] Marton Elek commented on HDDS-3755: --- *Decouple EC from this feature:* For me storage-class, is a

[jira] [Commented] (HDDS-3481) SCM ask too many datanodes to replicate the same container

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130610#comment-17130610 ] Marton Elek commented on HDDS-3481: --- {quote}bq. Should we consider balancing the replication source

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1019: HDDS-3679. Add unit tests for PipelineManagerV2.

2020-06-10 Thread GitBox
timmylicheng commented on pull request #1019: URL: https://github.com/apache/hadoop-ozone/pull/1019#issuecomment-641967456 > FYI: can reproduce it on linux, locally. > > It seems to be disappeared when I upgraded my rocksdb version in the main `pom.xml`: > > ``` > -

[jira] [Resolved] (HDDS-2880) Rename legacy/current ozonefs to isolated/share

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek resolved HDDS-2880. --- Release Note: They are refactor and renamed during HDDS-3458. We don't have isolated any more as we

[jira] [Commented] (HDDS-3519) Finalize network and storage protocol of Ozone

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130596#comment-17130596 ] Marton Elek commented on HDDS-3519: --- I think it's very close to any intra-service protocol (as far as

[jira] [Resolved] (HDDS-1328) Add a new API getS3Bucket

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek resolved HDDS-1328. --- Resolution: Won't Fix We don't need it after the recent s3 volume mapping change. > Add a new API

[GitHub] [hadoop-ozone] elek commented on pull request #1019: HDDS-3679. Add unit tests for PipelineManagerV2.

2020-06-10 Thread GitBox
elek commented on pull request #1019: URL: https://github.com/apache/hadoop-ozone/pull/1019#issuecomment-641960612 FYI: can reproduce it on linux, locally. It seems to be disappeared when I upgraded my rocksdb version in the main `pom.xml`: ``` -6.6.4 +

[jira] [Commented] (HDDS-3761) "ozone fs -get" is way to slow than "ozone sh key get"

2020-06-10 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130574#comment-17130574 ] Sammi Chen commented on HDDS-3761: -- Hi [~msingh] and [~rakeshr], Please hold on the investigation, the

[GitHub] [hadoop-ozone] timmylicheng edited a comment on pull request #1019: HDDS-3679. Add unit tests for PipelineManagerV2.

2020-06-10 Thread GitBox
timmylicheng edited a comment on pull request #1019: URL: https://github.com/apache/hadoop-ozone/pull/1019#issuecomment-641767465 Dump shows it's related to rocksdb. Not sure if it's related to multiple DBs being merged. Stack looks weird to me. Any ideas? @nandakumar131 @elek @xiaoyuyao

[GitHub] [hadoop-ozone] elek commented on pull request #1038: HDDS-3754. Rename framework to common-server

2020-06-10 Thread GitBox
elek commented on pull request #1038: URL: https://github.com/apache/hadoop-ozone/pull/1038#issuecomment-641935708 @nandakumar131 Not sure if understood your proposal. Do you propose to create separated projects for separated framework/services? Like `framework-eventqueue`, or

[jira] [Updated] (HDDS-3662) decouple finalize and destroy pipeline

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3662: - Labels: pull-request-available (was: ) > decouple finalize and destroy pipeline >

[GitHub] [hadoop-ozone] timmylicheng opened a new pull request #1049: HDDS-3662 Decouple finalizeAndDestroyPipeline.

2020-06-10 Thread GitBox
timmylicheng opened a new pull request #1049: URL: https://github.com/apache/hadoop-ozone/pull/1049 ## What changes were proposed in this pull request? Decouple finalizeAndDestroyPipeline. Close pipeline should only update the pipeline state, it’s the job of the caller to issue

[jira] [Commented] (HDDS-3764) Spark is failing with no such method exception

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130482#comment-17130482 ] Marton Elek commented on HDDS-3764: --- Sure, I merged it. Yes, it seems to be duplicate (Spark uses Hadoop

[jira] [Resolved] (HDDS-3715) Improvement for OzoneFS client to work with Hadoop 2.7.3

2020-06-10 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek resolved HDDS-3715. --- Fix Version/s: 0.6.0 Target Version/s: 0.6.0 Resolution: Fixed > Improvement for

[jira] [Updated] (HDDS-3715) Improvement for OzoneFS client to work with Hadoop 2.7.3

2020-06-10 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3715: - Labels: pull-request-available (was: ) > Improvement for OzoneFS client to work with Hadoop

[GitHub] [hadoop-ozone] elek merged pull request #1036: HDDS-3715. Improvement for OzoneFS client to work with Hadoop 2.7.3.

2020-06-10 Thread GitBox
elek merged pull request #1036: URL: https://github.com/apache/hadoop-ozone/pull/1036 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop-ozone] elek merged pull request #945: HDDS-3622. Implement rocksdb tool to parse scm db

2020-06-10 Thread GitBox
elek merged pull request #945: URL: https://github.com/apache/hadoop-ozone/pull/945 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HDDS-3755) Storage-class support for Ozone

2020-06-10 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130397#comment-17130397 ] maobaolong commented on HDDS-3755: -- [~arp] Yeah, i agree with decouple EC from this feature. [~elek] I

[jira] [Comment Edited] (HDDS-3755) Storage-class support for Ozone

2020-06-10 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128866#comment-17128866 ] maobaolong edited comment on HDDS-3755 at 6/10/20, 8:06 AM: [~elek]Thanks for

[jira] [Resolved] (HDDS-3771) Block when using ’ozone fs -cat o3fs://xxxxx.xxxx/xxx‘

2020-06-10 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mingchao zhao resolved HDDS-3771. - Resolution: Cannot Reproduce > Block when using ’ozone fs -cat o3fs://x./xxx‘ >

[jira] [Commented] (HDDS-3771) Block when using ’ozone fs -cat o3fs://xxxxx.xxxx/xxx‘

2020-06-10 Thread mingchao zhao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130368#comment-17130368 ] mingchao zhao commented on HDDS-3771: - This problem is no longer present in the latest version of the

[jira] [Commented] (HDDS-3683) Ozone fuse support

2020-06-10 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130367#comment-17130367 ] maobaolong commented on HDDS-3683: -- [~msingh] [~aryangupta1998] [~nanda] Thanks for your help to let me

[jira] [Updated] (HDDS-3725) Ozone sh volume client support quota option.

2020-06-10 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sammi Chen updated HDDS-3725: - Target Version/s: 0.7.0 > Ozone sh volume client support quota option. >

[jira] [Updated] (HDDS-3683) Ozone fuse support

2020-06-10 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong updated HDDS-3683: - Attachment: screenshot-2.png > Ozone fuse support > --- > > Key:

[jira] [Updated] (HDDS-3683) Ozone fuse support

2020-06-10 Thread maobaolong (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong updated HDDS-3683: - Attachment: screenshot-3.png > Ozone fuse support > --- > > Key:

  1   2   >