[GitHub] [hadoop-ozone] smengcl edited a comment on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl edited a comment on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603999618 Test failure assessment: - acceptance failure is from smoketest `ozonesecure-mr-mapreduce`. I recall witnessing the same failure before but

[jira] [Updated] (HDDS-3088) maxRetries value is too large while trying to reconnect to SCM server

2020-03-25 Thread Mukul Kumar Singh (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDDS-3088: Priority: Major (was: Blocker) > maxRetries value is too large while trying to reconnect

[GitHub] [hadoop-ozone] elek commented on issue #702: HDDS-3241. Invalid container reported to SCM should be deleted.

2020-03-25 Thread GitBox
elek commented on issue #702: HDDS-3241. Invalid container reported to SCM should be deleted. URL: https://github.com/apache/hadoop-ozone/pull/702#issuecomment-603972493 Note: see the discussion in the jira: https://issues.apache.org/jira/browse/HDDS-3241

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3279: - Labels: pull-request-available (was: ) > Rebase OFS branch > - > >

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Status: Patch Available (was: Open) > Rebase OFS branch > - > > Key:

[jira] [Updated] (HDDS-3074) Make the configuration of container scrub consistent

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3074: - Labels: pull-request-available (was: ) > Make the configuration of container scrub consistent >

[GitHub] [hadoop-ozone] elek commented on issue #720: HDDS-3185 Construct a standalone ratis server for SCM.

2020-03-25 Thread GitBox
elek commented on issue #720: HDDS-3185 Construct a standalone ratis server for SCM. URL: https://github.com/apache/hadoop-ozone/pull/720#issuecomment-603977290 Thanks to work on this @timmylicheng I am confused a little bit because this was the last message on the mailing list:

[GitHub] [hadoop-ozone] smengcl opened a new pull request #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl opened a new pull request #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721 ## What changes were proposed in this pull request? Merge commits on master branch to OFS dev branch: `git merge master` Also need to manually apply a

[GitHub] [hadoop-ozone] cku328 opened a new pull request #722: HDDS-3074. Make the configuration of container scrub consistent.

2020-03-25 Thread GitBox
cku328 opened a new pull request #722: HDDS-3074. Make the configuration of container scrub consistent. URL: https://github.com/apache/hadoop-ozone/pull/722 ## What changes were proposed in this pull request? When we set `hdds.container.scrub.enabled` (in ozone-site.xml) to true,

[jira] [Commented] (HDDS-3241) Invalid container reported to SCM should be deleted

2020-03-25 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066888#comment-17066888 ] Marton Elek commented on HDDS-3241: --- bq. The issue described by you is a case where the user had changed

[GitHub] [hadoop-ozone] smengcl commented on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl commented on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603999618 Test failure assessment: - acceptance failure is from smoketest `ozonesecure-mr-mapreduce`. I recall witnessing the same failure before but not doing

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #696: HDDS-3056. Allow users to list volumes they have access to, and optionally allow all users to list all volumes

2020-03-25 Thread GitBox
smengcl commented on a change in pull request #696: HDDS-3056. Allow users to list volumes they have access to, and optionally allow all users to list all volumes URL: https://github.com/apache/hadoop-ozone/pull/696#discussion_r398019230 ## File path:

[GitHub] [hadoop-ozone] xiaoyuyao commented on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB.

2020-03-25 Thread GitBox
xiaoyuyao commented on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB. URL: https://github.com/apache/hadoop-ozone/pull/665#issuecomment-603963226 Thanks @elek for the additional info. That helps. I'm just curious on the bottom-right, why rocks DB suddenly had a

[GitHub] [hadoop-ozone] avijayanhwx commented on issue #722: HDDS-3074. Make the configuration of container scrub consistent.

2020-03-25 Thread GitBox
avijayanhwx commented on issue #722: HDDS-3074. Make the configuration of container scrub consistent. URL: https://github.com/apache/hadoop-ozone/pull/722#issuecomment-603974479 LGTM +1 pending CI. This is an automated

[GitHub] [hadoop-ozone] jojochuang commented on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
jojochuang commented on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603983510 You actually don't need to submit a PR. Simply git pull --rebase, and then git push -f to force push the branch. But good to see additional check

[GitHub] [hadoop-ozone] smengcl commented on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl commented on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603995561 > You actually don't need to submit a PR. Simply git pull --rebase, and then git push -f to force push the branch. But good to see additional check

[GitHub] [hadoop-ozone] vivekratnavel commented on issue #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread GitBox
vivekratnavel commented on issue #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata. URL: https://github.com/apache/hadoop-ozone/pull/714#issuecomment-604043200 Acceptance test failure is not related to the change. It fails due to no space available on disk

[GitHub] [hadoop-ozone] smengcl merged pull request #703: HDDS-2945. Implement ofs://: Add robot tests for mkdir.

2020-03-25 Thread GitBox
smengcl merged pull request #703: HDDS-2945. Implement ofs://: Add robot tests for mkdir. URL: https://github.com/apache/hadoop-ozone/pull/703 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop-ozone] sodonnel commented on issue #719: HDDS-3270. Allow safemode listeners to be notified when some precheck rules pass

2020-03-25 Thread GitBox
sodonnel commented on issue #719: HDDS-3270. Allow safemode listeners to be notified when some precheck rules pass URL: https://github.com/apache/hadoop-ozone/pull/719#issuecomment-604029510 Getting intermitted unit test failure on: ``` [ERROR] Errors: [ERROR]

[GitHub] [hadoop-ozone] smengcl edited a comment on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl edited a comment on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603999618 Test failure assessment: - acceptance failure is from smoketest `ozonesecure-mr-mapreduce`. I recall witnessing the same failure before but

[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
fapifta commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r398210290 ## File path: hadoop-ozone/dist/src/shell/hdds/hadoop-functions.sh ## @@ -1541,6 +1541,21 @@

[jira] [Commented] (HDDS-3282) ozone.http.filter.initializers can't be set properly for SPNEGO auth

2020-03-25 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067254#comment-17067254 ] Xiaoyu Yao commented on HDDS-3282: -- {code:java} SCM start failed with exception

[jira] [Created] (HDDS-3282) ozone.http.filter.initializers can't be set properly for SPNEGO auth

2020-03-25 Thread Xiaoyu Yao (Jira)
Xiaoyu Yao created HDDS-3282: Summary: ozone.http.filter.initializers can't be set properly for SPNEGO auth Key: HDDS-3282 URL: https://issues.apache.org/jira/browse/HDDS-3282 Project: Hadoop Distributed

[GitHub] [hadoop-ozone] arp7 commented on issue #643: HDDS-2339. Add OzoneManager to MiniOzoneChaosCluster

2020-03-25 Thread GitBox
arp7 commented on issue #643: HDDS-2339. Add OzoneManager to MiniOzoneChaosCluster URL: https://github.com/apache/hadoop-ozone/pull/643#issuecomment-604061845 Folks - any further comments on this? If not can we get this in?

[jira] [Updated] (HDDS-3273) OM HA: getconf must return all OMs

2020-03-25 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle updated HDDS-3273: -- Summary: OM HA: getconf must return all OMs (was: getconf must return all OMs) > OM HA:

[jira] [Assigned] (HDDS-2976) Recon throws error while trying to get snapshot in secure environment

2020-03-25 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-2976: - Assignee: Siddharth Wagle (was: Shweta) > Recon throws error while trying to get

[jira] [Created] (HDDS-3281) Add timeouts to all robot tests

2020-03-25 Thread Hanisha Koneru (Jira)
Hanisha Koneru created HDDS-3281: Summary: Add timeouts to all robot tests Key: HDDS-3281 URL: https://issues.apache.org/jira/browse/HDDS-3281 Project: Hadoop Distributed Data Store Issue

[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #718: HDDS-3224. Enforce volume and bucket name rule at create time.

2020-03-25 Thread GitBox
bharatviswa504 commented on issue #718: HDDS-3224. Enforce volume and bucket name rule at create time. URL: https://github.com/apache/hadoop-ozone/pull/718#issuecomment-604138297 > > I have a few comments. > > Thanks, Bharat for your comments. I will make those changes requested but

[jira] [Updated] (HDDS-3164) Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Ratnavel Subramanian updated HDDS-3164: - Description: Add a Recon API endpoint to serve missing containers

[jira] [Created] (HDDS-3280) Ozone BaseHTTPServer should honor ozone.security.enabled config

2020-03-25 Thread Vivek Ratnavel Subramanian (Jira)
Vivek Ratnavel Subramanian created HDDS-3280: Summary: Ozone BaseHTTPServer should honor ozone.security.enabled config Key: HDDS-3280 URL: https://issues.apache.org/jira/browse/HDDS-3280

[GitHub] [hadoop-ozone] swagle commented on issue #718: HDDS-3224. Enforce volume and bucket name rule at create time.

2020-03-25 Thread GitBox
swagle commented on issue #718: HDDS-3224. Enforce volume and bucket name rule at create time. URL: https://github.com/apache/hadoop-ozone/pull/718#issuecomment-604141630 > > > I have a few comments. > > > > > > Thanks, Bharat for your comments. I will make those changes

[GitHub] [hadoop-ozone] hanishakoneru opened a new pull request #723: HDDS-3281. Add timeouts to all robot tests

2020-03-25 Thread GitBox
hanishakoneru opened a new pull request #723: HDDS-3281. Add timeouts to all robot tests URL: https://github.com/apache/hadoop-ozone/pull/723 ## What changes were proposed in this pull request? We have seen in some CI runs that the acceptance test suit is getting cancelled as it

[jira] [Updated] (HDDS-3281) Add timeouts to all robot tests

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3281: - Labels: pull-request-available (was: ) > Add timeouts to all robot tests >

[jira] [Updated] (HDDS-3164) Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated HDDS-3164: Resolution: Fixed Status: Resolved (was: Patch Available) Thank you for the

[GitHub] [hadoop-ozone] avijayanhwx merged pull request #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread GitBox
avijayanhwx merged pull request #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata. URL: https://github.com/apache/hadoop-ozone/pull/714 This is an automated message from the Apache Git

[GitHub] [hadoop-ozone] avijayanhwx commented on issue #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread GitBox
avijayanhwx commented on issue #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata. URL: https://github.com/apache/hadoop-ozone/pull/714#issuecomment-604104783 Thank you for the contribution @vivekratnavel.

[jira] [Updated] (HDDS-3282) ozone.http.filter.initializers can't be set properly for SPNEGO auth

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3282: - Labels: pull-request-available (was: ) > ozone.http.filter.initializers can't be set properly

[GitHub] [hadoop-ozone] xiaoyuyao opened a new pull request #724: HDDS-3282. ozone.http.filter.initializers can't be set properly for S…

2020-03-25 Thread GitBox
xiaoyuyao opened a new pull request #724: HDDS-3282. ozone.http.filter.initializers can't be set properly for S… URL: https://github.com/apache/hadoop-ozone/pull/724 ## What changes were proposed in this pull request? Change to use the Initializer class from hadoop-common while keeping

[GitHub] [hadoop-ozone] fapifta commented on issue #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
fapifta commented on issue #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#issuecomment-604118733 Thank you for addressing my concern, sounds reasonable, and I can accept the current solution. I agree that most of the users

[jira] [Assigned] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang reassigned HDDS-3283: Assignee: runzhiwang > Improve write efficiency by improving RocksDB writeBatch >

[jira] [Created] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
runzhiwang created HDDS-3283: Summary: Improve write efficiency by improving RocksDB writeBatch Key: HDDS-3283 URL: https://issues.apache.org/jira/browse/HDDS-3283 Project: Hadoop Distributed Data Store

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Attachment: screenshot-2.png > Improve write efficiency by creating container in parallel. >

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once

2020-03-25 Thread GitBox
bshashikant commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once URL: https://github.com/apache/hadoop-ozone/pull/709#discussion_r398305798 ## File path:

[jira] [Commented] (HDDS-3254) Datanode memory increase so much

2020-03-25 Thread Shashikant Banerjee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067369#comment-17067369 ] Shashikant Banerjee commented on HDDS-3254: --- [~yjxxtd], after killing the datanode process, do

[jira] [Updated] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3283: - Attachment: screenshot-1.png > Improve write efficiency by improving RocksDB writeBatch >

[jira] [Updated] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3283: - Description: >From the image, you can find the whole write cost 140ms, but RocksDB

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once

2020-03-25 Thread GitBox
runzhiwang commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once URL: https://github.com/apache/hadoop-ozone/pull/709#discussion_r398310403 ## File path:

[jira] [Commented] (HDDS-3254) Datanode memory increase so much

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067371#comment-17067371 ] runzhiwang commented on HDDS-3254: -- [~shashikant] I have stop send request to s3gateway for one day, but

[jira] [Commented] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067287#comment-17067287 ] runzhiwang commented on HDDS-3283: -- I'm working on it. > Improve write efficiency by improving RocksDB

[jira] [Commented] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067325#comment-17067325 ] runzhiwang commented on HDDS-3240: -- I'm working on it > Improve write efficiency by creating container

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower

[jira] [Updated] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3283: - Description: >From the image, you can find the whole write cost 140ms, but RocksDB >writeBatch of two OM

[jira] [Reopened] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang reopened HDDS-3240: -- > Improve write efficiency by creating container in parallel. >

[jira] [Issue Comment Deleted] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Comment: was deleted (was: duplicated with https://jira.apache.org/jira/browse/HDDS-3244) > Improve

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once

2020-03-25 Thread GitBox
bshashikant commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once URL: https://github.com/apache/hadoop-ozone/pull/709#discussion_r398305798 ## File path:

[GitHub] [hadoop-ozone] elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin

2020-03-25 Thread GitBox
elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin URL: https://github.com/apache/hadoop-ozone/pull/687#discussion_r397824173 ## File path: hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/WithScmClient.java

[jira] [Created] (HDDS-3279) Rebase OFS dev branch: adapt changes from master branch commits

2020-03-25 Thread Siyao Meng (Jira)
Siyao Meng created HDDS-3279: Summary: Rebase OFS dev branch: adapt changes from master branch commits Key: HDDS-3279 URL: https://issues.apache.org/jira/browse/HDDS-3279 Project: Hadoop Distributed Data

[GitHub] [hadoop-ozone] elek commented on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB.

2020-03-25 Thread GitBox
elek commented on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB. URL: https://github.com/apache/hadoop-ozone/pull/665#issuecomment-603812461 > I mean do we see better BLOCK_CACHE_DATA_HIT Yes:

[jira] [Created] (HDDS-3278) Acceptance test ozonesecure-mr-mapreduce failing

2020-03-25 Thread Stephen O'Donnell (Jira)
Stephen O'Donnell created HDDS-3278: --- Summary: Acceptance test ozonesecure-mr-mapreduce failing Key: HDDS-3278 URL: https://issues.apache.org/jira/browse/HDDS-3278 Project: Hadoop Distributed Data

[jira] [Resolved] (HDDS-3243) Recon should not have the ability to send Create/Close Container commands to Datanode.

2020-03-25 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan resolved HDDS-3243. - Resolution: Fixed PR merged. > Recon should not have the ability to send Create/Close

[GitHub] [hadoop-ozone] avijayanhwx merged pull request #712: HDDS-3243. Recon should not have the ability to send Create/Close Con…

2020-03-25 Thread GitBox
avijayanhwx merged pull request #712: HDDS-3243. Recon should not have the ability to send Create/Close Con… URL: https://github.com/apache/hadoop-ozone/pull/712 This is an automated message from the Apache Git Service. To

[jira] [Assigned] (HDDS-3178) Add unit tests for OMGetDelegationToken Request and Response

2020-03-25 Thread YiSheng Lien (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YiSheng Lien reassigned HDDS-3178: -- Assignee: YiSheng Lien > Add unit tests for OMGetDelegationToken Request and Response >

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Description: Merge commits on master branch to OFS dev branch: {{git merge master}} Also need to

[GitHub] [hadoop-ozone] avijayanhwx commented on issue #712: HDDS-3243. Recon should not have the ability to send Create/Close Con…

2020-03-25 Thread GitBox
avijayanhwx commented on issue #712: HDDS-3243. Recon should not have the ability to send Create/Close Con… URL: https://github.com/apache/hadoop-ozone/pull/712#issuecomment-603937582 Thanks for the review @vivekratnavel & @bharatviswa504.

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage URL: https://github.com/apache/hadoop-ozone/pull/668#discussion_r397857851 ## File path:

[GitHub] [hadoop-ozone] cxorm commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.

2020-03-25 Thread GitBox
cxorm commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling. URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r397964460 ## File path:

[GitHub] [hadoop-ozone] elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin

2020-03-25 Thread GitBox
elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin URL: https://github.com/apache/hadoop-ozone/pull/687#discussion_r397878003 ## File path: hadoop-hdds/docs/content/tools/SCMCLI.md ## @@ -1,27 +0,0 @@ Review comment:

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Description: Need to manually apply a couple of changes in master branch to OFS classes as well: -

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback URL: https://github.com/apache/hadoop-ozone/pull/678#discussion_r397830892 ## File path:

[jira] [Created] (HDDS-3277) Datanodes do not close pipeline when pipeline directory is deleted.

2020-03-25 Thread Mukul Kumar Singh (Jira)
Mukul Kumar Singh created HDDS-3277: --- Summary: Datanodes do not close pipeline when pipeline directory is deleted. Key: HDDS-3277 URL: https://issues.apache.org/jira/browse/HDDS-3277 Project:

[jira] [Updated] (HDDS-3279) Rebase OFS dev branch: adapt changes from master branch commits

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Description: So far need to apply

[GitHub] [hadoop-ozone] elek edited a comment on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB.

2020-03-25 Thread GitBox
elek edited a comment on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB. URL: https://github.com/apache/hadoop-ozone/pull/665#issuecomment-603812461 > I mean do we see better BLOCK_CACHE_DATA_HIT @xiaoyuyao Good questions, I checked it and saved the

[GitHub] [hadoop-ozone] elek commented on issue #578: HDDS-3053. Decrease the number of the chunk writer threads

2020-03-25 Thread GitBox
elek commented on issue #578: HDDS-3053. Decrease the number of the chunk writer threads URL: https://github.com/apache/hadoop-ozone/pull/578#issuecomment-603816449 > Can we make it a data driven decision? I looked at the linked document. I didn't understand the test methodology. E.g.

[GitHub] [hadoop-ozone] timmylicheng commented on issue #668: HDDS-3139 Pipeline placement should max out pipeline usage

2020-03-25 Thread GitBox
timmylicheng commented on issue #668: HDDS-3139 Pipeline placement should max out pipeline usage URL: https://github.com/apache/hadoop-ozone/pull/668#issuecomment-603840741 Before pipeline max: 13, pipeline actual: 12 Pipeline count on node: 1 Pipeline count on node: 5 Pipeline

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #715: HDDS-3253. Initialize Recon metrics for prometheus at /prom endpoint

2020-03-25 Thread GitBox
avijayanhwx commented on a change in pull request #715: HDDS-3253. Initialize Recon metrics for prometheus at /prom endpoint URL: https://github.com/apache/hadoop-ozone/pull/715#discussion_r397943155 ## File path:

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage URL: https://github.com/apache/hadoop-ozone/pull/668#discussion_r397858137 ## File path:

[GitHub] [hadoop-ozone] elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin

2020-03-25 Thread GitBox
elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin URL: https://github.com/apache/hadoop-ozone/pull/687#discussion_r397875948 ## File path: hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/SafeModeCommands.java ##

[jira] [Updated] (HDDS-3266) Intermittent TestSecureOzoneRpcClient failure due to DEADLINE_EXCEEDED

2020-03-25 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDDS-3266: -- Attachment: org.apache.hadoop.ozone.freon.TestRandomKeyGenerator-output.txt > Intermittent

[jira] [Updated] (HDDS-3266) Intermittent TestSecureOzoneRpcClient failure due to DEADLINE_EXCEEDED

2020-03-25 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDDS-3266: -- Priority: Blocker (was: Major) > Intermittent TestSecureOzoneRpcClient failure due to

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Summary: Rebase OFS branch (was: Rebase OFS dev branch: adapt changes from master branch commits) >

[jira] [Commented] (HDDS-2665) Implement new Ozone Filesystem scheme ofs://

2020-03-25 Thread Sammi Chen (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066412#comment-17066412 ] Sammi Chen commented on HDDS-2665: -- Hi [~smeng], thanks for working on the feature. Need every user to

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once

2020-03-25 Thread GitBox
runzhiwang commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once URL: https://github.com/apache/hadoop-ozone/pull/709#discussion_r397568527 ## File path:

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
runzhiwang commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r397670171 ## File path: hadoop-ozone/dist/src/shell/hdds/hadoop-functions.sh ## @@ -1541,6 +1541,21 @@

[jira] [Commented] (HDDS-2665) Implement new Ozone Filesystem scheme ofs://

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066473#comment-17066473 ] Siyao Meng commented on HDDS-2665: -- Hi [~Sammi], as of the current impl in HDDS-2929, in order to use the

[jira] [Created] (HDDS-3276) Multiple leasemanager timeout exception while running MiniOzoneChaosCluster

2020-03-25 Thread Mukul Kumar Singh (Jira)
Mukul Kumar Singh created HDDS-3276: --- Summary: Multiple leasemanager timeout exception while running MiniOzoneChaosCluster Key: HDDS-3276 URL: https://issues.apache.org/jira/browse/HDDS-3276

[GitHub] [hadoop-ozone] elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r397758480 ## File path: hadoop-ozone/dist/src/shell/hdds/hadoop-functions.sh ## @@ -1541,6 +1541,21 @@ function

[GitHub] [hadoop-ozone] mukul1987 merged pull request #711: HDDS-3250. Create a separate log file for Warnings and Errors in MiniOzoneChaosCluster.

2020-03-25 Thread GitBox
mukul1987 merged pull request #711: HDDS-3250. Create a separate log file for Warnings and Errors in MiniOzoneChaosCluster. URL: https://github.com/apache/hadoop-ozone/pull/711 This is an automated message from the Apache

[jira] [Resolved] (HDDS-3250) Create a separate log file for Warnings and Errors in MiniOzoneChaosCluster

2020-03-25 Thread Mukul Kumar Singh (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh resolved HDDS-3250. - Fix Version/s: 0.6.0 Assignee: Mukul Kumar Singh Resolution: Fixed Thanks

[GitHub] [hadoop-ozone] elek commented on issue #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
elek commented on issue #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#issuecomment-603768420 > It would be better to restrict the regex to something which is more close to GC stuff, like something that matches to GC case

[GitHub] [hadoop-ozone] timmylicheng opened a new pull request #720: HDDS-3185 Construct a standalone ratis server for SCM.

2020-03-25 Thread GitBox
timmylicheng opened a new pull request #720: HDDS-3185 Construct a standalone ratis server for SCM. URL: https://github.com/apache/hadoop-ozone/pull/720 ## What changes were proposed in this pull request? Standalone SCM RatisServer (Please fill in changes proposed in this fix)

[GitHub] [hadoop-ozone] elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r397784329 ## File path: hadoop-ozone/dist/src/test/shell/gc_opts.bats ## @@ -0,0 +1,41 @@ +#!/usr/bin/env bash

[jira] [Updated] (HDDS-3185) Standalone SCM RatisServer

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3185: - Labels: pull-request-available (was: ) > Standalone SCM RatisServer > --

[GitHub] [hadoop-ozone] elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r397785874 ## File path: hadoop-ozone/dist/src/shell/hdds/hadoop-functions.sh ## @@ -1541,6 +1541,21 @@ function

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
runzhiwang commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r397793264 ## File path: hadoop-ozone/dist/src/shell/hdds/hadoop-functions.sh ## @@ -1541,6 +1541,21 @@

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback URL: https://github.com/apache/hadoop-ozone/pull/678#discussion_r397795529 ## File path:

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback URL: https://github.com/apache/hadoop-ozone/pull/678#discussion_r397795809 ## File path:

  1   2   >